Multi-core Wimp

We don’t need another hero…

Maybe we do?

Maybe we do?

Is this the point where you dive into a telephone box and re-appear with your undies on the outside? :)

Is this the point where you dive into a telephone box and re-appear with your undies on the outside? :)

Isn’t that just taking your trousers off? :)

BTW, No room in my local phone box for that now. Full of bloody books!

Is this the point where you dive into a telephone box and re-appear with your undies on the outside? :)

Gosh, I don’t think I meant me! I hope I didn’t. It sounds out of character.

Anyway, piss-taking out of the way, I was thinking: If I put a veneer between the applications and the WindowManager module that caught all the Wimp SWIS, could I avoid having the Wimp try to manage tasks/the memory by having each task appear to the manager as a RM task (intercepting Service_Memory), and doing the TaskSlot memory management by myself?

The veneer would be a RM handling the Wimp_* calls, and call the legacy WindowManager module code in a controlled way: one private word shared between cores, called only by one core at a time, and each task keeping a list of visible rectangles per window up until the point that an application affects the display layout.

In the meantime, I’ll try to work out what ModHandReason_FindEndOfROM_ModuleChain is for.

The veneer would be a RM handling the Wimp_* calls, and call the legacy WindowManager module code in a controlled way:

That sounds rather like a suggestion that came up before about one method of peeling the Task handling out of the Window Manager.
I think it got a thumbs up then

If I put a veneer between the applications and the WindowManager module that caught all the Wimp SWIS, could I avoid having the Wimp try to manage tasks/the memory by having each task appear to the manager as a RM task (intercepting Service_Memory), and doing the TaskSlot memory management by myself?

This question has two folds:

1) Can you trick an application to believe it’s talking with the Wimp while it is not? The answer is Yes and you can do that in multiple ways.
I would not recommend ways that requires to recompile an Application or re-assemble it.
Also remember interpreted languages may have slightly different needs (for instance BBC BASIC is a RM module itself), Lua is an ELF that now also uses SharedLibraries, so SOModule and riscos/so is the library that access the WIMP calls. Python similar etc.

2) Can you use your own Module to trick apps etc.? Yes, intercept the relative vectors and use them to execute your own WIMP. That should work for everything. If your module is capable also to switch between kernel space and user-space you can start making the apps behave correctly and access return blocks in user-space instead of kernel (there is not such a distinction in RISC OS memory, so by describing it as distinct I just mean copy or remap the data in an area that is not directly your module assigned memory)

To do that for 1 and 2 is going to be a bit of work, but I think it can be done. If you want a secure OS however you really need to make your module capable of working in both contexts (a bit like the SharedCLibrary), and this will probably need some performance thoughts.

The veneer would be a RM handling the Wimp_* calls, and call the legacy WindowManager module code in a controlled way:

I have nothing about Simon’s work, but I cannot not notice the irony in this thread with statements made by some like “we should make the system secure and make it so that the use cannot hack into it” and here we have “developer: should I intercept the Wimp calls? forum users: Yes that’s a good way to do it”.

could I avoid having the Wimp try to manage tasks/the memory by having each task appear to the manager as a RM task (intercepting Service_Memory), and doing the TaskSlot memory management by myself?

I’m not sure.

Firstly, Service_Memory will only be checked during Wimp_Initialise if the CAO is not in application space. And if Service_Memory is unclaimed during that check the Wimp will immediately free the current AMB node.

Secondly, sometime the Wimp will try to peek at blocks you’ve given it (for example here; it looks like if you build the Wimp with appropriate debugging set up you’ll get a trace of when this happens). Normally it would try to page in your task to inspect the block, but in this case it’s totally unexpected because it doesn’t have an associated AMB for your task, but the block address is in the app slot; I don’t know what it would do in that case.

I don’t know what it would do in that case.

Crash.

I cannot not notice the irony in this thread

Yes, it does rather seem like hack upon hack to drag things kicking and screaming into something that vaguely resembles the twenty first century.

That said, the alternative is rewriting the entire window manager and task handling.
Any takers?

rewriting the entire window manager

Such that it mostly behaves in the same largely undocumented ways that applications have come to rely on ;-)

in the same largely undocumented ways that applications have come to rely on ;-)

Shhh! nemo might be lurking! ;)

Such that it mostly behaves in the same largely undocumented ways that applications have come to rely on ;-)

Haaa we have a plan! :)

Such that it mostly behaves in the same largely undocumented ways that applications have come to rely on ;-)

I think you’d need an NCOS shim to deal with that to ensure Paolo’s plan works.

Thanks, folks, for the responses. What I’m looking at is having the mapping in and out of tasks and all memory to be managed by new, C, code. If I can avoid modifying the legacy code at all, that’s my first choice (because I’m likely to break it). I’ve found, with the right magical invocation (replicating legacy memory structures, and a few magic numbers), I can use legacy SWI implementations at will. For example, I have OS_Heap protected by a lock that means only one core at a time can use it. (I can only hope that interrupt handlers don’t, at the moment.)

I’ll try to clarify my thinking:

• The new kernel will manage memory. Most memory will be shared between all the cores, but kernel workspace will be unique to each core, as will the mapped task slot (0×8000-whatever).

• Existing modules will be initialised separately on each core.

• New modules can be marked as multiprocessing-aware, which will mean that the kernel will ensure that all cores wait until the first has initialised that module, and all those cores will share the same private word. (This is implemented already.)

• Such a module will wrap the legacy Window Manager, intercepting its SWIs etc.

There will be two points of view: the application, and the Wimp.

• The Wimp will think that all but one application is sitting in Wimp_Poll, waiting for their Poll Word to be set (which will happen whenever the task needs the Wimp for anything but an update/redraw.

• The application will be calling Wimp_Poll, as normal, and being fed Idle responses, until Wimp messages start flying, or they want to move a window.

When those applications try to redraw or update the content of their windows, they will be provided with rectangles that are visible. When some task has modified the window stack in any way, the veneer module for each task will pretend to redraw all of their windows, and quickly store the returned rectangles for later use.
Independent redraw/update loops will be terminated by any request that’s likely to move windows.

Firstly, Service_Memory will only be checked during Wimp_Initialise if the CAO is not in application space. And if Service_Memory is unclaimed during that check the Wimp will immediately free the current AMB node.

OK, I think I can make that work. The real Wimp_Initialise will only be called by the module, and it can claim Service_Memory, so that should stop the Wimp from trying to do anything with the current task memory.

I cannot not notice the irony in this thread

Don’t I know it! The thing is, without apps, there’s no point. Using the Window Manager to provide services to applications (there’s nothing to say that those applications have to be running in 32-bit memory or code, just communicating with the veneer module) seems like a fairly safe way to go. Existing RO applications will be sandboxed in non-secure 32-bit modes, and new applications (and device drivers) in a 64-bit secure memory model to move forward. Being able to have some concurrency with older applications might be a nice bonus.

I can only hope that interrupt handlers don’t…

Of course they do ;-) There’s code in OS_Heap to complete any foreground operation’s request before proceeding with a request from IRQ mode.

Existing modules will be initialised separately on each core

So we get four RAM discs? And all other DA users? Who gets to own other physical resources like the Ethernet?

Of course they do ;-) There’s code in OS_Heap to complete any foreground operation’s request before proceeding with a request from IRQ mode.

Of course they do. :(

There will be two possible situations:

1. The interrupt occurs on the core that’s performing the OS_Heap

1. The interrupt occurs on another core, in which case the lock will block the interrupt handler until the other core has completed it’s call to OS_Heap

The lock would then have to record which core has claimed it, so that the IRQ OS_Heap call can proceed regardless in the first case, and simply wait for the other core to get on with it, in the second. Does the code at least check that both calls are on the same heap? Could there be a separate IRQ heap? (There can be a separate lock on each heap.)

So we get four RAM discs? And all other DA users? Who gets to own other physical resources like the Ethernet?

No, we make the RamFS module (all filesystem modules) multiprocessor aware. The RMA is shared between all the cores already. Devices can be owned by whichever core happens to initialise them, as long as they can route the appropriate interrupt to that core. TBD.

The RMA is shared between all the cores already.

I hope, unless some sort of flag is set, only one instance of a module will be active at any given time.

No, because each core needs a different context. So, each core should have access to an independent FontManager.

Think of it as four computers running RISC OS and cooperating to share resources.

Will a Wimp application ever be moved to a different core? It will expect to have the same font handles.

I would be tempted to progress this by having a whitelist of modules which have been shown to be MP-safe, and only initialise those on each core.

I have nothing about Simon’s work, but I cannot not notice the irony in this thread with statements made by some like “we should make the system secure and make it so that the use cannot hack into it” and here we have “developer: should I intercept the Wimp calls? forum users: Yes that’s a good way to do it”.

RISC OS is a single user enthusiast OS with no security, and I think (hope) all users are aware of that. We are not in the position in having average Joe users that need protecting from themselves by running dodgy software from the internet. There are so many useful hooks in RISC OS we can use to do innovative stuff, which have to be blocked on other OS due to security exploits. I don’t think we want to lose the ability do pretty much anything in RISC OS, as it’s what makes it fun and still useful. Where we do need security is for the OS to protect itself from buggy applications, so the whole machine doesn’t fall over.

OK, I think I can make that work. The real Wimp_Initialise will only be called by the module, and it can claim Service_Memory, so that should stop the Wimp from trying to do anything with the current task memory.

If you do that, the Wimp will keep the AMB node attached to the task and manipulate it as usual.

1. CAO outside app slot and Service_Memory unclaimed: AMB freed on Wimp_Initialise
2. CAO outside app slot and Service_Memory claimed: AMB kept by Wimp and paged in and out when polling/peeking

We still don’t know what happens if you pass it a pointer to a block in your app slot when you’re doing 1). You either need to do an experiment, or investigate the Task macro in the Wimp source to work out what it will try and do.

We had some previous discussion about this where Jon Abbott was trying do things with AMB’s.

No, because each core needs a different context.

And what happens then, given the RMA is shared, if different core versions of a module store state within itself. Or will each core run a physically different version of the module? Could be interesting if it responds to events or service calls…

access to an independent FontManager

Hmm, so each core will have its own VDU context then? I can see this getting interesting. ;-)

by having a whitelist of modules which have been shown to be MP-safe

Why a whitelist? There’s a flags word with thirty one unused bits…

that need protecting from themselves by running dodgy software from the internet

Yet.