Multi-core Wimp

versions of a module store state within itself

They should be taken out and shot. They aren’t ROM-able are they? And if third-party, you have no idea, so don’t whitelist for multi-core instantiations.

Could be interesting if it responds to events or service calls…

Oh there are so many interesting ways to fail!

Why a whitelist? There’s a flags word with thirty one unused bits…

I think Simon’s trying to get going by using unmodified stuff. Putting bits in modules might not be a good idea when they are subsequently found to NOT be MP-safe.

No, they aren’t Roman like ¹, but I would imagine there are plenty of third party modules that do such a thing.

The idea of putting a bit into the flags should only be done when a specific module has been demonstrated as safe, of course.

There’s a bit of a chicken and egg here in that some sort of whitelist will be needed in order to know which modules to allow (and see if it works)…

Oh there are so many interesting ways to fail!

Oh, I’m just thinking of how many things the PRM notes as not being reentrant.
One needs to take a lot of care in event and service call handlers (otherwise FileCore sulks mightily), so… yeah. We ought to do a whip round to buy a pillow to insert between head and desk.

¹ Swipe-type does not like the word “ROMable”, figures I meant “Roman like”. Okay, if you say so……

There’s a bit of a chicken and egg here in that some sort of whitelist will be needed in order to know which modules to allow (and see if it works)…

If knocking up a replacement kernel in C, changing a whitelist for testing is easy-peasy! Quicker than running !Builder probably.

Adding a mutex on OS_Heap is just the tip of the iceberg. Any data structures in memory that is shared across cores will need locks. And defined lock hierarchies.

And what happens then, given the RMA is shared, if different core versions of a module store state within itself. Or will each core run a physically different version of the module? Could be interesting if it responds to events or service calls…

I’m getting a horrible feeling that the answer to this question will start “yes, but…”. Aren’t modules only supposed to store state via their private word? The ROM modules obviously can’t write to their code area.

Hmm, so each core will have its own VDU context then? I can see this getting interesting. ;-)

Yes, I think so.

Will a Wimp application ever be moved to a different core? It will expect to have the same font handles.

In that case, they probably won’t be allowed to move! A problem to be solved when it becomes apparent that it’s really a problem, I think.

Aren’t modules only supposed to store state via their private word?

There’s “supposed to” and “works for me” ;-) I have never done anything perverse like that, honest…

In that case, they probably won’t be allowed to move! A problem to be solved when it becomes apparent that it’s really a problem, I think.

Would be a pity to get dumped onto the same core as a resource hog then!

File handles need to be global – consider the Desktop Save protocol.

Thinking of which, Wimp Broadcast messages really means Serially-Cast to all tasks: you don’t want four cores trying to write out their Desktop Save command sequences simultaneously. Or to have the same file loaded into four different editors when double-clicked upon.

If you do that, the Wimp will keep the AMB node attached to the task and manipulate it as usual.

My problem is that I don’t know what “as usual” is, hence this thread!

Assume, for a minute, that, as far as the Window Manager is concerned the CAO is always outside app slot and Service_Memory claimed; will the Wimp try to page anything in or out if there aren’t any tasks running in a task window or from a Wimp Slot? I presume that’s the situation when there are only module tasks providing the Desktop.

My idea is to copy the poll block out of the application’s memory and into the RMA, at which point I can swap out that application and replace it with another background or multi-processing Wimp application.

Thanks for the link, I’ll check it out.

File handles need to be global – consider the Desktop Save protocol.

They would be. Like I said, all file systems would have to have a MP-aware version or veneer.

Thinking of which, Wimp Broadcast messages really means Serially-Cast to all tasks: you don’t want four cores trying to write out their Desktop Save command sequences simultaneously. Or to have the same file loaded into four different editors when double-clicked upon.

Hence the module tasks approach; the Wimp thinks it’s in charge of all the Wimp tasks, but really there are multiple applications spinning through their poll loops, waiting for their poll words to be set and being given the last known and still valid rectangles to redraw or update.

When one of those applications does something to potentially affect other tasks, like opening a window or sending a message, the veneer will get the Wimp to choose it as the current application by setting the veneer’s poll word and waiting to be recognised, at which point the action is passed on to the Window Manager proper.

Something like that, anyway. It’s still a bit hand-wavey, hence this thread.

There’s “supposed to” and “works for me”

Modules written by such sick individuals could either be limited to a single instance on a single core, or load a separate copy of the code for each core.

I’ll worry about getting relatively well-behaved things working first!

My idea is to copy the poll block out of the application’s memory and into the RMA, at which point I can swap out that application and replace it with another background or multi-processing Wimp application.

Wimp_Poll isn’t the only Wimp SWI that takes a pointer to a block and expects the application to keep the block there. For example looking at Wimp_CreateMenu in the PRM:

If R1 is a window handle, the Wimp will open that window as the menu otherwise the menu structure must remain intact as long as the tree is open. The Wimp does not take a copy, but uses it directly.

So you’d need a wrapper for Wimp_CreateMenu as well to copy that block to the RMA. But then might an application modify its block without you knowing? And you’d want to know of course when you’re allowed to discard the block ideally.

I’m not sure the PRM is always explicit about whether or not the Wimp takes a copy of a block for each SWI. Maybe someone can give a list?

Assume, for a minute, that, as far as the Window Manager is concerned the CAO is always outside app slot and Service_Memory claimed; will the Wimp try to page anything in or out if there aren’t any tasks running in a task window or from a Wimp Slot? I presume that’s the situation when there are only module tasks providing the Desktop.

• If you intercept all pointers to Wimp SWI’s and change them to point to RMA, then as you say that is the existing module task situation, and so it won’t page anything for that task, because there’s no AMB node associated with it to page in or out.
• If you don’t intercept all pointers, and a Wimp SWI gets passed by your application (say &8000) … that’s the thing we don’t have an ansswer to yet, and inspecting the Task macro might help.

(The Task macro is what the Wimp uses to ‘peek’ at your app slot when your application doesn’t have control through Wimp_Poll.)

Modules written by such sick individuals could

It’s not “sick”, it’s simply a recommendation (because the module handler can free the block pointed to by the private word when killing the module).

The problem comes when you have modules that need to dynamically allocate extra memory, especially that which isn’t known until runtime. The base block, pointed to by the private word, may hold pointers to the other claims, but since there’s no tagging whatsoever in the module area, they can easily become orphaned.

Hmm, I’ve not looked (CMHG isn’t open), but how do C modules use the private word?

Also, it may be that a module that needs a larger claim sets up a dynamic area. What happens if multiple versions of the module try to set up the same dynamic area?

Finally, one of my modules maintains a structure on disc, with the indexes held in the RMA (under control of the module). To have the indexes and the file get out of sync with each other would pretty much make it blow up.
The module is capable to detecting bogus index data, tossing it, and recreating it. But this is at startup when doing an initial scan of the data. Different instances on different cores with different ideas of what the file should contain, it would probably go horribly wrong.

Not just that, but the module opens a socket and responds to connections. Thus, multiple versions of the same module simply wouldn’t make sense.

I can imagine a fair few modules are similar.

Or to have the same file loaded into four different editors when double-clicked upon.

<spits tea across the room>

That would actually be kind of funny. It’s mine! No, it’s mine! I was first!

the Wimp thinks it’s in charge of all the Wimp tasks

The problem is, the Wimp does a fair amount of futzing around with paging in the background, unbeknownst to the app and not documented anywhere. The aforementioned exploding menus issue is an example. Calling Wimp_TransferBlock (can be used when dragging a file directly between apps) does done task swap magic in order to work.

I have personal experience – a million years ago I had a little app that sat on an event and tidily unlinked and relinked itself around calls to Wimp Poll. The horrible crashiness that ensued made it quite clear that Wimp_Poll was far from the only thing that switches memory around in the shadows.

I think a problem we’re going to run into is that the PRM (original, HTML, and Wiki) document how to use the system, not how it actually works internally.

So things like this may well crash into a myriad of weird little edge cases that aren’t described anywhere and will only be resolved by figuring out what the code does.

You know, like the difference between when the BASIC guide describes, and what the interpreter actually accepts. ;)

Hmm, I’ve not looked (CMHG isn’t open), but how do C modules use the private word?

It points to a block of RMA (claimed during module init) that serves as storage for the module’s static variables (including any ‘hidden’ ones from SharedCLibrary), plus a few extra words for making the module relocatable. The block is freed during module final.

But then might an application modify its block without you knowing?

I would think there would be consequences for changing a menu’s contents without re-making the menu. The application would be changing it without the knowledge of the Wimp as well, wouldn’t it?

And you’d want to know of course when you’re allowed to discard the block ideally.

That’s true. I suppose the worst case is just to leave it there until the next menu is opened, or the task exits.

Indirect icon names as well, I suppose. Perhaps locking the menu-opening task into memory for the duration would work. Perhaps, at some point, I’ll just have to bite the bullet and modify the Wimp. (Maybe starting with a MenuRejected service call?)

(The Task macro is what the Wimp uses to ‘peek’ at your app slot when your application doesn’t have control through Wimp_Poll.)

It’s on my list of things to look into!

I hope this is a silly question and the answer is “no, obviously not”, but can modules receive service calls before they’ve been initialised?

The code says:

; if ChocolateService, we must not add to service chains yet, because we have
; to make sure module’s init entry is called before service entry (command
; and SWI hashing don’t have these worries – they won’t be used yet)

So this implies, no, modules won’t get service calls prior to init.

Oh, good. So Messages is failing to initialise for another reason…

Are FileSwitch, ResourceFS and TerritoryManager initialised beforehand? Some of those early modules have to be started in the right order.

A long time ago I proposed the idea of containerising RISC OS (running multiple single tasking instances) to avoid some of the work of making all the OS and user modules APIs thread (multi-core) safe. However things such as device drivers and filing systems can only have a single instantiation shared by all the OS instances. As there can only be a single Wimp, and to achieve this it would need to be completely re-writing, no amount of bodging around the existing Wimp would give a coherent system.

Are FileSwitch, ResourceFS and TerritoryManager initialised beforehand? Some of those early modules have to be started in the right order.

It’s not that. I stupidly assumed that the return from SWI procedure had changed for all modules, not just the kernel SWIs. Things are progressing.

A long time ago I proposed the idea of containerising RISC OS (running multiple single tasking instances) to avoid some of the work of making all the OS and user modules APIs thread (multi-core) safe. However things such as device drivers and filing systems can only have a single instantiation shared by all the OS instances. As there can only be a single Wimp, and to achieve this it would need to be completely re-writing, no amount of bodging around the existing Wimp would give a coherent system.

That sounds familiar. My approach is to have MP-aware modules wrap modules that produce handles so that they’re consistent, have the MP Window Manager allow applications to poll and update windows without bothering the Wimp unnecessarily, and to split memory management off the Wimp. It probably won’t work, but maybe it will get closer to a usable solution.

It’s not unreasonable to have a single Wimp; there’s a single user, who should be the most important component of the system.