Multi-core Wimp

I’m working on a replacement kernel for RISC OS written in C, and designed with multi-core processors in mind. I’d like to put down a few ideas for how Wimp programs could work in that environment. Clearly, you can’t have a separate Wimp module running for each core, so I would propose a multi-processing-aware module on all but one cores and a low-overhead Wimp Task to interface to the existing Wimp module. Or “just” modify the WindowManager.

Firstly, has this been discussed somewhere already?

Is this an exhaustive list of calls that potentially swap out the current task?

• Wimp_Poll(Idle)
• Wimp_SendMessage
• OS_Exit

Message passing (e.g. Data Load, etc.).
That needs all other tasks to be performing a Wimp_Poll(Idle) in order to receive the message.
But what if two independently running tasks call Wimp_SendMessage at the same time?

I suggest a new call, Wimp_Release (Wimp_NoMessages), to indicate that the task isn’t going to send a message before its next call to Wimp_Poll, so other tasks are free to continue? The task’s programmer can choose when that’s appropriate. Not calling it results in normal Wimp (cooperative multitasking) behaviour, but if all tasks (but one) have called it, they’re free to run independently. When the task that wants to send a message calls Wimp_SendMessage_, nothing happens until all running tasks have called WimpPoll(Idle) (or have exited).

Redraw/update window loops.

Wimp_Poll returns with a Redraw_Window_Request message from the Wimp, if the task calls Wimp_Release before Wimp_RedrawWindow, other tasks will be provided rectangles to redraw/update concurrently. Ditto for Wimp_UpdateWindow loop.

Assuming all the programs are designed to play nice with the Wimp, synchronisation for redraws etc., shouldn’t take long, right?

Is this an exhaustive list of calls that potentially swap out the current task?

No.
As has been raised in the problems of the exploding menus, the Wimp has a tendency to muck around with tasks “in the background”.

But what if two independently running tasks call Wimp_SendMessage at the same time?

That’s not your big problem. Your big problem is how you’re going to cope with Wimp_MessageRecorded.

The system relies upon a simple round-robin behaviour, and if the message passes all tasks with no claimers, it’s bounced back to the caller as unclaimed. When you start having asynchronous tasks, it gets more difficult to know how to handle this.

other tasks will be provided rectangles to redraw/update concurrently.

Yes, it’s probably more logical to have redraws on a per task basis, so one task not in a position to redraw doesn’t cock everything up.

to indicate that the task isn’t going to send a message before its next call to Wimp_Poll

You don’t always necessarily know.

That said, it may be that working out how to handle recorded messages might, by extension, suggest how to handle messages in general.

Total can of worms. Graphics system state? WrchV handlers per task?

the Wimp has a tendency to muck around with tasks “in the background”

Could you give me an example? Like are tasks pre-empted and TaskWindow tasks given some time, or is it just when a Wimp SWI is called?

Your big problem is how you’re going to cope with Wimp_MessageRecorded

I think my suggested approach could work for that; if a task has been scheduled and tries to send a message, nothing happens until all the asynchronous tasks have called Poll. At that point, the standard behaviour occurs, passing the message around the tasks and potentially back to the originator. Every task behaves like a RO2 task for the duration.

> to indicate that the task isn’t going to send a message before its next call to Wimp_Poll
You don’t always necessarily know.

Don’t you? As the programmer, you’ll probably know you’re never going to send a message in response to, say, a Scroll_Request, not until after the next Poll.

bq. to indicate that the task isn’t going to send a message before its next call to Wimp_Poll

You don’t always necessarily know.

Sometime you know just when you are about to return to the Wimp.

Total can of worms. Graphics system state? WrchV handlers per task?

One of each per core. Only one task running on a core at a time. I have currently got each core running a copy of DrawMod and rendering independently. I should probably look at fonts, asap.

Sometime you know just when you are about to return to the Wimp.

Couldn’t you just remember that and do a quick Poll, first? This behaviour would only be for multiprocessing-aware tasks.

I think my suggested approach could work for that; if a task has been scheduled and tries to send a message, nothing happens until all the asynchronous tasks have called Poll. At that point, the standard behaviour occurs, passing the message around the tasks and potentially back to the originator. Every task behaves like a RO2 task for the duration.

A classic case being dragging data from one application to another. The recipient wouldn’t be able to ‘Release’ until it had finished loading the data, which could be lengthy, otherwise the sender would attempt to delete the file it had just sent.

So if a module installs a WrchV handler, how are calls to this serialised?

Could you give me an example?

I very specifically did.

Well, I’m sorry, I’ve never heard of “exploding menus”. Does Wimp_CreateWindow swap out the task, then?

All sorts of batshit crazy stuff happens in the Wimp, see https://www.riscosopen.org/tracker/tickets/519 for instance.

An ‘exploding menu’ bug has just been fixed, see https://www.riscosopen.org/forum/forums/4/topics/16112#posts-126704

So if a module installs a WrchV handler, how are calls to this serialised?

One per core. Separate instances of the module will run on each core.

A classic case being dragging data from one application to another.

Wouldn’t that simply be a reversion to the current behaviour, for the duration of the transfer?

So if a module installs a WrchV handler, how are calls to this serialised?

One per core. Separate instances of the module will run on each core.

So when any module is loaded it’s instantiated separately on all cores?

A classic case being dragging data from one application to another.

Wouldn’t that simply be a reversion to the current behaviour, for the duration of the transfer?

Yes, but that’s just the kind of operation users would hope other tasks would be able to continue during!

Thanks for the links, I’ll have a read. Worst case, I’ll replace the window manager and task window code, I’ve got some code around here that did something similar way back when, and the task window (not, if I understand correctly, being entitled to manipulate the screen etc.) could be implemented differently. What matters is what it does, not necessarily how it does it.

The current task window could usefully be replaced by GraphTask

So when a module is loaded it’s instantiated separately on all cores?

As it stands, yes, but that’s optional.

Yes, but that’s just the kind of operation users would hope other tasks would be able to continue during!

Personally, I’m OK with things pausing when I’ve asked for something to happen. Can the sending task do nothing during the writing of the file, no Polls?

Personally, I’m OK with things pausing when I’ve asked for something to happen. Can the sending task do nothing during the writing of the file, no Polls?

If the data is being transferred using RAM transfer, no, as the receiver will repeatedly ack back with a recorded message. If it’s going via a file, then the sender will receive a DataSaveAck during a poll and usually can take as long as it wants to compose the file before sending it onwards with another recorded DataLoad.

And then the data needs to be consumed by the receiver before that issues DataLoadAck.

Perhaps the task mangler could just drag the sender of a directed message and its intended receiver onto the same core?

I’m probably wrong, but that sounds like it would allow tasks to keep updating the display while the file is being prepared (and RAM transfer shouldn’t take long at all, there’s a lot of bandwidth).

Incidentally, I expect the Window Manager would have one central workspace shared between cores with simple locks to protect it.

I’m probably wrong, but that sounds like it would allow tasks to keep updating the display while the file is being prepared (and RAM transfer shouldn’t take long at all, there’s a lot of bandwidth).

That’s probably true. But the pair still need to block until the receiving task has processed all the data subsequent to its transfer.

[Edit: Amusingly the Filer sends a DataLoad to the target via User_Message_Recorded (and has done so since 1988). This would lead to the above scenario blocking the Filer as well as the DataLoad recipient on a drag-to-icon. I can’t think of any reason why the Filer would care about not getting a DataLoadAck in that instance, other than most of the Data Transfer Protocol messages became recorded and I thought that it seemed best to use recorded messages more consistently. Might be interesting to spin a Filer that just sent it via User_Message.]

I’m working on a replacement kernel for RISC OS written in C, and designed with multi-core processors in mind. I’d like to put down a few ideas for how Wimp programs could work in that environment. Clearly, you can’t have a separate Wimp module running for each core, so I would propose a multi-processing-aware module on all but one cores and a low-overhead Wimp Task to interface to the existing Wimp module. Or “just” modify the WindowManager.

Stop right there.

The WindowManager is a client of the system. It should not be the core of task switching. Any attempt to continue the current system with the WindowManager being a central part of the system is just layering more hacks upon hacks upon hacks. The correct way to deal with this is to create a proper process system, and a mechanism by which those processes communicate with one another and modules. One thing that you have to remember is that the Window Manager isn’t a singular part of the OS. It’s just A part of the OS, and anything that it can do should be doable by any other part of the OS.

Is this an exhaustive list of calls that potentially swap out the current task?

It should be expected that there will be multiple processes running simultaneously. It should be expected that some of those are tasks. It should be expected that some of those are not tasks, and merely provide services for other processes.

It should be expected that any process be able to be prempted at any given time – this is already true for taskwindows, and should be expected in other places.

DO NOT TRY TO BOLT MULTI-CORE BEHAVIOUR INTO THE CURRENT SYSTEM.

It is would be terrible archtectural decision, and perpetuate the flaws of the past.

The question of whether you’re swapping out the current task should be entirely irrelevant. User mode processes should be able to be present in memory or not; executing or not. When running, they may request data from the system through SWI calls (for example OS_File, Socker_Recv, or Wimp_Poll) and they may process that data at their leisure.

Graphics contexts may or may not exist per process – decision to be made (my intent was that graphics context could exist per process or be shared, or non-existance). The existing behaviour of the Wimp controlling tasks is an antique and just as fragile.

I’ve spent a lot of time thinking on this over the years, and the only sane way to deal with the problem of multi-core is to strip back the problem to its roots. Introduce processes. Make the OS control the system (rather than the current model which is that the application controls the system). Allow processes to interact with one another. Provide the existing ‘environment’ within the process model in a way that does not interfere with other processes. Move wrchv and rdchv to a stream system through the (equivalent of) environment handlers, so that processes may communicate with one another without the use of vector handlers. Disallow vectors in paged space (application space). Replace the graphics model with a more extensible mechanism that allows contexts to be used per process.

The result of this is a process system which can mostly retain the behaviour of RISC OS Classic, but which is able to be maintained by the OS – it will be more resilient and more debugable because processes will be able to be terminated independant of their own misbehaviour (ie they don’t control the system, so they can be killed), and because it will be possible to pause and inspect processes. The subprocess (child invocation/fork) behaviour becomes much less involved, and memory management stops being a user/developer problem but becomes an OS problem – where it is more appropriate to be centrally dealt with.

Where is the Window Manager in this? Well, it ‘just’ falls out of things. You have to rewrite a chunk of it because it is no longer in control, but that just means that instead of managing the task switching, the queues that are being processed become yield to the system – Wimp_SendMessage puts the message on a queue and blocks (as required) and Wimp_Poll polls messages off the queue. There are problems such as those that you’re asking about but they’re WAAAAY down the line at the end of the work to revamp the system.

What about redraw? Graphics context per process takes care of that. Each task has a view on to the system through its graphics context. Now you might choose to make that a single context is present which the Wimp manages and you mutex out access from other tasks when they’re not meant to be writing to the screen, but that’s really perpetuating the old behaviour. Better, you make the processes have their own contexts which they write to, and then they inform the window manager that they’re ready to render. Those contexts might be independant or they might be (effectively) windows on to a real context. Switching to a context based behaviour means that compositing on the desktop becomes possible.

And where is TaskWindow? Well, it’s pretty much gone. TaskWindow essentially becomes a process which has stream input and output environment handlers, which are attached to a subprocess that you’re running. It takes data from those streams and posts them as wimp messages to the task that’s providing the display. BUT, it doesn’t even need to exist. The task that’s displaying a ‘taskwindow’ could just as easily be doing that work itself – handling the stream input from its children.

If you built the right behaviour in from the ground up, by starting with the principle that the OS controls the system in order that there be a way to manage multiple core execution and system resilience, then you have to build a different process model, and you have to replace the VDU/Input IO system, and you have to bring the graphics system into the purview of the process (or make it a resource that it can access – it cannot be central). This was the direction that Select was going, and the manner in which it would have been addressed.

So many problems become easier, and the system becomes much more functional and resilient when you start to architect it in a better way.

There are some notes on this from my earlier rambles:

http://gerph.org/riscos/ramble/futuredirection-resilience.html#Applicationspaceandenvironments

TL;DR: Multi-core is NOT something that you should bolt on to current RISC OS. You need to lay the foundation in the Kernel, in the process model, in the manner in which things communicate. And you need to do that first.

(of course, if you are at the tail end of redesigning RISC OS to have a process model and handle these problems, then that’s great – but honestly, if you’re at that point, you wouldn’t need to ask these questions because the answers would be clear)

101%. As soon as ‘task switching’ was grafted into the Window Manager it was clear that was the wrong place for it.

Stop right there.

God has spoken. :-)

Multi-core is NOT something that you should bolt on to current RISC OS.

It is not ideal, but I get the impression from this thread that it’s an attempt to modernise things without making it incompatible (or as compatible as possible) to existing tasks.
Sure, it’s easy to argue that bolting such features on to the current method is horrible hack upon horrible hack (uh, ‘cos it is…), but what we are looking at is a situation akin to ideas for a 64 bit version of RISC OS, namely if it doesn’t run known applications, what will it run?

In a way, our history is the hangman’s noose.

it was clear that was the wrong place for it.

And yet… :-p

> But what if two independently running tasks call Wimp_SendMessage at the same time?

That’s actually an easy one with a well understood solution. Messages are put on a queue which is protected by a mutex, semaphore or some other barrier. Thus any task attempting to push to the queue while another task is already doing so stalls until the queue becomes free and the message can be added.

But see above regarding the WIMP being the wrong place to handle message passing and task switching. Tasks exist (at least in theory) independent of a GUI. A task can be attached to a console or to nothing at all.

What about redraw? Graphics context per process takes care of that.

Yes, printing would use seperate contexts so that they don’t need to intercept all kind of vectors which prevents printing in the background.

Same for redirection to a sprite so that it doesn’t cause side effects (like resetting output to the first screen bank on exit).