Multi-core Wimp

as soon as it calls Wimp_SendMessage, obviously it gets switched

No. It doesn’t switch tasks. Wimp_SendMessage puts the message on a queue.

sorry I rephrased, wrong use of the term switch, I meant frozen.

P.S. the mentioning of a task getting “frozen” even if just for the time it takes to put a message in the queue is to clarify that there is no asynchronous execution in that particular path. No other reasons.

Maybe I’m being picky, but again it’s important to understand what’s going on in the WindowManager…

Of course it’s not being picky, that’s why I started this conversation.

So, Wimp_StartTask swaps out the calling task, creates a new Slot/process, runs the new program in it, which either runs to completion, or calls Wimp_Poll (according to https://www.riscosopen.org/wiki/documentation/show/Wimp_StartTask at least), before control returns to the caller. I’d describe that as swapping out the caller, but I think I see your point.

My favourite aspect of RISC OS is that it responds to its user (of which there is only one at a time). If I request something that requires a program takes over the whole computer, that’s my decision. If I realise I’ve made a mistake, I can hit escape and the system will return to its previous state, just as responsive as before. There’s no minutes-long toppling over, where the system just won’t accept there’s something wrong until it can’t cope and kills something I wanted kept.

Filters, and their handlers sound like a bloody nightmare, but I expect they’re manageable. Where’s the documentation on them?

@ Simon

one thing that may be useful to experiment with (or at least that’s what I have tried to experiment with), to create a new Module OS Messages and work it out until you get a new one that can reasonably well exchange messages across multiple cores and respect all the requirement of the old Wimp Messaging.

At that point it should be “easier” to replace the Wimp messaging infrastructure with one that can deliver messages across multiple cores and has no issues with the asynchronous nature of multiple cores.

To experiment with this on RISC OS, you can try Jeffrey SMP library (at least this is what I have started to play with).

New SWI should be presented as “OS_SendMessage” etc. and ensure that when broadcasting a message all cores can access the queue to fetch a message that may be required by an hypothetical task they are running.

Just my 0.5c

@Paolo Getting people enthused is not in my skills set. No worries, I’m just here to learn.

BTW some useful links:

Wimp sources:
https://gitlab.riscosopen.org/RiscOS/Sources/Desktop/Wimp

Filter:
https://gitlab.riscosopen.org/RiscOS/Sources/Desktop/Filter

Also a lot of messaging routines (IIRC) should be located between Wimp07.s and Wimp09.s.

Getting people enthused is not in my skills set.

I beg to differ. You’re doing fine in that department.

Filters, and their handlers sound like a bloody nightmare, but I expect they’re manageable. Where’s the documentation on them?

As with all of these questions… the Programmers Reference Manuals document these things. They’ve not changed significantly since the RO 3 manuals were written.

New SWI should be presented as “OS_SendMessage” etc. and ensure that when broadcasting a message all cores can access the queue to fetch a message that may be required by an hypothetical task they are running.

Yeah, just hack some more on to the kernel. Don’t bother actually doing any proper design and actually address the structural problems… just throw another kernel system call in for a wimp concept… that’ll solve it… Just NO.

I’m pretty sure I’m wasting my time saying it again, but hacking stuff into the system without actually dealing with the fundamental issues for the purposes of the Wimp is entirely the wrong way to go about solving the problem.

I’m just a noob here, and I’m sure this will be regarded as heresy…

It’s been discussed many times before and it’s wrong for many reasons. There are many overly simplistic and unhelpful things about saying ‘just use a linux kernel instead of the RISC OS one’. As an engineer you’ll know the danger of “come on, all you have to do is…” :-D

[in the following discussion, I’m using ‘Linux’ as the non-RISC OS kernel because you cited that explicitly, but in most case this is completely interchangeable with other system kernels]

Firstly, the kernel behaviour for RISC OS and Linux are utterly alien. They don’t share a memory model, they don’t share a system call interface, or even system call semantics, they don’t have the same expectations of the system interfaces (IRQs, processor modes, physical memory, etc), they don’t have the same expectations about what the kernel is and does even.

• RISC OS expects the application to be in control of the system (ie current execution state)
• Linux expects the Kernel to be in control of the system.

• RISC OS expects the application to have full control of the system (ie SVC mode, even down to manipulating page tables)
• Linux grants the application USR mode access and that’s it.

• RISC OS has (and expects there to be) no process model.
• Linux has a process model baked into its very essence.

• RISC OS requires graphics and character semantics akin to a BBC.
• Linux has completely different graphics primitives and character primitives.

• RISC OS allows extensions of any of its interfaces in different manners available to an application.
• Linux explicitly disallows any user space component affecting any other.

And that’s just some very simplistic examples of where there is disparity before you get into any details.

All of those things can be changed, with a varying degree of compatibility, but in order to do that you need to understand the differences and what must be retained and what doesn’t need to be. Whatever you do needs to at least retain the sembalance of a RISC OS system, otherwise you’re just creating a new OS. And the tools that you used on the earlier versions of RISC OS need to work otherwise what’s the point?

Similarly, if you start ripping out parts of the Kernel because they don’t match the shape of the system that you’re trying to build, you break its assumptions and find yourself in equally hot water. If you break the assumption that user space cannot write to Kernel space, how much do you break Linux? (and by that I’m meaning USR mode access to resources allocated for SVC space use). Because that’s one of the areas that RISC OS allows for. Yeah, it’s one that it’d be great to remove, but RISC OS user space applications can promote themselves to SVC mode instantly (though they shouldn’t for sanity’s sake), and of course modeuls execute in SVC mode, so applications essentially have proxy access to the system anyhow.

Basically the only things that you would ever take forward from a separate kernel would probably be the memory management, and system startup. Everything else is so utterly alien to RISC OS that it’s not going to be useful in the Kernel. That’s not to say that other components aren’t useful – they just don’t need to be (and should not be, in the RISC OS philosophy) in the Kernel.

It’s already been sort-of done here:

Your reference to ‘sort of done’ is entirely incorrect (unless I’ve misunderstood that project) – that’s a port of the RISC OS emulation to run closer to Linux. It doesn’t use the Linux Kernel as part of its base – it’s a user space application like any other, which just happens to expose the linux system call interface to RISC OS. It is not ‘using the Linux kernel’. It just happens to run on Linux and provide access to the APIs.

It’s still an admirable project, but it’s not in any way an example of running on a Linux kernel. If that were the case, then RISC OS Pyromaniac would be an example of running on a Linux, OSX or Windows kernel – and that’s … no.

But probably going forward you could assign each RISC OS app to it’s own thread

Nope… because each ‘application’ expects to be in its own world, but threads share address space. Plus, what you think of as an application is just a mirage provided by the Window Manager. Remember that you could implement a Wimp task with something like Pseudo-code:

SWI Wimp_Initialise
WHILE alive
  SWI Wimp_Poll, args TO reason, block
  IF ! system("/<MYApp$Dir>.HandleReason {reason} &{hex(block)} &{hex(my_application_state)}") THEN alive=FALSE
ENDWHILE
SWI Wimp_CloseDown

Yeah, that’s an entirely invented syntax, but the principle of having subprocesses which call Wimp SWIs is one which would work. You’d need a slightly custom handler for your language runtime (placing it in high memory, and expecting your subprocesses to run in the space left), with all your workspace set aside so that it’s not affected by the loaded application, but it’s entirely possible. This gives you two specific things that POSIX kernels (and almost any other) will not let you do – subprocess accessing the memory of its parent (because that’s how you’re passing the blocks of state – yes you can use shared memory but that’s actively set up, not implicitly available… although now I think about it a custom use of vfork and loading data into memory could work to make this happen – and the ‘subprocess’ wouldn’t actually be a subprocess, it could be a fork with shared memory access but oh dear god no… stop, it’s so awful – you just shouldn’t try to wedge those semantics together), and system calls affecting the parent environment (because calls to Wimp_CreateWindow etc would have effects that persist into the parent.

- how long you guys have been discussing how to move RISC OS forward in the areas listed about (years & decades)

I feel that this is largely because a lot of armchair developers with little experience of actually architecting and putting together such a system who do a lot of the talking. (I’m not directing that comment at you, but at the target of your comment)

Speaking as somebody who had been working towards making RISC OS more accessible and manageable for many years, and who has written a distinct version of RISC OS, I feel that those who have the ability to do make such changes and the inclination won’t actually be posting about it, so you won’t see them. They’ll be getting on and doing it, pulling in those competent people that they see can help. The corollary of that is that those posting here have either not got the ability, or not got the inclination.

Yeah, just hack some more on to the kernel. Don’t bother actually doing any proper design and actually address the structural problems… just throw another kernel system call in for a wimp concept… that’ll solve it… Just NO.

Charles, I am always for redesigning the OS correctly. My post was to help Simon exploring the OS that has a lot of quirkiness.

I am not suggesting to add an official new set of SWI. We had plenty of chats (also live), so you should know well my personal position on the matter. If one has to repeat absolutely everything every-time he/she post something we’ll end up writing books just to say hello.

Modelling software and testing is the best way to understand how a redesigned OS has to interact with the applications. Otherwise your very point makes NO sense, why? Because then let’s bloody use Linux and run the old Apps in an emulator <—- read it, so NO just NO let’s stop this way of approaching others posts.

I’m pretty sure I’m wasting my time saying it again, but hacking stuff into the system without actually dealing with the fundamental issues for the purposes of the Wimp is entirely the wrong way to go about solving the problem.

Who said “hacking stuff”? He needs to model his changes, so that they interact with an Application as the Application expects them to do. Are you suggesting to go bling and design an entire new OS basing his understanding on the PRM that is not fully correct and not even up-to-date??? If so yes you are wasting time (in general)

RISC OS expects the application to be in control of the system (ie current execution state)
Linux expects the Kernel to be in control of the system.

It’s late, and I’m tired, but isn’t that exactly what you were advocating? http://gerph.org/riscos/ramble/futuredirection-resilience.html#Applicationspaceandenvironments

So, there’s documentation on how to create a filter, pages 301 to 310. Except *Filters suggests there are filters for display redraws, as well as polls.

Pre-filters can change the event mask, and nothing else.
Post-filters can change what the result of Wimp_Poll is, although I can’t see how they can add messages.

Either way, they’re effectively part of the task they’re associated with, running with that task swapped into memory.

and by that I’m meaning USR mode access to resources allocated for SVC space use

It’s a lot more innate than it seems, given that modules frequently hand back pointers to bits of themselves to applications, so user mode programs would at least need read access to the module area…

It’s been discussed many times before and it’s wrong for many reasons. There are many overly simplistic and unhelpful things about saying ‘just use a linux kernel instead of the RISC OS one’. As an engineer you’ll know the danger of “come on, all you have to do is…” :-D

As an engineer I know the value of saying “come on, all you have to do is…”
- I’ve often had engineers both h/w & s/w say to me ‘it can’t be done’ …. before eventually realising, it not only can be done, but it’s actually a good idea… (^_^)
- mainly s/w engineers ;)

Of course, the underlying OS choice could be any number OSs
- Linux if just an obvious possibility
- that offers all of what RISC OS is currently missing…

User Mode vs Kernel Mode
- yes, it’s a good thing that the app and OS are protected from one another
- and that the app thinks it has the whole machine to itself…

About RISC on Linux
https://www.riscosopen.org/forum/forums/9/topics/9064
‘Ported to run on linux’
- not an end in itself – as it’s just runs as a user app
- my point was to take it stage further – eliminating the parts of Linux not needed to support RISC OS…

Yeah, that’s an entirely invented syntax, but the principle of having subprocesses which call Wimp SWIs is one which would work. You’d need a slightly custom handler for your language runtime (placing it in high memory, and expecting your subprocesses to run in the space left), with all your workspace set aside so that it’s not affected by the loaded application, but it’s entirely possible. This gives you two specific things that POSIX kernels (and almost any other) will not let you do – subprocess accessing the memory of its parent (because that’s how you’re passing the blocks of state – yes you can use shared memory but that’s actively set up, not implicitly available… although now I think about it a custom use of vfork and loading data into memory could work to make this happen – and the ‘subprocess’ wouldn’t actually be a subprocess, it could be a fork with shared memory access but oh dear god no… stop, it’s so awful – you just shouldn’t try to wedge those semantics together), and system calls affecting the parent environment (because calls to Wimp_CreateWindow etc would have effects that persist into the parent.

- I would need to learn a lot more about RISC OS to be able to respond :o)
- or possibly understand Timothy Baldwin RISC OS on Linux port more…

Sorry got to go – need to take my daughter swimming…

It’s late, and I’m tired, but isn’t that exactly what you were advocating? http://gerph.org/riscos/ramble/futuredirection-resilience.html#Applicationspaceandenvironments

Seems to me that that Gerph bloke agrees with most of what I was trying to say
;)

@ Martin

- my point was to take it stage further – eliminating the parts of Linux not needed to support RISC OS…

Not sure what you mean with that. Linux Kernel (as an example) is configurable and that is as far as you can go with removing portions of it without breaking it. None of the configuration options would help in reimplementing RISC OS Kernel using Linux Kernel.

Just an example:

Linux Kernel can access user-space (using get_user/put_user/copy_from_user/copy_to_user etc), but a user-space process on Linux cannot access the Kernel space (unless using some drivers based memory remapping technique which per-se is a “pro-active” activity between a driver and a user-space process).
While on RISC OS a user-space process CAN access the kernel. For instance, whenever a SWI returns to your program a pointer to some data, you can access it directly without doing anything special.

There is nothing you can remove from the Linux Kernel to allow it to operate as RISC OS would, unless you remove the entire concept of memory protection¹, which to my understanding, is something we want to add to RISC OS, right?

So, to stay practical, there is nothing preventing ROOL (or Simon in this discussion) from implementing correct parameters copy in the RISC OS kernel like it is now (so the RISC OS Kernel will start avoiding manipulating user-space data directly). This, contrary of that superficial comment Charles made in response to my comment to Simon, would be very useful already, why? Because it would help implementing user-space threading that can be modelled after Jeffrey SMP. Right now, if a user process uses threads and the kernel modify directly some user-data bad things can happen.

However, the problem is the other way around, to implement a mechanism that avoid a user-space process from accessing the modules returned values directly in the module memory will break the user-space process².

Unless someone design some mechanism to use some form of memory remapping from a module to user-space, in which case the module will have to be re-written to use such new API, there is no way to use old applications in a memory-protected RISC OS.

Now, let’s forget for a moment if such an approach is correct, good or whatever, but certainly it doesn’t exists, so it’s not a matter of removing pieces from Linux Kernel. It’s more a matter of re-designing RISC OS correctly from a user-space stand point (at the very very minimal) and then decide which best approach could be used to implement such design.

And, just to be clear, here I am barely scratching the surface.

¹ And then also fix all Linux Kernel to work without memory protection in place.

² Which is why, it maybe time to start accepting a break with compatibility with the past and run old applications in an emulated environment.

Martin, to quote, leave a blank line. Then type bq. and then a space, and then paste in what you wish to quote. Leave a blank line between that and your reply.

So, the following:

bq. This is a quote!

Will do this:

This is a quote!

- mainly s/w engineers ;)

Happens quite a bit. I tend to find “it can’t be done” as a euphemism for “I can’t be arsed to do it”.

- yes, it’s a good thing that the app and OS are protected from one another

Remember the heritage of RISC OS is the BBC MOS written for a 6502-based machine that was one of the better equipped home computers in the home computers boom of the 1980s.

There’s one of the original developers around, so I’d like to pose a question – when creating RISC OS it of the earlier Arthur, did you ever imagine you’d be using something that contains a lot of that same code thirty four years later on a machine with a gigahertz speed processor and a gigabyte of memory, all on a dinky board that can be powered from a battery?

[to put this into context, the original ARM2 clocked 8MHz, and in the beginning Acorn was offering the A305 with half a *mega*byte of memory onboard (I don’t know if any were actually made, as it doesn’t leave much for programs), machines that had many of them supplied with only double density (800K) floppy discs because harddiscs were hugely expensive]

maybe time to start accepting a break with compatibility with the past and run old applications in an emulated environment.

This is the same issue that was raised with the discussion of the 64 bit incarnation for newer processors; and it implies essentially writing two operating systems from the ground up. The shiny new and improved RISC OS, and whatever system is going to be used to provide a proper emulation of the original API (which could be interesting on non-ARM32 targets).

Why? Because the greatest OS ever written is not so great if there’s no software for it.

The problem? Developers. Or, rather, a lack of them.

So I suspect we’ll be patching bits onto the current OS until we run out of 32 bit capable processors. Then somebody clever will make an emulator of the 32 bit world, and we’ll be able to patch bits on to the current OS for a while longer…

Martin, to quote, leave a blank line. Then type bq. and then a space, and then paste in what you wish to quote. Leave a blank line between that and your reply.

Thanks – updated – I did try but didn’t realise the quote needed to start on the same line …
;)

This is the same issue that was raised with the discussion of the 64 bit incarnation for newer processors;

It’s actually worst lol I am already using most of the software I need in an emulator, pretty much all the MIDI software, some of which doesn’t even run of RISC OS 4.39, so it has to be pure RISC OS 3.7x up to 4.02 (maybe with some patch!).

The problem? Developers. Or, rather, a lack of them.

I agree and I would add, also dead developers, or disappeared ones. I would like to maintain all the old MIDI Applications for RISC OS if the original authors can read this message, are willing to donate their old sources, are still alive, they haven’t sold they sources to companies that have no interest in RISC OS or have lost them (there is this too!)

There’s one of the original developers around, so I’d like to pose a question – when creating RISC OS it of the earlier Arthur, did you ever imagine you’d be using something that contains a lot of that same code thirty four years later on a machine with a gigahertz speed processor and a gigabyte of memory, all on a dinky board that can be powered from a battery?

Just one gigabyte ;-)

No, of course not, to all of those. We had dual-MEMC running before I left (end of 1988), giving a whopping 8MB, so the 64MB virtual address space was still huuuuge. Thought that a 512MB hard drive would be beyond the lifetime of RISC OS systems…

Oh yes, A305s were made during Arthur times, but I think most were upgraded quite quickly once users wanted to run something beyond “Hello World”, or to run any commercial software. We discovered down the pub one day from higher manglement than they even wanted to produce a 256KB version, blissfully unaware that 1MB was our target.

As soon as ‘task switching’ was grafted into the Window Manager it was clear that was the wrong place for it.

And yet…

Time pressure. You would imagine that you’d leave the OS developers to that task wouldn’t you? Would you expect them to have to write all the applications for the two Apps discs as well? ;-)

Cooperative multi-tasking with N processors should give the WIMP the task to keep track of up-to N concurrent tasks running, right?
Since SWIs are used this would imply that each processor can handle its own SWI calls and within those SWI communicate with the WIMP.
Having access to 64 bit capable systems it would make sense to run the WIMP and kernel in a 64 bit environment and manage 32 bit application spaces from there. Thus allowing new 64 bit application spaces alongside 32 bit. (possibly even JIT when 32 bit is not supported like on the Apple M1)
Direct access to hardware from applications has always hampered the progress of OS development with RISC OS. However it is clear that we can run RISC OS as a Linux application on both 32 bit and 64 bit ARM Linux platforms. In simple terms, the abstraction layer that handles the hardware access appears to work really well and could be used as a base for implicit sandboxing of applications. If all WIMP calls would in turn communicate with a ‘central’ WIMP it might even work..

Direct access to hardware from applications has always hampered the progress of OS development with RISC OS.

Out of interest, asides from fiddling with the screen buffer (singletasking, no Wimp app should be doing this), what is this hardware access? There are system calls for serial, parallel, IIC, and so on. Things shouldn’t really still be trying to prod the underlying hardware.