Thinking ahead: Supporting multicore CPUs

aaaaa

aaaaa

which is what make it different from other OS.

I do believe that I mentioned that “RISC OS is not Linux” once or twice.
This is why I keep pointing you at the PRMs. It is a hefty read, but the most detailed. Please, please, understand that the core of RISC OS dates from the late mid eighties. It predates anything you know of Windows. It predates Linux. The original base, Arthur, just predates X11. And while Unix and C and plenty of UIs (Amiga Workbench 1, Atari TOS, etc) existed prior, I would imagine that RISC OS predates pretty much every “framework” you know.

I point this out only to suggest that you discard any preconceived notions of how you think these things should work and look more closely at how RISC OS actually does work. It is quite different, but necessary to understand at your level, the PMT level, if you aren’t too be bitten by “Filecore in use” and other problems.

If you look back at my previous posts, you’ll see me referring to the Wimp as the “task switcher” several times. I was dropping hints. ;-)

To answer a question that was posed earlier – how do you pre-empt a program when it is in a SWI call?
Simple.
You don’t.

To interrupt at that level would imply using CallAfter or CallEvery which you will quickly discover carries numerous restrictions on what you can do when you are called. In my code, I tend to use CallAfter to schedule a Callback, which RISC OS will do when it is “unthreaded” and control returns to the user mode application (the exit of a SWI, etc). It is safer to do many things then. For a start, filesystem access will work…

How did Wimp2 cope with issues of global state changing? Like a program that sets the CSD and expects it to remain set while it does some file operations? (Not that I’m advocating that method, but it is possible when you know the system won’t rip control away from under your feet.)

aaaaa

aaaaa

aaaaa

Like a program that sets the CSD and expects it to remain set while it does some file operations?

The same way as usual – it doesn’t; and the application should be using <Meh$Dir> style paths and leaving the CSD alone – for good reason.

aaaaa

Started up the PC as replying longer on a mobile phone is… not the most pleasant (or sane) idea!

Well the thing is, believe me, nobody sensible would want to develop multi threaded application on a system that has a buggy implementation.

I’m not sure we can call RISC OS “multi threaded”. Not in the sense that you mean.

Would you make something like apache2 or mysql being able to work efficiently with a system like that ?

You don’t.
The last time I looked (around 2003ish), apache was a large monolithic style chunk of command line code that the system loads and executes as a sort of “background task”. It has no UI, it exists outside of a GUI environment, it will run quite happily with no GUI present.

There are two ways to port something to an “alien” platform.

• Bend the platform to match the software. One could argue that the likes of CygWin are an attempt to munge the Windows API into something that will allow Linux-style code to be ported with minimum changes. In an ideal world, the original source would have #ifdef sections to allow for the code to be built on different systems, but that doesn’t always happen.

Or:

• Bend the software to match the platform. This is the correct way to port, but without being able to feed the changes back to the original source it does risk leaving you in the cold when a new version is produced and the modifications would need to be applied afresh. On the other hand, it does permit one to use the proper platform specific behaviour. Do you know that the correct way to request help from a program in DOS (Windows command line) is to suffix the command with /? – but you try that with a lot of programs ported from the Linux world. mp4box /? will reply Error creating file /?: Requested URL is not valid or cannot be found.

Or, more specifically, to your apache example. You would not expect to build the code and have it run unless you were happy to single-task that one application only.
I would imagine that apache contains a “wait for event, deal with it” loop at its heart. So you initialise as a Wimp task during the init/preamble, you poll in this event loop, and you throw in some extra polling for parts of the code that run as a loop but may “take some time”. Also bolt on some minimal Wimp API so you have an iconbar with an Info and Quit menu, plus can respond to system messages like PreQuit and Quit.
With a little bit of luck, you might have something enormous capable of serving static web pages. But with many quirks and potential problems – the ReWriteEngine will need to be disabled, our filesystem is nothing like Linux. The way of detecting what “type” a file is will need to be completely rewritten, you can’t look at file suffix. Indeed, the whole path handling doesn’t even look remotely like POSIX. And that’s just for starters.

Application shouldn’t have to know about all those restriction and to manipulate interrupt themselves.

Depends what you are trying to do. I have written a Teletext interface program with a (really) basic script interpreter in BBC BASIC for RISC OS and VB5 for Windows. In both cases, the software “talks” to the hardware at register level using an intermediary that handles the actual IIC communications. Under RISC OS this is either the system IIC bus, or my PtrIIC module (to fake IIC using the printer port). Under Windows, it is a DLL called PtrIIC. Under NT-kernel Windows it needs GIVEIO.SYS to allow the DLL to access the hardware as I never sussed how to build a hardware driver using lcc or OpenWatcom.
At any rate, the application itself does not directly talk to the hardware. Neither does it have any truck messing with interrupts or scheduling or things like that. It runs either in “Viewer” mode where it behaves like Teletext on a TV, or it runs in “Script” mode where it processes script commands. It cannot do both at the same time for the basic technical reason that the generic teletext adaptor that my hardware works with only has 2KiB SRAM provided, and as such it is only capable of one page fetch at a time. If I had 8KiB onboard, the EuroCCT chip can do four simultaneous fetches which could lead to all sorts of cool things (like prefetching in the background, script and viewer at the same time); but back then single SRAM chips were expensive, so…

But the thing is you can’t just know when an application is calling a routine or not. It’s all machine code in the memory that can be preempted to and from at any single instruction.

What, you mean when a SWI is being called? Yes, it is all machine code in memory, yet strangely enough RISC OS knows. Hint: Try looking at the “superstack”. But, as I said, there is a world of difference between CallAfter/CallEvery and Callback. The first two are more or less guaranteed to happen after a defined interval of 2cs or more, but you WILL be interrupting whatever is happening including OS routines. You cannot sensibly change things here without a fair degree of pain. Callback, on the other hand, happens at an unspecified time (when RISC OS is “free”) which is usually as soon as the SWI completes – which can be mere nanoseconds later, or not. At this time, a great many more things may be done.

But what you say is basically that all the threading restriction specific to riscos should be handled by user code at application level, using some kind of tricks, where all this stuff should be done by the system and it should provide a clear api with a clear framework to say how to handle those case.

No, I’m saying that unless you want a lot of pain, your pre-emptive system will need to integrate within the Wimp itself. Or possibly the kernel if you think you are going to be capable of running something like apache.

Then… As the “service provider”, it will fall upon your PMT code to:
a, Handle all of the necessary contexts to allow things to work. (or puss out like Acorn did on many things ;-) )
and:
b, Be entirely backwardly compatible and invisible to applications written beforehand (like how the Wimp can still run a RISC OS 2 aware program)

Finally, back to this:

Application shouldn’t have to know about all those restriction

No, applications should not have to know about numerous restrictions. But the programmer should. To give a good example, there are numerous things that work by the OS “calling your code”. OS_Claim, the various types of callbacks… It is imperative that an application either remove such claims prior to a poll, or place them in module space that is not paged out. The kernel can help you – OS_DelinkApplication and OS_RelinkApplication, but it is up to you to do this. The Wimp does not.

To see what it does and doesn’t do, look at https://www.riscosopen.org/viewer/view/castle/RiscOS/Sources/Desktop/Wimp/s/Wimp03?rev=4.6#l28 and https://www.riscosopen.org/viewer/view/castle/RiscOS/Sources/Desktop/Wimp/s/Wimp03?rev=4.6#l79 and https://www.riscosopen.org/viewer/view/castle/RiscOS/Sources/Desktop/Wimp/s/Wimp03?rev=4.6#l287.

aaaaa

Like a program that sets the CSD and expects it to remain set while it does some file operations?

The same way as usual – it doesn’t; and the application should be using <Meh$Dir> style paths and leaving the CSD alone – for good reason.

Like I said, I’m not advocating it – clearly it’s not a good idea. But the fact remains, if the program flow can be interrupted by preemption, all global state has to be saved and restored by the system, doesn’t it? I’m sure there are many more subtle examples than the CSD, though I can’t think of them off the top of my head. I have something in the back of my mind telling me that Wimp_OpenTemplate does something silly in this respect, but I can’t remember what it is.

EDIT: Ah, right here in this very thread: https://www.riscosopen.org/forum/forums/5/topics/406?page=2#posts-7078, along with a lot of other very interesting information.

<alien language alert>

It use CurrentX/CurrentY properties and print method that are not supported into vb.net.
i’m not sure how to convert this to vb.net,i don’t know much about visual basic, do you know how this function could be made to work in vb.net easily ?

Google came up with this:

CurrentX, CurrentY properties
GDI+ doesn’t support the notion of the “current point” and the methods that draw lines and polygons must always specify the starting point and the ending point. For this reason, no VB.NET control supports the CurrentX and CurrentY properties.

and:

Print method
VB.NET doesn’t support this method. Instead, you should use the DrawString method of the System.Drawing.Graphics object. This latter method only works with pixels, so you should convert from the current ScaleMode settings.

It looks like something like this may work (if not, blame Google, not me!):

Dim GFX as graphics = Picture1.CreateGraphics()
GFX.DrawString("Your text here", New Font("Corpus New", 12), New SolidBrush(Color.Black), 50, 200)

Do you really need to do all of that for each line to be drawn? How retrograde.

Or i should keep it in vb6 ?

Uh… Maybe. ;-)

</alien>

But the fact remains, if the program flow can be interrupted by preemption, all global state has to be saved and restored by the system, doesn’t it? I’m sure there are many more subtle examples than the CSD, though I can’t think of them off the top of my head. I have something in the back of my mind telling me that Wimp_OpenTemplate does something silly in this respect, but I can’t remember what it is.

There’s no link between Wimp_OpenTemplate, Wimp_LoadTemplate and Wimp_CloseTemplate. If a switch occurs between calls to Wimp_OpenTemplate and Wimp_CloseTemplate, by the time the task gets control back again there’s a possibility that another task has called Wimp_OpenTemplate or Wimp_CloseTemplate – meaning that remaining attempts to use Wimp_LoadTemplate could be referencing the wrong Templates file, or none at all…

Thanks, Steve. Yes, that’s it. In fact, h0bby1, you might want to go back and re-read the start of this thread. It’s very interesting!

I’m sure there are many more subtle examples than the CSD, though I can’t think of them off the top of my head.

Anything with state.

Using OS_GBPB to read a list of directory entries? [it will generally work, but what happens if the directory changes (or is deleted) during the read?]
Any called-back code [if you delink your OS_Claim claims, you won’t crash the OS but you will miss events if they happen when you are paged out]
Any Font call that isn’t explicitly passed a font handle.
Anything that depends upon CSD [either explicitly or through being lame and using Obey$Dir].

That said, RISC OS does try to mitigate the problems; you can’t do much if files change while data is being read on them.
You can shift called back code to the RMA and have it set a value in memory that is checked by the Wimp as a pollword so you won’t miss events.
Most Font SWIs can be passed a font handle so nothing needs to rely on any concept of “the current font”.
You can use All$Dir and the like to not rely upon the CSD remaining the same.

This is why I pointed out the age of RISC OS to h0bby1. If there were big glaring show stoppers, we’ll have crashed into them long before now. One does not need to rely upon the CSD being anything at all. There are other methods that are more appropriate.
Obviously one cannot protect against everything that could happen, we just take appropriate steps to protect against that which is likely to happen and – well – if a user wants to try to Copy files from a directory structure that they are also Deleting… good luck to ’em. ;-)

Interesting reading about the Template Open/Read/Close issue. The Wimp doesn’t even pass back a file handle to give to ReadTemplate. Having said that, in the context of an application, loading templates is supposed to be an “atomic” task performed prior to the first poll. So it was likely never envisaged that a task switch may occur while loading templates.

aaaaa

Obviously one cannot protect against everything that could happen, we just take appropriate steps to protect against that which is likely to happen and – well – if a user wants to try to Copy files from a directory structure that they are also Deleting… good luck to ’em. ;-)

Well, yes. However, the thought had crossed my mind a few times that with the advent of tools like Locate 2 which can run multiple searches on a filing system at once, accidentally deleting something that you’re also searching isn’t an impossible scenario. Especially if you consider networked drives.

I suspect that Windows might have similar issues, of course: I’m not aware that searching a drive locks its contents. However it might be relevant that the .NET APIs that I’m aware of all return a directory’s contents in one atomic lump – unlike OS_HeebieJeebie, which allows you to collect the contents of a directory by remembering some opaque state, performing multiple polls and trusting that things don’t change until you’re done¹.

¹ Those polls could be some time apart, too. Locate 2 grabs something like 1k (off the top of my head) of data from OS_GBPB at a time, and stores it on a stack to allow it to recurse down through directory structures. If your search’s root directory is too large to fit into that 1k, and the first objects returned are all directories containing lots of further objects (and so on), it could be quite a while before OS_GBPB is called for the root directory again with the state it returned right back at the start. By that point, who knows what’s happened (or, indeed, what other tasks might have done on null events – consider things like Transient).

In writing applications that scan directories and alter their contents – say deleting empty subdirectories or renaming files – one has to remember not to make the changes until the scan has completed. Most scripting languages provide functions to scan directories, and they usually do this by returning a table of the directory’s contents (e.g. readdir in Perl or listdir in Python). Such functions consume storage in proportion to the number of items in the directory. By contrast, RiscLua provides a for-loop structure with no such penalty:

for file_name, file_type in riscos.dir (dirpath) do ..... end

aaaaa

.. have a way to know which wimp task is the current task when they(?) are executed?

SWI TaskManager_EnumerateTasks (see PRM 3-315) tells you the currently active tasks. As for the current task, that is presumably the one calling this SWI, is it not? ;)

Currently, does the system ( fileSwitch/fileCore for ex) have a way to know which wimp task is the current task when they are executed ?

You want Wimp Items 5 for the current task handle.

aaaaa

It need an Id to differenciate it from other task that could also be using it in the same time :)

“Same time”? One processor, one task at a time, PMT or CMT