Identifying the active application slot

One of the things I’ve been looking at recently is that ARMEABISupport is using OS_Memory 0 to try to identify the current application slot, a practice that’s a bit flawed.

One of the main places the code calls OS_Memory 0 is from within Wimp pre-poll & post-poll filters, in order to update a “current app” variable that’s in the module’s workspace. So my initial thoughts were to remove the OS_Memory 0 calls and use the Wimp task handle to identify the app instead. If the pre-poll filter is called before the app is paged out, and the post-poll filter is called when the next app is being paged in, it should work fine, even though an application slot may change Wimp task handle several times during its lifetime (the pre-poll filter can be used to update ARMEABISupport’s idea of what the task handle is for the current app).

But, pre-poll & post-poll filters aren’t suitable for this. They’re (mostly) only called by Wimp_Poll, so actions which can cause an app/task switch without calling Wimp_Poll (like Wimp_StartTask) won’t generate a “pre-poll, post-poll” sequence when starting the new app. Also, even for Wimp_Poll, it’s possible to not get a “pre-poll, post-poll” pair; Wimp_Poll can exit to the caller with an error after it’s made the pre-poll call. E.g. if you use a pre-poll filter to mark the app as “suspended” then your code will think the app is suspended even though it’s still running.

Potential solutions I can think of:

1. Document OS_AMBControl 9 as being OK for public use. This returns the ID of the current AMB node (i.e. application slot). However it was only added to the kernel 9 months ago, so isn’t that great a choice for software which wants reasonable backwards-compatibility (which would probably make it unsuitable for ARMEABISupport)
2. Document DomainId as being OK for public use. The Wimp uses this location to store an identifier for the current app slot (and for Wimp tasks, it forms the low 16 bits of their task handle). However DomainId is located in kernel workspace/zero page, so software will have to read its location using OS_ReadSysInfo 6 (or fall back to a hardcoded address on older OS versions. Luckily, I think old OS versions have always stored it in the same location).
3. Add something new. We could use this opportunity to fix some other issues (like the fact there’s no service call when application slots are created/deleted). But depending on what we do it may be difficult for software that wants to be backwards-compatible (add the feature to a new softloadable Wimp?)
4. Rework ARMEABISupport so that it gets told the active app/task/process in all SWI calls (and rework the way it’s using abort environment handlers, so that each handler knows what the active app is, instead of relying on the global “current app” variable). This would fix things for ARMEABISupport, but obviously won’t do much to help the next program that finds itself in a similar situation.

Any thoughts?

actions which can cause an app/task switch without calling Wimp_Poll (like Wimp_StartTask) won’t generate a “pre-poll, post-poll” sequence when starting the new app

This in my opinion is a bug and can cause a crash as it does not give the application the opportunity to tidy up before its paged out. I think it’s gone unnoticed since day 1, as very few apps have used Filters to any extent.

I’d go with option 2 as it’s the most well known about and is already public by virtue of the fact it’s on the Wiki.

I can’t comment on the other options as AMBControl has no documentation available. Personally I’d like to see it made public, fix the bug where it corrupts memory page flags when switching tasks and have a public method to force a task switch to a specific DomainID.

This last point would open up the opportunity for someone to develop PMT using legal methods (aka PixieDust™)

The application will not be polled out. That is until it calls the Wimp_Poll SWI when it will always be polled out and another task polled in. It may be polled out (I suppose) while it asks the OS to do something like save a file but it will have been polled back in again by the time the OS call returns so won’t know it was polled out for a time. It is exactly the same with Wimp_Poll – while that call is executing the application may be polled out but it won’t know as it will have been polled back in again by the time the Wimp_Poll call returns control to the application.

Modules are quite different. They stay ‘polled in’ but don’t occupy application space.

Applications which are also modules are (presumably) more complicated.

If Wimp_Poll is not called by an app that is starting up then it doesn’t matter as all the other applications will not be sentient at the time. Only when the starting application makes its first Wimp_Poll call will it be paged out and the application in question polled in. So a new task will just ‘appear’ between Wimp_Poll calls of an existing application. All other applications will always be tucked away out of application space when a particular application is given control.

That is why applications can only communicate with each other by sending messages – there is only one application running at any one time. Next time it runs it looks for a reply..

So far as an application is concerned it knows which is the active application – it is always itself.

It can only be a module task that might need to know which is the currently paged in application and as soon as it does know, the information will be out of date as that application may have been paged out again by then.

The application will not be polled out.

Wimp_TransferBlock, among other stuff.

Agree with Jon.

The application will not be polled out

Sorry, I meant to say “paged out”

There are several occasions when apps are paged out of memory, without being notified.

There are several occasions when apps are paged out of memory, without being notified.

Well of course they are. Just not running applications. If you page an application back in (it cannot run at any other time) to tell it that it is paged out, then by the time it can receive the information, it is no longer true.

When an application calls Wimp_Poll it expects to be paged out while this call executes and knows it will be paged back in when the call returns. Whether it is actually paged out depends on whether other applications are running at the time but in any case it does not need to know this.

I know EABI is a new calling convention, but I don’t know what the ARMEABISupport module does, or why it wants to know which domain is in use. Could someone please say so that we can see what the problem being addressed is?

I imagine the typical user of domainid is trying to trap vectors for a specific application (a la TaskWindow); is this the case here?

Jon, what do you have in mind here? What tidying are you wanting to do before you’re paged out?

I need to do two things:

1. Take a copy of the AppSpace page table entries. This is only required as AMB currently corrupts page flags when taskswitching
2. Shutdown code outside of AppSpace that vector back into AppSpace

I know EABI is a new calling convention, but I don’t know what the ARMEABISupport module does, or why it wants to know which domain is in use. Could someone please say so that we can see what the problem being addressed is?

ARMEABISupport is responsible for stack and memory management for Unixlib based programs compiled with GCC 8 and above. Rather than a chunked stack, ARMEABISupport allows programs to use flat stacks where memory pages are mapped in on demand (similar to Linux), so if a program only uses 4k of stack, then only 1 page of memory is mapped in. The stacks are stored on a per app basis to improve look up times within the abort handler. Storing them on a per app basis also makes it easier to delete the stacks when the app exits (this includes pthreads stacks as well as the main stack).

I’m also working on a new mmap implementation where the mmap allocations are owned by the app that made them.

This all has to work outside the desktop too, and within TaskWindows, so does the domainid give a unique and valid value outside the desktop and within TaskWindows? Do multiple TaskWindows have different domainids or does the domainid come from the TaskWindow provider?

This will all have to get hell of a lot more complicated if RISC OS is ever to do SMP.

There are several occasions when apps are paged out of memory, without being notified.

If there was a service call/vector for when application slots are created, deleted, suspended & resumed then that would make some things a lot easier. E.g. features like VFP, hardware breakpoints/watchpoints, app-local use of hardware performance counters, app-local dynamic areas, etc. are all things that need actions performing when a task switch occurs. At the moment the only reliable way to add new “per-process state” is to add the necessary code directly into the Wimp. Ideally any API we invent for this should also cope with task windows as well (e.g. TaskWindow will probably want to suspend/resume the task manually, to make sure nothing goes wrong while TaskWindow is in its Wimp_Poll loop).

This all has to work outside the desktop too, and within TaskWindows, so does the domainid give a unique and valid value outside the desktop and within TaskWindows?

Yes.

Do multiple TaskWindows have different domainids or does the domainid come from the TaskWindow provider?

They’ll have different domain IDs (since each task window is a different Wimp task)

This will all have to get hell of a lot more complicated if RISC OS is ever to do SMP.

A problem I’m well aware of! :-(

At the moment the only reliable way to add new “per-process state” is to add the necessary code directly into the Wimp

Thus proving that Task switching is currently handled in the wrong place in the OS.

Ideally Any API we invent for this should must also cope with task windows as well

Fixed.

is currently handled in the wrong place in the OS.

That and a dozen other things. ;-)

If there was a service call/vector for when application slots are created, deleted, suspended & resumed then that would make some things a lot easier.

Yes, long overdue.

Should probably also include when slots change size as well. Doesn’t the current implementation (namely, Switcher) work using Wimp Messages?

If there was a service call/vector for when application slots are created, deleted, suspended & resumed then that would make some things a lot easier. E.g. features like VFP, hardware breakpoints/watchpoints, app-local use of hardware performance counters, app-local dynamic areas, etc.

That sounds the right direction, but I think the vector should be called instead when the task is created/destroyed/switched.

App-slots are created (Wimp_StartTask), destroyed (Wimp’s exit handler; Wimp_CloseDown for last task; Wimp_Initialise for appslot-less module task) and switched (Wimp_Poll, Wimp_StarTask). The problem would be when a ‘pure’ module task starts, which destroys the app-slot. It feels to me that all the per-process information you mention should not be thrown away in this case! This indicates that we call the vector in all the above situations except for during Wimp_Initialise.

TaskWindow could make use of this. It wouldn’t care in normal operation, as it knows when it’s going to Wimp_Poll. However, it currentyl checks domainid all the time (in the vectors it traps), due to the fact that the child might have called Wimp_StartTask without it knowing (i.e. the issue Jon raises); it could instead sit on the vector, and keep its state updated.

There is also an obscure issue mentioned in the source docs, where it can lose track of spooling over a Wimp_StartTask. Getting a call through the vector might be used to fix this. Another task window issue is that if a child does Wimp_StartTask it will spew loads of control characters into the task window; I don’t remember why exactly that happens, but putting the CallAVector in the right place in Wimp_StartTask hopefully allows us to fix this.

App-slots are created (Wimp_StartTask), destroyed (Wimp’s exit handler; Wimp_CloseDown for last task; Wimp_Initialise for appslot-less module task) and switched (Wimp_Poll, Wimp_StarTask). The problem would be when a ‘pure’ module task starts, which destroys the app-slot. It feels to me that all the per-process information you mention should not be thrown away in this case! This indicates that we call the vector in all the above situations except for during Wimp_Initialise.

“Pure” module tasks still have an application slot (AMB node), they just set the size to zero bytes.

The slot size is literally the only thing TaskManager looks at when it decides where to list the task – if the initial slot size is zero it puts it in the “module tasks” group, else in the “application tasks” group. (Try dragging the Next slot down to zero bytes, then open a task window)

Doing

SYS "OS_AMBControl",9 TO ,,,n : PRINT n

reports the same as the number of entries I see in “Application Tasks”, and when I quit Free the number doesn’t change.

If I’ve understood right, Wimp_Initialise branches to rationalisememory which (if the app slot is not in use) ends up at deallocateslot calling OS_AMBControl_Deallocate?

Looking at the source docs, I see that the GrowShrink reason code will free the node if you shrink to zero pages anyway.

Ah, I hadn’t looked that deep.

In that case, yes, it’d need to be tied to something like the domain ID, not the application slot. (I’ve avoided the word “task”, since to me that implies “Wimp task”, but we want to track things regardless of whether they currently have a task handle)

If there was a service call/vector for when application slots are created, deleted, suspended & resumed then that would make some things a lot easier

A vector would be the way to go as its time sensitive.

Also take note that the Wimp switches tasks to grab indirected data such as Icons etc. From memory the Macro involved is Task? I think that then calls mapin / maptaskin or some such function.

Essential there’s going to be quite a lot of calls to the vector outside of Poll. Obviously there’s no need to touch the Wimp as you’ve already recoded Wimp08s to use AMB for taskswitching.

Is it possible to make parts of AMBControl public? I need to force task switching by DomainID, but am very reluctant to call anything in the OS that isn’t publicly documented. This would allow me to support 26bit Wimp apps under the ADFFS JIT and hopefully provide a drastic improvement in app performance vs Aemulor.

This would allow me to support 26bit Wimp apps under the ADFFS JIT and hopefully provide a drastic improvement in app performance vs Aemulor.

If you do that and don’t win something in the RISC OS Awards I really, really, really want to see the actual winner.

Also take note that the Wimp switches tasks to grab indirected data such as Icons etc. From memory the Macro involved is Task? I think that then calls mapin / maptaskin or some such function.

Essenial there’s going to be quite a lot of calls to the vector outside of Poll.

As far as I know, those inspections of task memory always happen inside Wimp_Poll? The only other place I found paging out so far is in Wimp_TransferBlock where it can ‘occaisionally’ happen (the procedure being described as ‘potentially dangerous’ !!).

My question then is, when should (in functional terms) the vector chain be called? It seems to me that it should be called once after control leaves the application, and once before control returns to the application.

The simplest way would be to do it blindly on every entry and exit to/from Wimp_Poll/Wimp_StartTask (and maybe TransferBlock), though this would mean that unnecessary work is being done if the Wimp_Poll is actually returning back to the same application without peeking at anybody else. The alternative, more difficult way is to call the vector in Wimp_Poll if control is returning to a different task, or the first time the Task macro is used.

Jeffrey I imagine you will have some insight into when is best for performance counters to be notified?

Is it possible to make parts of AMBControl public? I need to force task switching by DomainID, but am very reluctant to call anything in the OS that isn’t publicly documented. This would allow me to support 26bit Wimp apps under the ADFFS JIT and hopefully provide a drastic improvement in app performance vs Aemulor.

Further 26bit support sounds good, but I’m not clear what your plan is here? Are you going to have multiple 26bit apps run ‘inside’ one single 32bit app, handling the 26bit Wimp_Polls yourself, and so needing to switch between the app-slots manually?

My question then is, when should (in functional terms) the vector chain be called?

The Vector should be called by AMBControl whenever it task switches. The Wimp doesn’t need to be touched as Jeffrey has already done the hard work of offloading task switching to AMBControl.

I’m not clear what your plan is here? Are you going to have multiple 26bit apps run ‘inside’ one single 32bit app

Under ADFFS 26bit apps run natively, so no different from any other app from the OS’s point of view.

My need for task switching is not actually anything to do with Wimp apps but 26bit Module support.

My question then is, when should (in functional terms) the vector chain be called? It seems to me that it should be called once after control leaves the application, and once before control returns to the application.

The main things to worry about are the appslot and the process state (environment handlers, redirection file handles, etc.). The vector call needs to occur before any of those are saved/paged out, and after they’ve all been restored.

For performance, we should definitely try and avoid calling the vector in situations where the Wimp doesn’t actually perform a task switch. But other than that I don’t think it matters too much when the calls occur (e.g. immediately after they’ve all been restored, vs. just before the Wimp returns to the task).

Jeffrey I imagine you will have some insight into when is best for performance counters to be notified?

It doesn’t matter too much when they get notified, as long as it happens at some point during the task switch sequence. If you’re profiling application code, then most modern machines allow the counters to be configured to only track events that occur while in user mode, so all the time spent in supervisor mode during task switching will be invisible anyway.

The Vector should be called by AMBControl whenever it task switches. The Wimp doesn’t need to be touched as Jeffrey has already done the hard work of offloading task switching to AMBControl.

Wrong on both counts, I’m afraid. AMBControl is a RISC OS 3.7 feature, long before my time poking around in the OS sources. And it only deals with remapping application space – all of the other process state (like the environment handlers) are handled elsewhere.

we should definitely try and avoid calling the vector in situations where the Wimp doesn’t actually perform a task switch

Which is why it should not be in the Wimp but in AMBControl, which should exit early if it’s being asked to map in the current DomainID.

Wrong on both counts, I’m afraid

I’m possibly misremembering then, but I could have sworn you made a lot of changes in the Wimp a few years back that were related to memory mapping.

And and only deals with remapping application space

We’re possibly talking about different requirements then as that’s the only thing I’m concerned with. It’s simply an informative call to say DomainID X is about to be paged out, DomainID Y has just been mapped in.

That should be enough to both track app CPU usage stats and what I’m after to know when the Wimp switches tasks to grab icons etc.