Environment handler etiquette

Several environment handlers are called in USER, so may not have a valid stack. What’s best practice in this scenario, should one check if the stack is valid first, or simply create a new stack and assume that the stack isn’t valid? Error for example may want to backtrace the stack, what happens if the triggering error occurred in another CPU mode?

The documentation for OS_DelinkApplication states the Wimp preserves environment handlers, how does the Wimp do this? Does it read them all prior to swapping to another task and associate any in AppSpace with the task? ie When a Module claims a handler, is it associated with the currently mapped task if it’s pointing at AppSpace?

Several environment handlers are called in USER, so may not have a valid stack. What’s best practice in this scenario, should one check if the stack is valid first, or simply create a new stack and assume that the stack isn’t valid?

You’ll have to provide your own stack. There’s no way that external code can know for sure if user mode has a stack or what type of stack it is.

Error for example may want to backtrace the stack, what happens if the triggering error occurred in another CPU mode?

For regular errors the kernel flattens the SVC stack and returns with the user mode registers restored (assuming the stack frame at the top of the SVC stack is consistent with a SWI call). Since regular errors should only occur as the result of a non-X SWI call (and non-X SWI calls should only be performed by user mode code) this should be fine.

https://www.riscosopen.org/viewer/view/castle/RiscOS/Sources/Kernel/s/Kernel?rev=4.12.2.46;content-type=text%2Fx-cvsweb-markup#l906

For serious errors (e.g. bit 31 of error number set) the exact behaviour varies, since it’s all under control of the error generator. Generally the exception dump area will be filled with a register dump, and one or more privileged mode stacks will be flattened, before the code calls non-X OS_GenerateError in order to raise the error.

https://www.riscosopen.org/viewer/view/castle/RiscOS/Sources/Kernel/s/Middle?rev=4.15.2.39;content-type=text%2Fx-cvsweb-markup#l709
https://www.riscosopen.org/viewer/view/mixed/RiscOS/Sources/HWSupport/FPASC/vensrc/riscos/end?rev=4.8#l180
https://www.riscosopen.org/viewer/view/mixed/RiscOS/Sources/HWSupport/VFPSupport/s/SupportCode?rev=1.2#l911

If you want to get a stack trace for serious errors then you’d probably want to use the new SeriousErrorV, which is intended to solve the problems of (a) how to get accurate stack dumps and (b) serious error handlers were flattening the stacks without letting other bits of the system know about it. However it perhaps isn’t quite ready for prime-time – for one it isn’t publicly documented yet (but there is some info in this header), and not all serious error generators are hooked up to it yet (actually I think it’s only the kernel that makes use of it currently), and there’s also no legal way for code outside the kernel to call the first reason code (since that one needs to be called in ABT mode in order to preserve the SVC stack)

The documentation for OS_DelinkApplication states the Wimp preserves environment handlers, how does the Wimp do this?

It saves and restores them all.

ie When a Module claims a handler, is it associated with the currently mapped task if it’s pointing at AppSpace?

It will be associated with the current task.

I’m not quite sure how you’d go about setting an environment handler at global scope – perhaps it’s only possible to do so before the Wimp starts? (e.g. the Debugger sets up the exception dump area to point to its own workspace, and *ShowRegs will only report from that area)

[mostly thinking aloud]

For regular errors the kernel flattens the SVC stack […]
Generally the exception dump area will be filled with a register dump, and one or more privileged mode stacks will be flattened […]

From the default error handler (s.Kernel):

     LDR    sp_svc, =SVCSTK-5*4    ; Just below top of stack

Hmmm, so is SVCSTK normally “empty”? I’m just trying to work out if the stack flattening is liable to muck anything up – but I guess if a module SWI enters and exits cleanly and everything is “completed” before handing control back to a user mode program, then stack flattening shouldn’t have any adverse effects.

Ah, yes.

SYS "OS_ReadSysInfo", 6, 0, 16 TO ,,svcstk%
PRINT ~svcstk%
DIM c% 31
P% = c%
[ OPT 2
  SWI "OS_EnterOS"
  MOV R0, R13
  SWI "OS_LeaveOS"
  MOV PC, R14
]
PRINT ~USR(c%)

Reports FA208000 in both instances. So when running a program, SVCSTK is essentially empty, only to be filled up when the OS (or some part, like a module) is entered? I see…

The documentation for OS_DelinkApplication states the Wimp preserves environment handlers, how does the Wimp do this? Does it read them all prior to swapping to another task and associate any in AppSpace with the task?

A lot of fanagling around with OS_ChangeEnvironment. Take a look for Wimp_Poll (s.Wimp03 I think) and try to unpick what is going on. As far as I can see, it doesn’t exactly preserve handlers, so much as set some generic handlers in the period between a task being paged out and a new task being paged in (at which time its handlers will be restored). I presume it sits on ChangeEnvironmentV and records when an application changes its handlers, so it knows what to restore when paging an application back in.

Talking of vectors and such, it is a shame that the Wimp never allocated one of the upper bits of the poll mask to having the Wimp call OS_DelinkApplication (and later Relink) on your behalf…

Hmmm, so is SVCSTK normally “empty”? I’m just trying to work out if the stack flattening is liable to muck anything up – but I guess if a module SWI enters and exits cleanly and everything is “completed” before handing control back to a user mode program, then stack flattening shouldn’t have any adverse effects.

I have a supplementary question that I think can be tagged on here.

What happens if you do not enter and exit user mode cleanly in a SWI? Say you change to user mode and then return to the address in R14_usr. Can you ignore the SVCSTK and assume that it gets overwritten by the next SWI, for instance.

I have been writing some SWIs that enter user mode to work with BASIC, and I have been doing it by the book. Then, on a whim, I tried what I said above and it seems to work and nothing crashes, at least so far. First, is it legal, and second, what are the likely consequences?

What happens if you do not enter and exit user mode cleanly in a SWI?

Apparently nothing? Looking at the vector entry code for a C module, it looks like a lot of stuff gets stacked that never gets unstacked, so I’m guessing that either the stack slowly fills up with abandoned rubbish, or returning back to user mode squashes the stack…

So far, my debug return code totally ignores the SVC stack, goes to USR mode, then jumps back into the program…with no obvious I’ll effects. Perhaps I ought to pick up R13_svc and print it out, see if it is growing or ….?

What happens if you do not enter and exit user mode cleanly in a SWI?

Things get left of the stack, which done enough times will overrun. AFAIK one should reset the SVC stack before doing so, or at least remove anything stacked during that SWI.

Things get left of the stack, which done enough times will overrun.

That might happen, but I am inclined to Rick’s take on this. I have found that the stack start address, on a page boundary, is the same every time I call my SWIs, which do not themselves reset the stack.

I am more concerned about anything that is not getting done that will be done when exiting the SWI cleanly from SVC mode. However, having read Rick’s desciption of how SWIs are called, I suspect this is not a problem.

Pushing my luck, here is another question. When working with BASIC you need to use R10, R11 and R12, which are not passed to SWIs. However, they are the very first items put on the stack by the SWI dispatch code. I can load them from a known fixed offset from R13_svc, but how future-proof is that? The alternative is to find the page boundary above R13_svc and the offset from there should be reliable. How far-out is all this? ;-)

That might happen, but I am inclined to Rick’s take on this. I have found that the stack start address, on a page boundary, is the same every time I call my SWIs, which do not themselves reset the stack.

I’ve just done a test, and… “The value of SVC SP is &FA207FB8”. Every time.

That’s with the vector handler stacking R0-R11 and R14, and the APCS function entry stacking R7, R11, R12, R14, PC…and me pulling none of that.

C module entry appears to:

• Stack R13! {R0-R11, R14} for generic vector entry
• Stack R13! {R7, R11, R12, R14, PC} for APCS entry

I was going to write some code to pick up and throw away those stacked registers; but there is no point. I wrote a simple test program to just call my module repeatedly, and each time in the SWI base address is the same.

I have the benefit that I get to return directly to the user mode program (my handler does not exit normally, it restores state and pushes a branch back into the program that called it). I think if you are a regular SWI call or vector handler, it is important to pull off anything pushed, because you don’t know what called you, or might have called you.

Either way, it looks like the SVC stack is flattened when it isn’t actively in use.

I have been writing some SWIs that enter user mode to work with BASIC, and I have been doing it by the book. Then, on a whim, I tried what I said above and it seems to work and nothing crashes, at least so far. First, is it legal, and second, what are the likely consequences?

It’s not legal, and I’m not quite sure why it seems to be working.

Consequences (apart from the SVC stack filling up) would be that the callback environment handler will stop working, because the kernel will only call it when the SVC stack is (about to become) empty.

I am more concerned about anything that is not getting done that will be done when exiting the SWI cleanly from SVC mode. However, having read Rick’s desciption of how SWIs are called, I suspect this is not a problem.

“Exiting cleanly” involves exiting via the kernel, so that it can trigger transient callbacks (OS_AddCallBack) and non-transient callbacks (OS_SetCallBack). That’s why we have OS_LeaveOS as a counterpart to OS_EnterOS.

Pushing my luck, here is another question. When working with BASIC you need to use R10, R11 and R12, which are not passed to SWIs. However, they are the very first items put on the stack by the SWI dispatch code. I can load them from a known fixed offset from R13_svc, but how future-proof is that? The alternative is to find the page boundary above R13_svc and the offset from there should be reliable. How far-out is all this? ;-)

I can think of changes that we might want to make which would easily break both of those approaches – but I suspect that finding the top of the stack (e.g. by rounding up to the page boundary, or by OS_ReadSysInfo 6) and fetching the values from there will be the most future-proof.

Usually code which needs access to “virgin” user mode registers will do so via a non-transient callback. But if you need to access R10-R12 for pretty much every SWI then setting up a callback each time might be a bit excessive.

Thanks Jeffrey. You are telling me what I suspected, and that things were looking too good to be true. ;-)

I will get access to R10-R12 via SVCSTK and use clean code.

It’s not legal, and I’m not quite sure why it seems to be working.

I can’t speak for Steve’s case, but for mine I found the reason in “Middle”:

10      LDR   stack, =SVCSTK
        LDR   r12, =ZeroPage+BrkAd_ws
        LDMIA r12, {r12, pc} ; call breakpoint handler

So that intentionally flattens the stack. I did wonder about why junk was left on it and not removed yet nothing went awry…

PS – the OS_BreakPt code (expectedly) blows up if you call it from SVC mode. Would it not be better to remove the SVC mode code, which obviously does not (and cannot) work, and instead return an error telling the user that it’s only supposed to be for USR mode code?

I think it’s best to consider OS_BreakPt a failure. It’s described as being for use by a debugger, but the Debugger module doesn’t use it, it doesn’t provide a breakpoint environment handler either. I also doubt that DDT uses it.

The fact that you can’t reliably use it to debug (and continue) within SVC mode code due to R14 being clobbered is a pretty serious problem. Plus flattening the SVC stack means that you might not get a useful stack dump.

I think it’s best to consider OS_BreakPt a failure.

It doesn’t see a lot of love, although I wouldn’t say it’s a total write-off. My understanding was that it was implemented for Application debugging and that they should be running in User mode. So it’s possibly useful for debugging C code – having said that, by the time you’ve written the Breakpoint handler and added the code to claim it, it would be quicker to simply BL to some debug code!

OS_BreakPt and the Debugger module are however performing different roles, I believe OS_BreakPt is meant to be inserted into your source code so you could for example display register values whilst your app is running, without actually interrupting it, or break when certain conditions are true. Debugger on the other hand is meant for debugging machine code, where the source isn’t necessary available and as such replaced the breakpoint address with a Branch to enter the handler to avoid SVC_R14 corruption and on resuming IIRC puts the original instruction back.

DDT doesn’t show the swi OS_BreakPt – in its listing.

During the updating of BreakPt module – which catches the swi – and displays the regs and flags in a multi-tasking window – at a point when the swi OS_BreakPt is called in your own code.

Running ‘DeskDebug’ at the same time as the above module – curdled things – so I guess it uses it. (DeskDebug is User mode only – as is the BreakPt module).

Can the swi OS_BreakPt can be lifted out – to see how it works – made to be able to be called in SVC mode!

Note that Reporters ‘swi Report_Regs’ (only low Regs) can be called in SVC mode – but following through the code is not quite so easy.

Using ‘BL Debug’ would work – if you have some spare space – use a bit of writing
space.

Just a note – there is ‘QDD’ A low-level Debugger (Module) for RISC OS – by Adrian Lees.

Press Alt-Alt to enter the debugger etc…

I suppose you could use a modified – simple swi module – do some checks – and print info to its private word or RMA space that could be checked by !Zap – ‘save workspace’.

Just a note -there is a debug module – ‘QDD’ by Adrian Lees 0.10 (17 Jun 2007).

A Low-level Debugger for RISC OS – Work In Progress – Press Alt-Alt to enter the debugger….

Jeffrey – as Jon says, OS_BreakPt was intended for use in USER mode programs, and while RISC OS appears to attempt to try to do something in privileged modes (and fails), there are warnings all over the place that OS_BreakPt is only intended for use in USER mode programs.

It still has a purpose. The module that I am writing responds to OS_BreakPt and does a variety of things (list registers, disassemble memory around PC, dump memory around register ‘x’, show contents of (USR) stack, etc.
It is intended when you want something more powerful than printing A, B, C to the screen (or finding a way to paste in calls to OS_ConvertHex#) but don’t want to go as far as running the program in DDT.
It may be a useful intermediate step.

Note, also, at the time when OS_BreakPt was devised, Acorn was of the impression that any serious programs would be written in assembler.

The fact that you can’t reliably use it to debug (and continue) within SVC mode code

Does DDT allow one to debug modules written in C? I was of the impression that DDT was also for user mode programs…

Colin:

DDT doesn’t show the swi OS_BreakPt – in its listing.

It wouldn’t. That’s what the program being debugged would call. DDT should have OS_BreakCtrl or OS_ChangeEnvironment. Of the latter, there are several calls. Unfortunately two of them appear to work by pulling registers off the stack and then calling the SWI; thus meaning the registers in use are set up elsewhere. I would not be surprised if DDT didn’t call OS_ChangeEnvironment 8 if only to disable it.

During the updating of BreakPt module – which catches the swi – and displays the regs and flags in a multi-tasking window

You are writing one as well? :-)

Mine works interactively in the command line. I might do window stuff as an option in the future, however for now I’m not even going to assume that the Desktop environment exists (RISC OS Pico, for instance).

Can the swi OS_BreakPt can be lifted out – to see how it works

It works about as you would expect it to. Work out the PSR, push it to saveregs. Work out PC, push it to saveregs. Push R0 to saveregs. Then using R0 as the saveregs pointer, push R1-R14 as well. Then jump into the breakpoint handler.
There’s also code to handle a non-user mode case. I can’t speak for ABT32 or the like, but it does fail with SVC32. This is insurmountable.

Kernel.s.Middle is where the code is.

– made to be able to be called in SVC mode!

Utterly completely and totally impossible. The calling of a SWI in SVC mode requires R14 to be saved beforehand. While you might be able to arrange for the caller to stack R14 prior to calling OS_BreakPt, with your debug code noticing the processor mode and replacing the corrupted R14 with the stacked one; what you can’t get around is the line I quoted from the source, LDR stack, =SVCSTK. That basically resets the entire SVC stack to its default value. I’m not sure why RISC OS does that, but it pretty much precludes the use of OS_BreakPt in SVC code.

Jeffrey – as Jon says, OS_BreakPt was intended for use in USER mode programs, and while RISC OS appears to attempt to try to do something in privileged modes (and fails), there are warnings all over the place that OS_BreakPt is only intended for use in USER mode programs.

So if there are warnings all over the place, why do we need it to also return an error to tell people that it can’t be used from SVC mode? :-)

In my opinion partial support for SVC debugging is better than returning an error.

Does DDT allow one to debug modules written in C? I was of the impression that DDT was also for user mode programs…

No idea. However, Debugger breakpoints work from SVC mode.

Rick the ‘BreakPt’ is by Thomas Leonard 2.04 04-Oct-1997 – since it gave a error – I though it would be interesting to try and understand vectors etc.

Quote from help QDD – (Adrian Lees)

QDD can be used to debug applications (Wimp-based or otherwise),
modules, device drivers etc.

It’s a common misconception that DDT cannot be used to debug non-application code, but in fact this is not true. The user interface, however, does not make this trivial in that you must manually run a specific client application of your module under DDT.

In fact QDD goes further than DDT in that it makes only minimal assumptions about the state of the OS, and uses a fairly minimal amount of OS code in its own operation so that it should be possible to use QDD to debug code that executes at times when it is not safe to re-enter the OS.

It’s a common misconception that DDT cannot be used to debug non-application code, but in fact this is not true. The user interface, however, does not make this trivial in that you must manually run a specific client application of your module under DDT.

What does that actually mean, specifically the part highlighted.
I interpret that as “no, DDT doesn’t support debugging module code unless you convert the module code to run as application code¹ and then debug it…”. Something like that?

QDD sounds good. One question I’ll have to put to Adrian one day is if QDD can be controlled via serial. I have my Pi hooked to a USB serial device so I can output tracing information and see it on a PC (HyperTerm) even if the RISC OS machine has crashed. It would be extremely interesting to be able to control a debugger in the same manner; especially if interacting with desktop applications²…

¹ But then, you can hardly be said to be debugging that which is broken…

² That is one of the reasons I have not considered multitasking applications in my BreakPt module; there are non-trivial problems with using windows and screen handling while an application (and indeed the rest of the machine) is “frozen” awaiting input.

In my opinion partial support for SVC debugging is better than returning an error.

Partial support?

Open a new file in Zap, press Space twelve times. Switch to Code view, then change those three words to:
SWI OS_EnterOS
SWI OS_BreakPt
SWI OS_Exit

Save it typed as Absolute. Run it. Instant abort as soon as the OS attempts to do something with the BreakPt SWI. I’ve not had the time to figure out where exactly it is failing, in theory it should make it to the breakpoint handler so long as you accept that R14 is trashed and so is your entire stack (so going into the breakpoint handler is a one way ticket).
Remind me, what part of “partial support” is flattening the stack? :-P

Instant abort as soon as the OS attempts to do something with the BreakPt SWI.

That’s a bug, not an unsolvable design flaw.

What should happen is that you get a complete register dump, apart from R14, which will be &DEADEAD

What actually happens is that it crashes trying to write the register dump due to a mistake made during the 32bit conversion.

Remind me, what part of “partial support” is flattening the stack? :-P

Flattening the stack, along with R14 being &DEADEAD, is what makes the difference between partial support and full support.

Flattening the stack, along with R14 being &DEADEAD, is what makes the difference between partial support and full support.

I suppose, if the BreakPt entry is fixed, it could work if you stack R3 and R4 and then copy R13 into R3 and R14 into R4. Then call OS_BreakPt as required.
Then, upon return, pick up the original value of R14 from the R4 returned, and then the original R13 from R3 (to “unflatten” the stack), then pull the saved values of R3 and R4…
Messy as hell, but not impossible.

FWIW, my module now fiddles R13 to correctly discard the junk put onto the stack during module entry. Not that it would work terribly well (it currently makes some application mode decisions such as refusing to let you set PC outside of the memory space of the current application). But it’s something that could be looked into, in time.

Just tried your prog on RO4 – adding a little extra of setting V.
It gives &1000801B -looks like it is setting the 32bit flag.