Request for community help - 32 bit documentation

I guess it wouldn’t be very nice of me to not give you a few pointers right now :-)

Personally I don’t think we really need a guide for “converting older software to run on 32 bit systems”. Instead, what we need is a guide for “converting old software to run on modern systems”. Which means it’ll need to cover at least the following:

1. PSR/32bit API changes
2. Video hardware changes (no more low colour/low resolution modes – good idea to document that one, Rick)
3. Everything that’s happened since the Iyonix. Unaligned load/stores, latent 32bit conversion issues, etc. – basically everything that’s already mentioned here

So far I’m seeing lots of #1 and #2 but none of #3. So statements such as “pure C programs will be fine once recompiled” are false, as the program might be intentionally (or unintentionally) performing unaligned loads/stores which the compiler isn’t able to detect (e.g. dereferencing unaligned pointers). Or maybe they’re incorrectly using a SWI which has had its parameters tightened up to deal with the 32bit address space. Or maybe they’re trying to use a low colour/low resolution screen mode – so the video considerations should really be a separate part of the guide altogether, as they could affect programs written in almost any language.

Personally I don’t think we really need a guide for “converting older software to run on 32 bit systems”. Instead, what we need is a guide for “converting old software to run on modern systems”. Which means it’ll need to cover at least the following:

Fair enough, always assuming the build for modern systems will run on RPC in 32 bit mode.

While we’d like to see everyone using truly modern systems for some people the Risc PC is truly modern because the OS had a 26 bit update a few years ago. So moving them to 32 bit without changing the hardware is a least one step along the road.

BTW. I hadn’t seen the primer (not sure why), so I’m sure lots of other people will have missed it.

So far I’m seeing lots of #1 and #2 but none of #3.

Hadn’t gotten to the hardcore stuff yet. You will notice that the current content is written in a (hopefully) non-scary manner. Which leads me to ask:

as the program might be intentionally (or unintentionally) performing unaligned loads/stores which the compiler isn’t able to detect (e.g. dereferencing unaligned pointers)

Isn’t it essentially the job of the compiler to make legitimate programs from source (or fail if there is a problem)? This would lead me to ask what common ¹ code sequences would exhibit such a behaviour and if such a thing is ever likely to work?

Or maybe they’re incorrectly using a SWI which has had its parameters tightened up to deal with the 32bit address space.

I shall add a note to mention this, but like with nemo’s point about the sprite area pointer not being necessary in SpriteOp calls passing sprite by address; if the OS should later require or vet this information, your program is wrong if you do something other than provide it as specified in the documentation.

That said, there may be some SWIs that have changed² from that published in the PRMs. Is there a list of such anywhere?

¹ People often quote “legal” BASIC as examples of why certain BASIC compilers are horrible – one I remember is a stack of DEFFNs followed by only one exit point. Legal, unfortunately. Horrible coding style? Never in doubt. It is easy to concoct bizarre code (IOCC makes a point of it), I’m just asking for legitimate code…

² By changed, I mean “won’t work if you do exactly what the PRM says”.

Isn’t it essentially the job of the compiler to make legitimate programs from source (or fail if there is a problem)?

Isn’t this the point where the documentation should be pointing out to the coder why it failed and what the correct method for that type of program segment?
Compiler messages are usually short and often a bit cryptic. This is understandable as the essay on the subject should be in the documentation. It strikes me that it’s also understandable that a compiler that can, if need be, output 26 bit style code should allow said 26 bit style code without errors being flagged.

Isn’t it essentially the job of the compiler to make legitimate programs from source (or fail if there is a problem)?

No, it’s the job of the compiler to produce code that does what you tell it to. If you’ve written *((int *)1) = 0; in your source then that’s what the compiled code should do. Maybe it will work fine on the target hardware. Maybe it will fail due to an address of 1 being invalid. Maybe it will fail due to an unaligned access. Maybe your program deliberately triggers aborts as part of its normal operation. Maybe it’s just a bug. The compiler can’t be expected to know what your intent was. The compiler can and should only do what you’ve told it to.

This would lead me to ask what common code sequences would exhibit such a behaviour and if such a thing is ever likely to work?

One common mistake that’s been seen in assembler sources is to use LDR to load from an unaligned address, when all that the code was interested in was a byte value. It’s feasible that similar mistakes might be made in C or other languages – accidentally using the wrong pointer type to access an unaligned location, but then only paying attention to the bottom few bits of the result so that on older systems no bug is actually visible.

Other bugs could arise from accidental use of null pointers. Fire up BASIC on RISC OS 3.1 and type !0=0. No error, and no crash because all you’ve done is overwrite the seldom-used branch through zero vector. Fire up RISC OS 3.7 and try the same thing. Still no error. Try it on RISC OS 5 and you’ll get an abort on data transfer, because zero page is now read-only in user mode. Hopefully one day we’ll also move to using high processor vectors by default, so that reading from null pointers will cause an abort as well.

That said, there may be some SWIs that have changed from that published in the PRMs. Is there a list of such anywhere?

No definitive list that I’m aware of. However this page makes note of some changes to the Wimp’s pointer validation, as well as containing a bit more information about pointer/memory related issues that otherwise valid code might bump into (e.g. accidentally performing signed comparisons of addresses). There’s also this page which documents a few more cases (OS_ReadLine, etc.).

Basically, if you’re planning on rewriting the documentation from scratch it would probably help if you’d read the original docs first ;-)

Basically, if you’re planning on rewriting the documentation from scratch it would probably help if you’d read the original docs first ;-)

…err… I meant here, not there.
But, then, that’s potentially one of the very things to transfer over from there to here… :-)
I’m confused. I need a cupcake.

Award-winning typo:

https://www.riscosopen.org/wiki/documentation/show/OS_AMBControl

“This SWI is for internal use only. You must now use it.”

(^_^)

Update: Fixed half an hour later.

I shall add a note to mention this, but like with nemo’s point about the sprite area pointer not being necessary in SpriteOp calls passing sprite by address; if the OS should later require or vet this information, your program is wrong if you do something other than provide it as specified in the documentation.

A very long time in the past I noticed that the only part of the 4-word messages file descriptor that was relevant to MessageTrans_Lookup etc was the 3rd. I used to exploit that in some of my code, but a long time ago I accepted that one should stick to the published documentation, whatever you might otherwise think.

but a long time ago I accepted that one should stick to the published documentation, whatever you might otherwise think.

This is the implicit danger in having access to the OS source code (or being a disassembly freak). You start to see places where things are “not necessary” and places where you can optimise your code by optimising within the scope of the OS¹. However, the OS should ideally be treated as a “black box” with a published API for while something might work today doesn’t mean it’ll work tomorrow, and so long as the OS follows the given API faithfully, it can change everything inside itself.

¹ I am guilty of this transgression. :-) I habitually set all four words of the sprite area control block prior to initialising a sprite area. Somewhere along the way, I noticed that that is all the initialise call does for private sprite areas, so in my OLED module, I just skip over the Init call.
Of course, should the OS later keep internal track of sprite areas defined and their sizes², I would be in the wrong here.

² This is extremely unlikely as it would be absolute hell trying to keep track of sprite areas of varying sizes held in movable locations of application memory. !Paint’s use of flex comes to mind; so perhaps the OS should do no more vetting than to ensure that a sprite pointer lies within the specified sprite area (and maybe that the pointer is actually accessible memory?).

Made some updates to the documentation. Hopefully it is better worded. The list of differences have been pulled out into a separate section, and made reference to this. Also (briefly) covered the unaligned loads issue. No point going into much detail here as the ARMv7 document covers it in a good level of detail. Made a start on the 32 bit issue.

One point to raise, though, is:

It’s feasible that similar mistakes might be made in C or other languages – accidentally using the wrong pointer type to access an unaligned location, but then only paying attention to the bottom few bits of the result so that on older systems no bug is actually visible.

Yes, it is feasible that this may happen, but I wonder if knowledge of the correct pointer type is something we will be wanting to go into? Is this not a language issue? We aren’t going to mention that on RISC OS, int == long, are we? That said, I would be somewhat surprised if the compiler lot you use something inappropriate (short?) as a pointer without throwing some sort of warning _unless) you use truly horrible code.

*((int *)1) = 0;

Swapping the ‘1’ for ‘0×8001’, the compiler does exactly what was asked of it:

MOV    R4, #0
MOV    R1, #&8000
STR    R4, [R1, #1]

leading to an Abort on Data Transfer at &80B0. Load the executable into Zap, go to &80B0, step back two instructions, there’s the STR unaligned staring back at you. Not hard to see what’s wrong with that instruction. :-)

Interestingly enough, the compiler still emits this code with -memaccess -L41, ditto if -cpu 7 is specified as well. Hmm…

Also did my (on-line) tax declaration. Given how scary what-the-hell French payslips are, it was actually remarkably easy – insert into this box how little I earned in the year, insert into this other box how many kilometres were expended going to/from work, tick this box to say I have a television (not for French stuff, but why not consider that my “payment” for the animé I do watch since Crunchyroll is caught up with stupid licencing restrictions) and the website pretty much did the rest, along with providing lots of pieces of paper to print out and store for all eternity…

Yes, it is feasible that this may happen, but I wonder if knowledge of the correct pointer type is something we will be wanting to go into?

I’m not suggesting that we go into detail on every little bug that a program might contain; that would be futile. I’m just saying (in perhaps a roundabout way) that we should avoid saying things like “recompile your code and it’ll be 100% fine”. Instead we can only say that it’ll probably be fine. That way the developer will (hopefully) still do a fair bit of testing himself instead of just assuming the recompile will have fixed all the problems. And if there is a problem, he can’t turn around and blame us for falsely telling him that it would be fine!

Interestingly enough, the compiler still emits this code with -memaccess -L41, ditto if -cpu 7 is specified as well. Hmm…

For performance reasons the compiler assumes that the programmer will know what he’s doing and won’t try deliberately using unaligned pointers – the runtime costs of checking every single pointer would be too excessive. -memaccess only really affects things like members in packed structs (e.g. for an int after a char) or halfword data types (where the CPU choice might mean LDRH/STRH aren’t available)

I just did a quick review of the 32 bit section of the wiki and this is coming along very nicely! I’ve not yet done a proper review – this can probably wait until it’s more complete – but it’s good to see something that IMO is much more presentable than the sprawl that it originated from. Hopefully, we can get the iyonix.com 32 bit site redirecting to this page when it’s ready.

Are you still working on this, Rick?

Not at the moment. Good weather recently, so there’s been a lot to do “in the garden”¹.

¹ Let’s see, there’s ~600m2 over there that we are slowly beating into submission using hand tools (!), about 500m2 over there that is mostly an easy mow, the ~150m2 of trees and brambles is a lost cause, the ~300m2 pond is a disaster (long story – thanks neighbour farmer for clearing your ditches and dumping the mud in what used to be a pond…); and that’s only discussing the east side. It’s a lot of work, though as I have a job, it’s mom that is doing a lot of it…