RiscPC update Aug2014

32OK

I don’t know much either but google found this.

An “informal lightweight mechanism” is alleged in a Drobe article from 2006 titled “Resisting change is short-sighted” talking about mandatory headers, AIF in this case. It may be that proper headers that include a 32bit flag superseded the end of file “32OK” notion which may have had some level of implementation in files including Utilities. As far as we know, I believe, actual testing for this data was not implemented, other than possibly by Aemulor?? (The actual reference is in the comments.)

“However mandating that all absolute files should have AIF headers is another matter, and isn’t something that should be introduced without warning by any party (I would have been equally unhappy if Castle had done it). Entire portfilos of software can’t be updated overnight even if there are still people willing to do it. AIF headers on non absolutes such as FFC utilities is plain wrong, there is already an informal lightweight mechanism for these for these, which is to put “32OK” in the last word."

Hope this helps.

Good Googling – it looks like we can add Druck to the (very short) list of people who might remember where 32OK came from.

To determine whether Aemulor looks for it (short of Adrian popping into this thread!) presumably we can just write a simple utility that reads the PSR and prints out the value? If we get back a 26bit CPU mode without a 32OK footer but a 32bit CPU mode with a 32OK footer then that’ll prove that it’s being used by something, and that’s really all that matters here.

Anyone got a copy of Aemulor?

Anyone got a copy of Aemulor?

Yes, on both the Iyonix and Raspberry Pi.

I found an Aemulor Support Forum post lurking on the Iyonix, and the original still exists. . The why does not matter much but here is the business.

quote
“The problem is an utility called FPlay, which isn’t marked as 32-bit compatible…. and Aemulor is therefore trying to execute it; since current versions of Aemulor don’t support 26-bit sound this doesn’t work! (FPlay waits for the sound to complete, but it never started! Smile

The FPlay utility inside Messenger Pro, the latest version of NewsHound and possibly other apps, should – IMHO – be marked as 32-bit compatible.

To be fair to the developers, RISC OS 5 doesn’t check utilities to see if they are 32-bit ok, and the decision to mark utilities as such was taken late in the Iyonix’s development, after the first release of 32-bit Messenger Pro if memory serves!

You can change your copy of FPlay by loading it into Zap and, making sure that you’re in ‘Code’ mode, go to the end of the file and type EQUS “32OK” and press Return. This adds the four bytes ‘3’ ‘2’ ‘O’ and ‘K’ (in that order!) to the end of the Utility file.

Then save it, and you should find FPlay runs correctly with or without Aemulor running and set to run all 26-bit code automatically.

We put in the ‘only run if dragged onto Aemulor’ option as a fallback in case of problems like this, and choosing that option does indeed avoid the conflict, but makes Aemulor less transparent in use."
end of quote

I wrote a bit of BASIC assembler, I am not up on this stuff so I won’t show it here, for shame mostly. It is an unheadered Utility that calls “OS_PlatformFeatures”,0 to find the returned value of bit 6. Bit 6 is set when run normally and clear when Aemulor’d. “32OK” on the end forced the the Utility in 32bit space returning bit 1 as set even when running via Aemulor. (I verified the bit 6 results with a BASIC prog filetyped as BASIC and BASIC26.)

I could not find anything in the Aemulor documentation.

For crying out loud, it’s four bytes on the end of your transient utility and every one in the RISC OS source tree does it.

and

You can change your copy of FPlay by loading it into Zap and, making sure that you’re in ‘Code’ mode, go to the end of the file and type EQUS “32OK” and press Return. This adds the four bytes ‘3’ ‘2’ ‘O’ and ‘K’ (in that order!) to the end of the Utility file.

Is the answer to document that “Aemulor may trigger problems with some Utilites. The test is to run the Utility with Aemulor disabled to check that it is indeed 32 bit safe. Any Utility that passes this test can be patched by adding four bytes ‘3’ ‘2’ ‘O’ and ‘K’ (in that order!) to the end.”

Then offer a small patch utility to add the bytes (if they don’t exist and the utility header is marked 32 bit)?

Sigh. You’re really spoiling for a fight aren’t you?

Not really. To be honest I could care less as I rarely write transients (I think I’ve written three in my entire life). I just found it unbelievable that something would interfere with a program requiring the application of this bit at the end of the file, pretty much everywhere, and this was not being pointed out in available filetype references.
Perhaps this doesn’t bother you. It should bother every single developer who doesn’t want to chase phantom “bugs” due to differences in how things are implemented between systems.

Granted, Aemulor has a non-trivial task to do and it is damn difficult to know (on the fly) if something is truly 32 bit safe without running code analysis, so it makes sense for the author of a Utility to flag it as being 32 bit safe.
This, however, being a fairly important change of behaviour really needs to be described and detailed. David Pitt performed some Google-fu and eventually dug up one Drobe article and a post on the Aemulor support forum. Given that this issue potentially affects all transient utilities, it demands better coverage than that.

Your conclusion is incorrect, because we are currently in possession of two established facts here:
1. Something expects or expected (historically) to see that footer, otherwise it wouldn’t exist

You will notice that I made very very specific mention of a standard installation of RISC OS 5; and the next paragraph poses the exact same question again with regards to RISC OS 5 with Aemulor running. That, I believe, is where the difference lays, and it is up to the reader to imagine (or try) what happens then.

to be conflated with an incorrect belief that it is not required.

Never said that. I think we have pretty much established that this is required. My primary question, all along, is that if this is required (and you have pointed to the entire codebase several times), why is this not clearly described somewhere already? As I said – osmosis.

I’m pretty confident that nothing in RISC OS 5 checks for it or cares whether it’s present or not.

I’m extremely confident. I’ve read the source. RISC OS will merrily load any old thing typed as Utility and proceed to attempt to run it. From a space claimed in the RMA, thankfully as User mode… (though I can sort of understand Gerph’s point of view here).

Maybe we should just say it is advised for reasons which are lost in the mists of time. Yes, I agree things like this need to be documented.

Good. Then we agree.

“However mandating that all absolute files should have AIF headers is another matter,

Indeed. It is a good idea and I can see his point, but I can also see the point that a lot of plain executable code will work right away without an AIF header. Certainly, you can write a perfectly good program in the BASIC assembler and not need to think about ReadWrite sizes, zero-init code, blah blah. Complicated stuff, not really necessary for a small program.

AIF headers on non absolutes such as FFC utilities is plain wrong, there is already an informal lightweight mechanism for these for these, which is to put “32OK” in the last word."

To be fair, it was not an AIF header. It was much reduced header and was based under some peculiar notion that doesn’t entirely make sense – the ROLtd documentation says “It is strongly recommended that all utilities include the correct values in order to ensure that truncated, or over-long files can be detected.” (note: the current handler does not appear to actually perform any checks!) but what I noticed was that the header has separate values for Read/Write and Read Only areas, something not actually supported by Transient Utilities (they don’t have areas and fancy AIF stuff like that); it really only needs a “the program is supposed to be this size” word.

The documentation on our wiki is available for community editing. Knock yourself out.

Done. I have linked from File Types to a new page that describes the format and use of Utilities.

<shudder>Textile tables</shudder>

So let’s point to this and consider the case closed, eh? (^_^)

a new page that describes the format and use of Utilities.

I’ll have a much more useful example up soon(ish).

The 32OK signature on Utility files was, I believe, something that Castle proposed, although I don’t have documentary evidence to hand, and don’t consider it important enough to wade through past emails/documentation etc. It didn’t originate with me, but Aemulor does test for it.

Its presence guarantees that the utility will never be run under emulation, even when invoked from a 26-bit application. If not present, Aemulor plays safe and will step in if (i) the utility is being executed by a 26-bit application, or (ii) its ‘run all 26-bit code automatically’ option is enabled. It knows nothing of any header on utility files.

Do be aware that if the utility code is run under emulation, then in addition to it seeing a 26-bit CPU mode, calls to OS_PlatformFeatures will return bit 6 as clear, indicating a non-32-bit platform.

I have linked from File Types to a new page that describes the format and use of Utilities.

I have added a page describing Sprite files (that was easy!) and I’m now working on describing Modules. It isn’t linked in to anything at this time as it is unfinished – you can find it here: https://www.riscosopen.org/wiki/documentation/show/File%20formats:%20Relocatable%20Module

Any takers for adding some more file type docs? If you want easy-peasy: Text, Command, Obey, etc. A bit harder are BASIC, Absolute, and Template. Insane is Font (I read the PRM and the FontEd help file and I’m still scratching my head).

I ought to see if any of my filetypes are (still?) relevant and put them on the list. Looking down the file types list, are all those unmarked file types really unallocated? <gulp!> There are loads with no tick on the left.

The 32OK signature […] guarantees that the utility will never be run under emulation

So we can tabulate what we now know

The rot13 header thing was added in Select’s FileSwitch 2.68 to coincide with the A9 Home but states that the header is not checked for when running on a 26 bit version. The total number of A9 Home machines still in use is probably small enough to reduce the table to just the shaded cells.

Even the ‘32OK’ tag isn’t really that truthful, since many 26 bit utilities are 32 bit. Part of the reason I recall that RISC OS 5 waves all utilities past is that even ones with

MOVS pc, lr

LDM sp!, {pc}^

Part of the reason I recall that RISC OS 5 waves all utilities past is that even ones with

MOVS pc, lr

and

LDM sp!, {pc}^

are generally OK as utilities are run in USR mode, so all that happens is the flag bits get randomised from the non existant SPSR.

On XScale (and older?) yes, on other architectures no. ARMv7 machines will get rather upset if you try either of those from USR mode. Not sure about ARMv6.

https://www.riscosopen.org/wiki/documentation/show/ARMv7%20compatibility%20primer#movs

I’ll have a much more useful example up soon(ish).

Done. It is a utility to list all open files. I had one from long long ago, but since it was a useful idea, I cracked open the PRMs and wrote one from scratch (wasn’t hard!). Processor-agnostic code, carries the ROLtd header and the Aemulor footer, so it ought to run on just about anything that is capable of running RISC OS. ;-)

Done. It is a utility to list all open files.

Nice one. I’ve tweaked the 32OK description just to clarify the fact that the 32OK marker doesn’t mean the utility is 32-bit; it means it’s safe to run on either 26- or 32-bit.

32OK doesn’t mean that it will will work with a A310 or RO3.10 emulator.

Not much info on the Read/Write value.

Would it worth the effort for RO5.xx to check now on Utility’s to see if they are flagged 32bit.

It would also be handy if the ‘C’ compiler/Linker add a extra flag for programs running on later ARM chips.

I did see some Utils on DP’s website not having the ‘32OK’ at the end.

Rick did a handy program !Scan32bit – perhaps a similar program for checking for ARM instructions that later ARM’s don’t like.

Not much info on the Read/Write value.

As I mentioned on the page, it is bizarre to have Read/Write and Read sizes in a file that doesn’t understand such a concept. I think mandating the AIF headers went to somebody’s head. ;-) Really only needed a file size word.

Would it worth the effort for RO5.xx to check now on Utility’s to see if they areflagged 32bit.

I think so, yes.

It would also be handy if the ‘C’ compiler/Linker add a extra flag for programs running on later ARM chips.

Can you write Utility code in C? If so, then yes, the linker should do this. If it is assembler only (and I think it is), then this is something for the programmer to know…

Can you write Utility code in C? If so, then yes, the linker should do this. If it is assembler only (and I think it is), then this is something for the programmer to know…

Now you can:
https://www.riscosopen.org/news/articles/2014/10/25/h2-sharper-tools-for-crafting-code

Can you write Utility code in C?
Now you can:

Oh, yup. There it is in big letters. Duh me.

Still, nice to have the option, and only a quarter century late (’eh Acorn?!). I wonder how many hoops needed jumping through to make that work, and if there are specific limits to what you can do.

The big difference here, in case it wasn’t immediately clear from the description is that Utilities do not replace the current application. No farting around with system() and/or trying to push the current application up in memory and hope like hell the called application doesn’t trash you. No, Utilities are little things that load (into the RMA), do their thing, then peacefully hand the baton back to the callee.
That we can now write them in C is marvellous. They are useful, but underused (perhaps due to the previous requirement to write them in assembler?).

I think so, yes.

To follow up on this – I think that RISC OS should check for the code to be flagged as 32 bit okay. However, unlike ROLtd, it should not outright refuse to run the program, better to pop up a message box (or other such warning – OS_Confirm in the CLI?) to say something like “This Utility may not work correctly or may cause the system to crash. Do you want to continue and run this Utility anyway?” [Yes / No]

Rick did a handy program !Scan32bit

Where did you get this from? I have no recollection of this, nor do I appear to have any source. I think I might need something to kick my memory. At any rate, it probably needs to be updated, a lot, if it is old (circa Y2K vintage).

I think I might need something to kick my memory.

I’ve spent the evening looking through some CD-R archives from around 2003. Nothing, so I guess it is later. I did come across some amusing stuff, like some of the RISC OS 2 kernel complete with amusing code comments (and, WOW, they really didn’t like the keyboard hardware!).
Also found a picture of me at school fete day, from 1988 (I’d be…14 I think). And, yes, my school uniform was utterly boring → http://www.heyrick.co.uk/random/pics/rick_1988.jpeg
Plus some interesting source codes and stuff that would probably make me cringe now. ;-) I wonder if I can get that crappy old terminal app that I wrote ( http://www.heyrick.co.uk/images/software/mterm/mtrm1.gif ) to compile and run on a Pi? Might be fun to see how far it gets before blowing up.

I’ve spent the evening looking through some CD-R archives from around 2003. Nothing, so I guess it is later.

No, it’s earlier. I’ve put copy for you here

Oh, okay… It’s from here: http://www.heyrick.co.uk/assembler/32bit.html

Looks like the source is BASIC, so shouldn’t be too hard to redo it in C, and modern machines have enough memory that it ought to be doable to load the entire file into memory for quicker scanning…

Thanks.

If anyone is interested, I have updated my program WrapAIF (that adds the header to transient utilities) to support adding the “32OK” post file signifier if desired.

It is available from http://resurgam-riscos.github.io/index.html

Then offer a small patch utility to add the bytes (if they don’t exist and the utility header is marked 32 bit)?

…WrapAIF… …It is available from http://resurgam-riscos.github.io/index.html

Yep, that would be the kind of thing.

Is there anyway of turning off the hardware checks in RO 5.21 – at startup – for
use with the emulators?

Is there any replacement for the *VIDCBandwidthLimit command – with the emulator its back from 16M colours to 256 @ 1280×800.

Is there anyway of turning off the hardware checks in RO 5.21 – at startup – for
use with the emulators?

How do you detect an emulator? If the emulation is good enough the ROM startup shouldn’t be able to tell. That said RPCEmu has the SyncClock module for example, something similar detected early on would give you half a shot.