Well, that was simple...

I wanted to get SocketManager working with RISC OS 5 because when working with socket code, it is useful to have a way to tidy up sockets and there’s no obvious method in RISC OS to list active socket handles…

…but the archive contains Justin Fletcher’s IServices module…it’s sort of like the X degrees of Kevin Bacon, trying to find older stuff that doesn’t use one of his many useful modules.

Since nobody can distribute 32 bit safe versions of his modules, I’ll briefly outline the steps used in case you want to mod-your-own and/or try other stuff. This isn’t just for his modules – the same process can be used to attempt to convert any module that is not 32 bit compatible and the licence/terms/whatever don’t permit distribution of modified versions or in cases where nobody has attempted to make a conversion.
Obviously, it shouldn’t (famous last words!) be too hard to convert smaller (under 10-15K) modules. The larger the module, the more pain you’re likely to experience.

Okay, here goes:

• Throw the original (26 bit) module into ARMalyser with the options Verbose and Assembly and the output format Text.

• Take the result and save it to disc. This is arguably the most important part of the process as ARMalyser can convert the executable into something objasm can assemble and annotate it to highlight things that need attention.

• Open it up in an editor.

• Step one, just above “ENTRY”, insert the following:

   AREA |Mod$HackCode|, CODE, A32bit

• Step two, check that the module header is correct. It may well end at the SWI decoding table offset. So append the following:

   DCD    0
   DCD   |thirtytwobitword|

|thirtytwobitword|
   DCD   1

• Search for ’"Caution" and look for all the entries that say “not 32bit safe”.

• → If the entry ends with a ^, remove that.
• → If the entry is ORRing something from R14 into PC, it is probably trying to set the V flag. Replace with:

   CMP   R0, #1<<31
   CMNVC R0, #1<<31
   MOV   PC, R14

• → If the entry is TEQP or TSTP or something (anything?) else, it is likely to be more intense flag twiddling. Find a datasheet to work out what is going on and the MRS/MSR to replace it with.

• → Note all the warnings for conditional after BL/SWI – the 32 bit API does not restore flags across calls, so if the module misbehaves, it may need some reworking if something depends upon flag restoration and doesn’t get it. I skipped over this and the module I modded “appears” to work, but I’ve only used as much as necessary for SocketManager to work.

• → It might be work taking a quick check for SWP as this won’t work on the Pi3 and other ARMv8 processors.

• →If it starts messing with hardware addresses, IOC etc… give up. Conversion is possible but non-trivial as that sort of hardware doesn’t exist on newer machines. Why do you think nobody has 32bitted Wimp2 yet? ;-)

• Save the modified source. Then drag it to objasm with Pre-UAL ARM selected.

• Save the resultant object file (with a different name).

• Now drag that object file to Link with the Module option selected.

• Save the final file to disc. It’ll offer it as “!RunImage” so you’ll want to rename it, obviously. ;-)

With a bit of luck, that’ll be sufficient. But it depends greatly upon how the module was written. Compiled C may do all sorts of evil things (like non-aligned LDR), while assembler written by hand may be a lot simpler to convert. In the case of the module that I converted, there were only three things that needed attention.

In the case of the module that I converted, there were only three things that needed attention.

There’s an amazing number of items that are that simple to convert.
Although some have a lot of instances of ^ it’s almost always a standardised exit each time.
BTW. Had you thought of running the original through !SID to derive the “source”?

Had you thought of running the original through !SID to derive the “source”?

This is arguably the most important part of the process as ARMalyser can convert the executable into something objasm can assemble and annotate it to highlight things that need attention.

(^_^)

DCD |thirtytwobitword|
|thirtytwobitword|
DCD 1

You’re not supposed to put the flags word immediately after its pointer, because then the header can’t be extended again in future.

When you chaps have sorted it out amongst yourselves, please let me know with a URL download.

When you chaps have sorted it out amongst yourselves, please let me know with a URL download.

Erm, relevant bit of Rick’s text:

Since nobody can distribute 32 bit safe versions of his modules, I’ll briefly outline the steps used in case you want to mod-your-own and/or try other stuff.

Basically Justin specifically said people were not allowed to distribute versions of his code modified to be 32 bit compliant.

Essentially, unless someone produces a patching code, disassembly twiddle and recompile is the only option.

You’re not supposed to put the flags word immediately after its pointer,

I’m not aware that there are any “official” guidelines regarding what to put after the header. Should we leave a few null words or something? The Wimp, for example, puts the module title directly after the header. For better or worse, as it is some sort of data… https://www.riscosopen.org/viewer/view/castle/RiscOS/Sources/Desktop/Wimp/s/Wimp01?rev=4.73#l2050

I’m not aware that there are any “official” guidelines regarding what to put after the header.

To be honest, it’s a flags word and we appear to have one bit in use.
The likelihood of needing to expand is what exactly?

The likelihood of needing to expand is what exactly?

I believe Sprow is referring to the Module header itself and the ridiculous way in which header values are validated when a Module is loaded. If a value at offset X in the header is below the length of the Module, and it’s expected to be an offset within the Module, the parameter is presumed to be valid.

The undocumented way of avoiding this, is to ensure the header is immediately followed by an instruction and not a value that may fall within the Module length. The obvious way to avoid this long term would have been to add a Module header version number when the 32bit flag was added, so the Module loader code knew what spec of header it was dealing with.

The obvious way to avoid this long term would have been to add a Module header version number when the 32bit flag was added, so the Module loader code knew what spec of header it was dealing with.

31 spare bits in the flags word…

I’ve never really understood the “don’t put the flags word after the header” mantra either. Either it’s about (a) making sure the OS doesn’t think the flags word is part of the module header (in which case the OS can just spot that the flags word offset is pointing to +4 bytes from the current location, and therefore there can’t be any more header words), or (b) allowing a module binary to be patched – in which case if you need 4 bytes of space you can probably just move the flags word to the end of the binary, and if you need more space you’ll be in as much difficulty as with any other scheme.

The problem of the kernel having to guess the length of the module header easily be solved once and for all by having a length field somewhere (top 8 bits of flags word = length in words of any extra header components? Or keep the current header length as-is and change the flags word to an “extended header block” with the length indicated in the same way)

the ridiculous way in which header values are validated when a Module is loaded.

Witness: https://www.riscosopen.org/viewer/view/castle/RiscOS/Sources/Kernel/s/ModHand?rev=4.11.2.13

Given the module header is data I don’t understand why Acorn did not include a word in the header to identify either the length or the type of module header, explicitly, instead of guessing things.

Or keep the current header length as-is and change the flags word to an “extended header block” with the length indicated in the same way)

I’d be inclined to want to replace the first two works with a word “RMOD” for identification, then a word giving the module header version (the length can be construed from the version). However this would interfere with the ability to have a literal instruction in the Start offset to jump into the code . . . if anybody ever used this?

Thus:

(top 8 bits of flags word = length in words of any extra header components? Or keep the current header length as-is and change the flags word to an “extended header block” with the length indicated in the same way)

Is probably the way to go (plus retaining better backwards compatibility). With the length in words it gives us around 1K to play with…

I’ve never really understood the “don’t put the flags word after the header” mantra either.

In my mind it’s almost like saying don’t put anything after the header which would make the header a bit lonely :)

The problem of the kernel having to guess the length of the module header easily be solved once and for all by having a length field somewhere (top 8 bits of flags word = length in words of any extra header components? Or keep the current header length as-is and change the flags word to an “extended header block” with the length indicated in the same way)

Yup. “Exactement” as David F might say.
I’m presuming modules with this modified header still work with earlier OS revisions (5.x)

IIRC the advice not to put the flags word immediately after the offset to the flags word is because its value is likely to be very small (either 0 or 1 at present, remaining small until a lot more flag bits have been defined) and therefore liable to be confused for another offset value by dumb module header parsers, not least the human kind :-)

and therefore liable to be confused for another offset value by dumb module header parsers, not least the human kind :-)

Seems like us human dumb header parsers expect it to be part of the header sequence.
Probably best to avoid the difficulty of reprogramming us. :-P

From observation modules that have the flags word immediately after the older header items seem to be more common.