What is code? History of an illusion.

Back in the late 1970s I bought an Acorn Atom because with it came a tutorial about writing 6502 assembler. In those days I entertained an illusion: that machine code was a just a chunk of memory containing processor instructions. Only later did I ask myself about where the code should be loaded, or how it used a stack. In other words, the tutorial, and later books like Arm Assembly Language Programming by Peter Cockerell, encouraged learners like me to overlook somewhat the role of runtime systems in the execution of code. The BBC BASIC assembler lets one write code into a DIMmed block, to be CALLed by the BASIC program, so that it is the BASIC runtime system that sets up the stack, the exception handlers and so on; this makes life easier for the programmer. What other runtime systems does the RISC OS programmer have available? Beside BASIC there are GNU C and Norcroft C, which I believe are rather different. There is also Charm and possibly other idiosyncratic systems. Both Cs have evolved through many versions, and I am not clear to what extent these are backward- or forward-compatible. I would be grateful if someone could point me at any RISC OS documentation that clarifies these issues.

In those days I entertained an illusion: that machine code was a just a chunk of memory containing processor instructions.

Machine code is a chunk of memory containing instructions.

Only later did I ask myself about where the code should be loaded, or how it used a stack.

These issues are generally dealt with by the “environment”. Unless you’re writing an OS ROM, there will be something that loads your code – regardless of what language it is written in. That something, the environment, may be simple (as in the case of writing a program in assembler to run from RISC OS’ command line) or it may be more complicated (the C runtime) or very sanitised (BASIC).

encouraged learners like me to overlook somewhat the role of runtime systems in the execution of code.

Ah, but remember in many cases you are not running assembler, you are running assembler through BASIC. This changes things greatly. It isn’t only the fact that you don’t need to use OS_GetEnv to work out where you put your stack, it’s also the fact that you can drop in and out of BASIC as required. Want a block of memory with powers of two in it? Just make a FOR…NEXT loop in BASIC poking the values into memory. It’ll be there when the program is run and the code “assembled”.

What other runtime systems does the RISC OS programmer have available?

Pretty much everything that calls itself a language ought to do some of the work. Failing that, one could fall back on the default RISC OS handlers.

Indeed, for a small assembler program that doesn’t have complicated memory requirements, one could simply use OS_GetEnv to read the end of application workspace, check it’s big enough, and set the stack there.
For more complicated programs, you’re better off using a higher level language. Since one can write modules in C, there’s not much need for assembler except as veneer glue and for speed critical code; though pretty much any of AmCog’s games (written in straight BASIC) show what’s possible without resorting to wodges of hand-tuned assembler. We needed it with 8MHz processors, not really with 800MHz processors…

Both Cs have evolved through many versions, and I am not clear to what extent these are backward- or forward-compatible.

NorCroft C, pretty compatible within architectural limits. A 26 bit 26/32 CLib will run old 26 bit code and newer 32 bit code (a misnomer, it’s more “neutral”).
This isn’t the entire story. The DDE compiler typically generates code that works on 32 bit machines, but also works on the likes of the ARM3! There are some compiler flags to fine-tune the code generation to work better for modern processors (such as the use of instructions like STRH). I may start introducing this for my code that is not intended to run on anything prior to RISC OS 5.
Clearly 26 bit code WILL NOT run on a 32 bit system. This is because the shared PC+PSR paradigm means you’ll frequently run into things like ORRing bit 28 in the PC which would have set the V flag, but will now set bit 28 of the instruction address. There’s no point having CLib even attempt to deal with this, it needs something like Aemulor or ADFFS.
Sadly, the DDE compiler, and CLib itself, is still stuck with FPE. If you use floats, you get FPE instructions in your code. It’s ludicrous when you consider the expense (in processing time) that we are still paying for Acorn’s reluctance to include FP hardware as standard given that every machine post-Iyonix has some sort of hardware FP inside. There’s snazzy hardware floating point (various types of VFP, some NEON) which can do all sorts of cool things…and we’re still depending upon a software emulation of an ancient FP chip.

The situation is likely to be similar with GCC (FP excepted), with the added fun of the AOF/ELF thing. While ELF may have attributes, it is not a native format (in any form) on RISC OS, so it needs something additional just to be able to load it. Maybe some day, there will be a non-GPL incarnation of the ELF loader that could be incorporated into RISC OS to make it more natively handled?

This brings us to BASIC. A program that worked on the BBC Micro will possibly still work on the Titanium (depends more upon what it wants rather than the language – does anything modern support a four-colour MODE?). Plain BASIC (non-FP/VFP) will even understand the special CALL addresses (&FFFx) to support OSWORD and the like from the BBC MOS.
It is also possible to write a BASIC program today on RISC OS 5 that will run on the Beeb. The degree of backwards compatibility is “variable”. For instance GCOL r,g,b won’t work on RISC OS 3.1 nor the ABC compiler, while SYS (and stuff like BEATS) won’t work on the BBC. That said, it’s worth noting well that pure BASIC not only entirely avoided the 26/32 bit issue, it entirely avoids questions of architecture – RISC OS runs on ARM, the Beeb used a 6502. But, hey, BASIC is BASIC…
[I’m deliberately ignoring The One True BASIC – you know, the one not written by Sophie Wilson that uses a different non-standard form of tokenisation ;-) ]

I would be grateful if someone could point me at any RISC OS documentation that clarifies these issues.

What, specifically, is your question? From the basic level, the “Program environment” chapter of PRM1 will tell you what RISC OS sets up. For C and BASIC, there is no real information as this is a language issue and code written in the language will benefit from these behind-the-scenes activities – for example, the argc/argv thing in main() that we all take for granted but never think how it is actually done, or the fact that 10GOTO10 doesn’t resemble any sort of instruction that any processor would recognise yet if your BASIC program starts with that, it works just fine.

The situation is likely to be similar with GCC (FP excepted), with the added fun of the AOF/ELF thing. While ELF may have attributes, it is not a native format (in any form) on RISC OS, so it needs something additional just to be able to load it.

There is of course a tool available to convert the GCC ELF stuff into a statically don’t-need-the-soloader-module RISC OS executable. elf2aif is its name. So what you describe is a non-issue when comparing GCC with DDE. The real issue that you cannot easily exchange AOF/ALF with ELF, so you cannot easily use the GCCSDK Autobuilder libs for Norcroft.

By the way, is there a good reference somewhere explaining AIF, AOF, ALF and others? I know http://www.riscos.com/support/developers/prm/objectformat.html, but I’m looking for something a lot shorter.

I tried to illustrate how you would approach a BASIC routine that was not fast enough and had to be translated into machine code for just the bit that needed speed. That was the !CountDn programme provided as part of the Raspberry Pi SD card image. It got the speed up from about 25s to 30cs to do 24 million comparisons (working out the solution for every possible target number given the tile selection just made so that you could select an ‘easy’ or ‘hard’ target as soon as the selection of the six numbers had been done).

but I’m looking for something a lot shorter.

Uh? It’s a number of binary formats, all that blurb is there for a reason! ;-)

What, specifically, is your question?

I think I mislaid it somewhere :(. What set me off was
the thought of how few are the tools in RISC OS for creating code. Also the notion of making a standalone version of a program that just ran on any RISC OS computer without having to ensure the presence of other resources. Programs generally need a heap for storing datastructures and a stack for implementing subroutines. A common convention is to have the stack grow down from the top of application memory and the heap grow upward; I think BASIC and GCC do this,
but Norcroft C doesn’t (stack chunks?). With previous versions of RiscLua I had an application Absolua which could scan a RiscLua program to see what libraries it used, bundle them all up with the main program, compile the bundle to bytecode and then prefix that with machine code for the Lua interpreter, using a bit of magic glue, and bingo you had a standalone Absolute file that should run anywhere. I cannot do that with the most recent version of RiscLua.
Your remarks about FP are very pertinent. It is quite frustrating. Perhaps we need a reincarnation of Craig-Wood (his Big Number module) to implement math modules of VFP and NEON code, offering convenient SYS calls. Fast Fourier transforms, continued fraction approximation of transcendental functions given by look-up tables of coefficients (which is how BBC BASIC does it), … in fact almost anything you are liable to find in Matlab or Mathematica. But what is the point of such a huge effort, when you can use Mathematica under Raspbian already?

Uh? It’s a number of binary formats, all that blurb is there for a reason! ;-)

I think there are many ways to “explain” a binary format. The link I posted is a reference documentation explaining every bit. I am looking for a more high-level explanation as an introduction to RISC OS development in general.

“AIF is the native format for RISC OS executables. AIFs are constructed by a linker like link or drlink from AOF or ALF files. An ALF is a bunch of AOF files put together. Compilers and assemblers like Norcroft C, GCC (up to version 3.×.x), ObjAsm, as and asasm create AOF format object files from source code files. Tools to process AOF files include A, B and C.”

But written by someone who knows about this stuff, not me!

the notion of making a standalone version of a program that just ran on any RISC OS computer without having to ensure the presence of other resources.

In some cases you can have that, in the form of statically linked library code, but, if you do, it comes with two costs: it’s much bigger than something that uses shared libraries, and, as a result, it will take longer to start up while the OS loads and relocates the code before it can begin to execute.

By the way, is there a good reference somewhere explaining AIF, AOF, ALF and others?

I found an overview article by Peter Naulls published on Drobe with a good introduction: http://www.drobe.co.uk/article.php?id=1236

A common convention is to have the stack grow down from the top of application memory and the heap grow upward; I think BASIC and GCC do this,
but Norcroft C doesn’t (stack chunks?).

Are you sure? The C runtime using the Shared C Library uses stack chunks, and this applies to both Norcroft and GCC when the latter links to the C Stubs. I don’t know what GCC does when linking with UnixLib, however. And what does Norcroft do if you use ANSILib?

I don’t know what GCC does when linking with UnixLib, however.

In the past it used to use chunked stacks – not sure if that’s still the case (I think it is, haven’t checked).

And what does Norcroft do if you use ANSILib?

Chunked stacks still.

While ELF may have attributes, it is not a native format (in any form) on RISC OS

!SharedLibs begs to differ.

Lots of programs, on all platforms, make use of shared libraries. There aren’t many that don’t. Again, there’s always an environment of sorts, whether it’s the BBC MOS or RISC OS or whatever…

The last “simple” executable programs were probably DOS .COM files and even they had an environment to make use of.

That said, there’s a tiny text editor on Linux systems, e3, which depending on how you call it, can emulate MicroEmacs (e3me), Wordstar (e3ws) and NEdit (e3ne). In about 3KB, not making any use of shared libraries. Don’t look for it on Raspbian, though, you won’t find it: it’s written in 8086 assembler.

I can’t see any reasons for running without support libraries if you are running binaries ontop of a OS.
If you really want to do something like that , just do something bare metal.

which depending on how you call it

Not unlike the embedded systems favourite “BusyBox”.

I can’t see any reasons for running without support libraries if you are running binaries ontop of a OS.

Where do support libraries stop and OS functions start? After all, CLib and Toolbox are part of the ROM…

The last “simple” executable programs were probably DOS .COM file

RISC OS transient utility? We’re still writing them today.

Where do support libraries stop and OS functions start? After all, CLib and Toolbox are part of the ROM…

I meant OS functions as well…