Cross compilation support

For those who don’t regularly check the CVS updates pages, I’ve just committed a bunch of changes that support cross compilation.

It started off because of work I was doing to the Norcroft toolchain. I wanted to be able to take advantage of development, analysis and debugging tools available on other platforms to assist in this task. Now, what is not commonly known is that most of the toolchain was always written with cross-compilation in mind – this should not be too surprising, as the RISC OS tools share a common heritage with ARM’s SDT toolchain (although ARM has moved on to new toolchains twice since then, with the ADS and then the RealView tools). Since we have to keep the source to most of the toolchain secret, the only hint of this in the public sources has been the “ccsolaris” directory in the squeeze sources – but believe me, it’s more ingrained in the secret sources. But the way it is implemented is by having separate buildsystems for each host OS and whether you’re using a cross or native compiler – a real maintenance headache. So I saw an opportunity to merge these together into a single unified makefile for each component, regardless of OS or toolchain, with immediate practical application. This seemed an ideal way to approach the completion of Pace’s cross-compilation project – an initial milestone on the path to full cross-compilation support. After all, it “obviously” couldn’t take all that long to do…

Well, after a few months of background work, I have now completed cross-compilation support for the CLX library and the following tools:

amu, aoftoc, bin2c, binaof, binasm, c++, cfront, cc, cmhg, ddt, decaof, deccf, diff, find, libfile, link, make, modgen, modsqz, objasm, objsize, resgen, squeeze, symedit, toansi, topcc, unmodsqz, xpand

• They all still build on RISC OS, targetting RISC OS, using the Norcroft toolchain
• They all now build on Linux, targetting RISC OS, using the Norcroft toolchain (this is new for some of the tools, and some of the others were suffering from neglect)
• Most of them build on Linux, targetting RISC OS, using the GCC toolchain (a completely new method of compilation). Exceptions are c++ and cfront (which compile but do not function correctly), ddt (which needs a special type of compilation that GCC doesn’t support) and make (because there isn’t an ELF version of RISC_OSLib available yet)
• Most of them build on Linux, targetting Linux, using the GCC toolchain. Exceptions are amu, ddt, make, diff, find and xpand because they are too tightly tied to running on RISC OS

When I say build, I mean these makefiles support all relevant build phases used in the RISC OS source tree, including clean, export and install. In fact in general, my approach has been to mirror the RISC OS build process as closely as possible, so there’s a shell script version of Env.!Common called Env/!Common.sh and a shell script version of the Env files themselves, but there are subtle differences required by the fact that other systems don’t share RISC OS’s global system variables, and instead have separate sets of environment variables for each process.

  $ cd path/to/RiscOS
  $ source Env/ROOL/CTools.sh
  $ cd Tools/Sources/diff
  $ make

This is close in spirit to the way a RISC OS build works. However, I’ve also come up with an alternative method which I think will be more practical for srcbuild to use when that gets converted (I anticipate it launching each build phase for each component in a separate process, which needs a single launch command):

  $ /bin/bash -c 'cd path/to/RiscOS && Env/ROOL/CTools.sh "cd Tools/Sources/diff && make -I path/to/RiscOS/BuildSys/GNUmakefiles"'

Obviously the former is rather more human-friendly, but it is still more typing than you can probably be bothered to do much of the time, so I reckoned we needed an equivalent of the !Mk, !MkClean, !MkExport and similar files you’ll see scattered through the source tree. These do handy things like changing directory for you, and running common build phases for you with common arguments, and are useful shortcuts for developers to use. So I’ve instroduced a mk.sh file into each component, which does a similar job. This even runs a likely Env file for you (although you can override that by specifying an alternate Env file on its command line). Example ways you might run a mk.sh file are

  $ cd path/to/RiscOS
  $ Tools/Sources/modsqz/mk.sh clean
  $ Tools/Sources/modsqz/mk.sh install ROOL/CrossTools
  $ cd Tools/Sources/modsqz
  $ ./mk.sh clean
  $ cd ../find
  $ ../modsqz/mk.sh

Remember that this is not supposed to be a replacement for srcbuild – I still foresee complete ROM or disc builds needing the centralised list of compile options and sequencing that you get from srcbuild’s Components file. But since we’re a long way from having a complete build working under cross-compilation, porting srcbuild remains a long way down the roadmap.

I hope others find this of interest.

Sounds good!

Are there any plans to release the Linux versions of the Norcroft tools, or are you more interested in shifting everything over to GCC?

But since we’re a long way from having a complete build working under cross-compilation, porting srcbuild remains a long way down the roadmap.

srcbuild is already mostly ported, isn’t it? I’m sure I saw some *nix-friendly code in there when I looked at the source a few months ago.

srcbuild is already mostly ported, isn’t it?

That was news to me – but then I haven’t looked at the srcbuild sources in years and I was never involved in the original Pace cross-compilation project so I never really got an idea of the planned roadmap. Clearly they had a different idea from mine about what’s a sensible order in which to tackle the problem!

Are there any plans to release the Linux versions of the Norcroft tools, or are you more interested in shifting everything over to GCC?

Well, the cross-compiling Norcroft tools have been built using GCC for as far back as I’m aware (since the Norcroft compiler itself only targets ARM). So once I had Linux makefiles that supported both Norcroft tools targetting RISC OS and GCC tools targetting Linux, it was relatively easy to combine the two to support GCC tools targetting RISC OS as well.

I don’t see us abandoning Norcroft tools any time soon. Castle wants to keep using it for official ROM releases, which pretty much requires us to keep support for it for the forseeable future. Plus, it would be a sad reflection on the platform if it could no longer maintain its “native” toolchain. And IMHO there are still some areas where the Norcroft tools excel – for example, more meaningful diagnostics that work better with the RISC OS throwback system, and a much nicer inline assembler syntax (see for comparison the implementation of bytesex_hostval here ).

That said, I also think it would be good for the long-term security of RISC OS itself for it to be buildable using more than one toolchain, but I can still see considerable effort being required to reach that point.

Regarding release of the Linux (or for that matter, Windows or Mac OS) versions of the toolchain, that’s technically feasible but contractually a bit of a grey area. Obviously it’s been a moot point until now, since the cross-compiling versions were suffering from lack of maintenance and wouldn’t build. Last time I talked to ARM about the tools, they were clear that they opposed any open-sourcing of the tools, but weren’t forthcoming about my requests to clarify their position about distribution of cross-compiling versions, on which the existing licence is ambiguous. Perhaps it is time I chased them about it again, especially since our relationship with them regarding the FPEmulator has been positive.

Are there any plans to release the Linux versions of the Norcroft tools, or are you more interested in shifting everything over to GCC?

...Regarding release of the Linux (or for that matter, Windows or Mac OS) versions of the toolchain, that’s technically feasible but contractually a bit of a grey area... Last time I talked to ARM about the tools, they were clear that they opposed any open-sourcing of the tools...

Do the original developers, Codemist have any link to the tools these days?

The toolchain was a joint development between Codemist, Acorn and ARM - Codemist were only really involved in the C compiler, not the associated tools like the assembler or linker, and Acorn primarily did porting work to RISC OS and various RISC OS specific tools like CMHG. The current licensing arrangement as I understand it is that Codemist exclusively licenced their work to ARM, and ARM sublicensed the RISC OS versions of the tools to Acorn – a licence which was subsequently transferred to Pace and then Castle. ROOL has an arrangement with Castle to develop and distribute the tools. Confused yet?

Neither Codemist nor ARM has contributed code changes to the tools since the mid 1990s. Major changes like C99 support and inline assembler tracked improvements to ARM’s ADS toolchain, but this was due to parallel in-house development at Pace and Castle instead of being as a result of continuous code drops from ARM as you might have assumed.