Compressed ROM images

Email me at info@riscosopen.org and I can send it you that way.

On a related note, I got the “Live RISC OS” ROM build working on a BeagleBoard – it boots entirely from a disc image in ResourceFS, with temp files held in RAMFS. This is great, but the ROM is 28MB in size and takes a bleedin’ long time to load!

Linux also takes ages to load. Is there a simple way to put something on the screen quite quickly that will give the user something to look at? Even just a coloured background with ‘Loading RISC OS’ would do. A simple progress bar would be even better. The user will be fooled into thinking that something important and time consuming is being done to prepare the computer for use rather than just a slow SD card interface doing something simple and quick but taking forever!

All problems are also opportunities. I have 256Mbytes NAND flash on my Beagleboard XM/ARMini (it was a series A board) and so could easily make my machine boot very quickly. This gives me a thought. How easy would it be to create a 16Mbyte plug-in ROM for the Raspberry Pi (don’t know what headers are available on that board). I presume that is too difficult. So there’s a commercial opportunity here – get the uboot/fatload bit speeded up and offer users a quicker booting RISC OS at a price.

Once you get into something that is not shipped on the SD card with the RPI then it is obviously simpler just to sell a bootable native RISC OS pen drive. A special one intended for the RPI should offer the option for the user to put a plain ROM image on the SD card in place of the large image containing the boot structure.

Dare I suggest therefore that the start up banner could say ‘if you want RISC OS to boot more quickly then purchase a native ROOL pendrive’. An advert, no less.

Linux also takes ages to load. Is there a simple way to put something on the screen quite quickly that will give the user something to look at?

Possibly. You can put memory read/write commands in u-boot scripts, so it would be possible to create a script which programs the display controller for a standard mode (e.g. 640×480 or 800×600) and then loads an image for it to display. However I’m not sure how compatible it would be with all the different beagleboard revisions – one of the things that often gets changed when they revise the board is the way that the power to the DVI interface is controlled.

So I think that if we want to get serious about reducing boot times or showing splash screens then I think the only way to do it would be to work on improving the u-boot source.

A simple progress bar would be even better.

I guess that would be possible as well, if you were to split the ‘live’ image up into several chunks and load new progress bar images inbetween each chunk. (Or much easier if we were working with the u-boot source)

This gives me a thought. How easy would it be to create a 16Mbyte plug-in ROM for the Raspberry Pi (don’t know what headers are available on that board). I presume that is too difficult.

From what I’ve read, I don’t think the memory bus is exposed on any of the expansion headers, so I don’t think this would be possible.

So there’s a commercial opportunity here – get the uboot/fatload bit speeded up and offer users a quicker booting RISC OS at a price.

Remember that u-boot is GPL. You can sell it, but anyone who buys it has the right to request a copy of the source code and release their own version for free. So you’d get one or two sales and then everyone would switch to using the free version.

The BCM2835 on RPi has (I assume) a first-stage bootloader in ROM sufficient to boot the GPU from a firmware image stored on SD, and the GPU bootstraps the ARM. It doesn’t use u-boot, and the code for the multi-stage boot is all from Broadcom so there’s no source. I’ll see if I can try padding out a jumbo Linux kernel and seeing how long that takes to boot.

Theo: at present, our “Live RISC OS” ROM is 20MB but this seems likely to grow towards 32MB as we add third party components into the ‘disc’ image.

Steve: You may have thought of this already, but remember that with the current ROM build system there’s a hard limit of 32MB on the ROM size, due to the range of the B/BL instructions (which C modules would be using for SCL static linking). So don’t go too crazy with adding extra components to the image :-)

In the future I guess we could allow for bigger images by moving the ResourceFS data blob to the end of the ROM image, past all the code sections.

Indeed. 32MB was our hard limit. Also, that’s the max size for the FAT12 partition (IIRC) so we’d likely stop short of there to ensure there’s a bit of room for the other files.

On a related note, I got the “Live RISC OS” ROM build working on a BeagleBoard – it boots entirely from a disc image in ResourceFS, with temp files held in RAMFS.

One possibility is to load the rom and the contents of ResourceFS separately [the CMOS setting determining the boot drive but the booting process deciding whether to spend the time fatloading a large resourceFS from the SD card). Hence the SD card contents (the boot process and the file ‘cmos’) determining the set up. The purpose of having everything for a full boot present on the SD card is (AFAIK) for the RPI user who has not yet committed to RISC OS sufficiently to get a RISC OS bootable pen drive fitted. A sort of omni-present RISC OS to tempt users to use it. Things like network settings may need to be adjusted once and having ‘riscos’ and ‘resourcefs’ fatloaded separately from the SD card allows the user to alter the boot image (by altering ‘resourcefs’ on the SD card) to suit personal preference. Presume CMOS RAM on the RPI will have to be a file ‘cmos’ on the SD card and a ‘!WriteCMOS’ utility in the boot image within ResourceFS?

Network settings are probably the most obvious candidate for needing tweaking but being tucked away in ResourceFS where the user will not be able to alter them.

Possibly. You can put memory read/write commands in u-boot scripts, so it would be possible to create a script which programs the display controller for a standard mode (e.g. 640×480 or 800×600) and then loads an image for it to display. However I’m not sure how compatible it would be with all the different beagleboard revisions

This comment about putting something on the screen was made with only the Raspberry Pi in mind, not the Beagleboard. It is the potential volume of the RPI and the possibility of attracting new users that would make an attractive and tempting user interface worth investing effort and the key time is when trying RISC OS for the first time. You only get one chance to make a first impression!!

So I think that if we want to get serious about reducing boot times or showing splash screens then I think the only way to do it would be to work on improving the u-boot source.

The BCM2835 on RPi […] doesn’t use u-boot, and the code for the multi-stage boot is all from Broadcom so there’s no source.

(For systems using u-boot) I don’t know where this Pandaboard project is, but hasn’t this already been looked at?

[Edit: There’s also barebox (link to docs saying OMAP supported).]

Terribly inaccurate timings with RPi Linux from release of reset button to Linux penguin:

• kernel.img size 4867904 bytes, boot time 5.5s
• kernel.img size 20971520 bytes, boot time 9.9s
• kernel.img size 33554432 bytes, boot time 13.8s
  courtesy of the stopwatch on my phone.

SD card a 2GB MMAGF02GWMCU (freebie from Terasic), reading via dd bs=1048576 to /dev/null on Linux in a USB reader says:

• 2021654528 bytes (2.0 GB) copied, 91.8212 s, 22.0 MB/s (the whole card)
• 33554432 bytes (34 MB) copied, 1.92318 s, 17.4 MB/s (32MiB kernel.img)

For comparison, a RPi Debian distro takes 24.4s to boot to a login prompt on the same SD card (which has a pretty high latency and so isn’t great for being used as an ext3 filesystem).

(Error of these measurements at least a second either way)

I’ve now got a working ROM compression system, supporting the squeeze & zlib algorithms. Some quick stats:

• Using an Iyonix as the build machine, a 4MB OMAP ROM can be compressed in 20 seconds using squeeze, producing an image just under 3MB in size.
• The same ROM compresses in 30 seconds using zlib, producing an image of just over 2MB
• Decompression time is quick, both on the Iyonix and BB (I don’t have accurate timings yet, but the BB seemed to decompress the zlib image in under a second)
• Using the zlib algorithm the 20MB OMAP live disc image can be compressed to 8.5MB, in about one and a half minutes. Decompression time (on the Iyonix) is 5 seconds. I haven’t tried booting the image on the BB yet, so I’m not sure how long decompression will take in practice (probably just a couple of seconds).
• Squeeze gave up trying to compress the live disc image – it’s too big/complex to be compressed using the single set of lookup tables the algorithm uses. It might be interesting to create a chunked version of the algorithm to see how well it performs if isn’t restricted to using one set of lookup tables for the entire image.

In a day or so I’ll add some timing code to the startup sequence and try booting the live image to see exactly how long things are taking and how much of a difference the compression makes.

At the moment it’s only the OMAP HAL that I’ve modified to support compressed ROM images. Adding support to the other HALs should be pretty easy, but the softload & romflash tools still need updating to cope with compressed images. I also need to make the ROM compressor add a checksum to the images in order to keep the Iyonix HAL happy.

On a related note, I got the “Live RISC OS” ROM build working on a BeagleBoard – it boots entirely from a disc image in ResourceFS, with temp files held in RAMFS

Will there be a version of this for IOMD?
(It occurs to me that it would need to ignore some of the CMOS settings on the host).

This would probably be the only way that RO 5 could be tried easily on an in-use Risc PC. (Until partion support is added).

Just as an aside, Linux has just mainlined Google’s Snappy compression, which claims to do 500MB/s decompress and 250MB/s compress, on a single core of an i7, 10x faster than zlib but with a 20-100% size penalty. It’s in C++, but doesn’t look much work to rework to plain C. BSD licence.

Some timings for booting the OMAP live image on a BB-xM:

• 7.2 seconds from power on to when u-boot starts to load the ROM image.
• 22.1 seconds to load the uncompressed (20MB) ROM image (905KB/sec)
• 1.9 seconds for the HAL to copy the ROM image to the high end of RAM (10.5MB/s)
• 0.9 seconds for the HAL to use DMA to clear the remaining 492MB of RAM (546MB/s)
• 0.4 seconds until the keyboard scan begins (basic kernel + HAL init)
• 8.2 seconds until the keyboard scan finishes
• 5.6 seconds until the RISC OS startup message appears (remaining kernel/HAL initialisation, module initialisation)
• 17.1 seconds until the desktop is entered (although the boot sequence still has a few seconds left at that point)
• In total, it took about 1:08 from power on to a functional desktop.

For the compressed image, relevant timings are:

• 9.3 seconds for u-boot to load the (8.5MB) ROM image (about 900KB/s still)
• 0.8 seconds for the HAL to copy the ROM image to the high end of RAM (about 10.5MB/s still)
• 0.9 seconds to decompress the ROM
• Total boot time saving is 13.03 seconds

The timings were taken using a 32kHz system timer, and were repeated twice for extra accuracy. The only thing which wasn’t measured this way was the ‘time to functional desktop’, where I just relied on !Alarm’s second count.

A few extra notes:

• u-boot spends 3 seconds waiting for user input before doing anything. Ordinarily this delay is configurable, but I’m not sure how you’d change it on a BB-xM due to lack of NAND. Maybe it can be set in the uenv.txt file.
• Making the HAL use DMA to copy the ROM image to the high end of RAM should reduce that time to practically zero
• The keyboard scan should take 5 seconds from start to finish, but at the moment it’s using a standalone timed loop instead of one that takes into account the other things that happen between the scan init & shutdown (e.g. CMOS cache initialisation & ROM decompression)
• I haven’t looked into the long delay at the start of the boot sequence

u-boot spends 3 seconds waiting for user input before doing anything. Ordinarily this delay is configurable, but I’m not sure how you’d change it on a BB-xM due to lack of NAND. Maybe it can be set in the uenv.txt file.

Possibly. How does that fit in with bootenv=boot.scr?

Great work, Jeffrey! This is all looking like worthwhile changes. I reckon there’s the possibility of getting RISC OS to boot from power-on in under 40 seconds – towards 30 seconds. That’d be fantastic.

Which reminds me, I (or someone) should look at the DHCP module – it’s bad enough that it stops the boot sequence while waiting for the DHCP response, but it’s even worse that it sits there for bleedin’ ages if none is forthcoming. In an ideal world, the Live RISC OS image would come pre-configured with the right Ethernet module(s), AutoSense files and network stack config set up to DHCP. That’d be the right answer for most people but you don’t want RISC OS to fail to boot because of it…

That would be very cool.

What would be even better is that if it can’t find a DHCP server it could revert to an IP address that you could set manually in Configure.

I’m not promising anything because there are other, higher priority things for me to look at, but it would be nice… :)

> What would be even better is that if it can’t find a DHCP server it could revert to an IP address that you could set manually in Configure.

Or use the same system windows does, if no such address has been configured.

Some timings for booting the OMAP live image on a BB-xM:

* 7.2 seconds from power on to when u-boot starts to load the ROM image.
* 22.1 seconds to load the uncompressed (20MB) ROM image (905KB/sec)
* 1.9 seconds for the HAL to copy the ROM image to the high end of RAM (10.5MB/s)
* 0.9 seconds for the HAL to use DMA to clear the remaining 492MB of RAM (546MB/s)
* 0.4 seconds until the keyboard scan begins (basic kernel + HAL init)
* 8.2 seconds until the keyboard scan finishes
* 5.6 seconds until the RISC OS startup message appears (remaining kernel/HAL initialisation, module initialisation)
* 17.1 seconds until the desktop is entered (although the boot sequence still has a few seconds left at that point)
* In total, it took about 1:08 from power on to a functional desktop.

I have improved on this slightly as I have a series A Beagleboard XM ARMini with 256Mbytes NAND flash as well as 512Mbytes RAM.

By issuing the commands:

OMAP3 beagleboard.org # fatload mmc 0 0×81000000 riscos

reading riscos

4194304 bytes read

OMAP3 beagleboard.org # nand write 0×81000000 280000 400000
NAND write: device 0 offset 0×280000, size 0×400000

 4194304 bytes written: OK

OMAP3 beagleboard.org # setenv bootcmd “echo Booting from nand …; run nandargs

; nand read 0×81000000 280000 400000; go 0×81000000;”
OMAP3 beagleboard.org # saveenv

Saving Environment to NAND…

Erasing Nand…

Erasing at 0×260000 — 100% complete.

Writing to Nand… done

OMAP3 beagleboard.org #

total time to RISC OS beep 27 sec, desktop at 1:10 (the network takes a little while to set up).

However even loading from NAND flash it takes 3.5s to load the 4Mbytes RISC OS ROM image.

It strikes me that one way forward for the Rapsberry Pi is to start up in Noboot, language=BASIC. That should get the start up time to about 30 seconds. Also (hopefully) we can load an image into memory and get the DVI interface working to display it even earlier. Then we can use the ‘TWIN’ command (which from BASIC attempts to do a ‘*’ command *TWIN) to do the boot – we can even make this make sense as the command ‘TWIN’ would then be starting up the ‘alter ego’ a RISC OS desktop!

The advantage of this approach is that the Raspberry Pi would then start up looking almost exactly like a BBC Model B computer – but with a bit more memory.

How about:

RISC OS 256MB

Cortex-A8 Processor

Acorn SCSIFS

BASIC

Type >TWIN to start up the RISC OS Desktop

ARM BBC BASIC V version 1.48 © Acorn 1989

Starting with 256155900 bytes free

>

ISTR reading something on the RPi forum about booting to the Linux command line rather than GUI. Your idea fits in very well with this.

Chris, That’s such a cool retro idea. The great thing is, BBC Basics already in the ROM, so it’s easy to do. I’ll stick a link to your post on RISCOScode so others can find there way here and comment if they wish to. Regards, Martin. http://www.RISCOScode.com

It is a cool idea. I wonder how many people’s memories that would jog! How many people’s fingers would, involuntarily, type:

10 PRINT "You smell!"
20 GOTO 10

;)

On a more serious note, and probably showing my ignorance, wouldn’t *DESKTOP do the job of getting you into the desktop rather than hacking something onto “TWIN”?