More slick installation of major updates

We have more progress!

Graham has agreed to add the following paths to the main distribution:

* Library = <Boot$Dir>.Library
* RO500Hook = <Boot$Dir>.RO500Hook
* Utils = <Boot$Dir>.Utils
* Printing = <Boot$Dir>.^.Printing
* Utilities = <Boot$Dir>.^.Utilities

When this happens, I’ll see about producing description files for everything else on the site, and then hopefully we’ll be ready to roll.

I have a few requests of someone at ROOL:

• Firstly, I do not expect everything to work perfectly first time. During the transition, an option to use either RiscPkg or the ‘old-fashioned’ system would be nice.
• Secondly, at present my package builder depends on the name of a given module zip file being equal to its title string. Can someone confirm if this is the case, or if it would be possible to make it so?
• Finally, would we gain anything from including another path called ‘Modules’ to hold all the components which can’t be softloaded, or should we hold off packaging those altogether?

The new version of RiscPkg is in! It contains all the paths above. The next thing to do is get those descriptor files produced and update my program to work with things other than applications.

I find the complete silence of ROOL on this thread rather worrying. I need your cooperation! We need to agree on a base URL to download packages from, and a location to store the descriptor files for each built component.

I find the complete silence of ROOL on this thread rather worrying.

Don’t worry, I don’t think it’s anything personal ;) As I understand it their day jobs are still eating away all their time/energy, to the point where they haven’t even had enough time to keep up with the forums recently, let alone rub enough braincells together long enough to come up with any answers to your questions. If you’re at the point where you can’t go any further without their input then I’d say firing off an email is your best bet.

Do you have a copy of the C/C++ tools, and thus the ability to run !Builder and build stuff yourself? Since you seem to be the person with the most free time and motivation with regards to making everything RiscPkg friendly, I have a feeling that any required changes to the build system, and the ‘live’ testing of the RiscPkg-ing will have to be done by you yourself.

As I understand it the process of creating the individual module/application zipfiles is currently a rather manual process, so you’ll probably want to talk to ROOL about exactly what steps they go through at the moment, and whether they had any plans/ideas on how they would extend the build system to make it fully automated. ROM images, on the other hand, already have a fully automated packaging process (not that that’s likely to help you much, since RiscPkg-ing a ROM image doesn’t make much sense!). I’ve had some experience with the innards of !Builder/srcbuild and the autobuilder, and how the build process works in general, so I can probably provide help if you get stuck with any of the technical bits.

Trust me, we’re all still around… the person who is most able to contribute his time to ROOL at the moment is Andrew, and his priorities are primarily the web site and other underlying infrastructure. I’m sure he’d have an opinion on some of this topic so I agree with Jeffery that firing us an email would help.

I keep finding myself working in Europe at the moment so I’ve not been able to do much other than a few autobuilds and C Tools orders here and there. And Ben was looking like he’d have some free time for ROOL but a new contract has come along starting on Monday :S

However, we are going to be doing as much as we can leading up to the Wakefield Show so hopefully we can clear some of the backlog of ToDo items by then…

My 2p on this stuff – once you’re happy with your package format then I would probably be able to help create a build process to create all of the main packages from source.

I agree that we should continue to distribute in the existing style for a transitional period, until we’re all happy that using a package manager is working properly. I’d also like to know that the GUI for the package manager is simple and obvious – ideally being one of the tools in amongst the Configure plug-ins.

I really don’t want to see (too many) individual modules packaged up – maybe just the main ones like CLib and Toolbox-related to start with. A lot of the ones on our site don’t (easily) softload so they would be more trouble than they are worth as packages.

Right, I know I’ve been silent for a while, but I’ve just finished MBBack 2.00, which was taking up most of my time for the past two weeks or so. I am now going to have a good crack at creating the descriptor files and updating my packager to use the new paths.

I don’t own a copy of the C tools, nor have I ever programmed in or compiled C, so I’m afraid I won’t be much use in the actual integration of the packaging, but I should be able to produce a command-line tool which will produce a package zip file from its components with the minimum of fuss.

At present my criteria for stuff which may be packaged is ‘anything not in the Modules directory’.

Off to start now…

Helloooo! The ChangeFSI zip file is still broken!

Right, I’ve now emailed the result of my efforts to info@riscosopen (two days ago now – whoops!). I have a command-line program which takes as its input a file or application to be packaged, an index file to add the new control record to, a special ‘descriptor’ file which lists the RiscPkg-specific characteristics of a program, and the version number of the program in question.

The most pressing issue at the moment is where the descriptor files are to be stored. I always assumed they would have to be uploaded to the CVS repository somewhere, as in the event they have to be changed this would certainly make things easier for the developers. Somewhere a database will have to be maintained of all the built components and their corresponding descriptor files.

I have limited myself at present to the applications in the three download sections which are not modules, with the exception of ChangeFSI, because (deep breath) THE ARCHIVE IS BROKEN.

The next step should be the insertion of an extra step into the build process to take the application which the file was built into, look up its current version number and location of the descriptor file, and call my program via the command line.

Regarding ROM updates (I nearly started a new topic – ROOL, please move this if appropriate):

1. Is the Iyonix Update Watcher suitable in its existing or modified form?
2. Is RiscPkg suitable and appropriate for updating ROM images?
3. Do new ROMs often require new boot sequences to function correctly and if so, could they invoke a reboot to a previous image in the absence of a new boot sequence?
4. Presumably it’d be best if there were a common application for use with all possible hardware setups.
   • Is this feasible and if so, how would such an app offer customisation for different h/w setups?
   • e.g. setup files containing h/w specific means of reading/writing an SD card or similar
5. Essential (IMHO) to cater for safe handling of “fallback” to previous (and/or default) image install
6. What about ROM image download naming conventions?
7. And ROM image locations? e.g. hardware vendors could point to their own URL via the above app, which could then redirect to the correct part of the ROOL site. Mirrored images could possibly also be provided on vendors’ own sites.

1. Is the Iyonix Update Watcher suitable in its existing or modified form?

It’ll need at least a little bit of modification (and I’m not sure that the source is even in CVS yet), but it should be suitable. It was used to distribute ROM updates in the past, right?

2. Is RiscPkg suitable and appropriate for updating ROM images?

I believe RiscPkg is designed purely for software which you can add to/remove from your hard disc. It wouldn’t be very easy for the RiscPkg package manager to handle firmware updates – it would have to know how to talk to each piece of hardware to find out what the current firmware is. And if you’re not using the RiscPkg package manager, then there’s little point in using RiscPkg archives.

3. Do new ROMs often require new boot sequences to function correctly

Yes and no. Any ROM/module updates are always designed with backwards compatability in mind, so except for major changes you should be able to update your ROM image without updating your disc image (although you’d obviously miss out on any improvements made to disc-based components)

However I do know that the current ROOL boot sequence checks the ROM version and complains if the ROM is too new. If you removed the ROM version check then it would probably boot OK, but it needs to know the ROM version so it can copy the right ‘hook’ folders over (they contain some softloadable modules and other ROM version specific tweaks) I think the next time the boot sequence will start complaining is when we hit RO 5.20, but I may be wrong (I think the only time I saw it complain was when I was trying to get the boot sequence working on my BeagleBoard, but some of the required files were missing from CVS)

and if so, could they invoke a reboot to a previous image in the absence of a new boot sequence?

It would probably be far too difficult to design a ROM image to do that. If you wanted you could probably do it as part of the boot sequence (e.g. the first thing the boot sequence does is check the ROM version, and then softloads an older ROM if necessary). But then, what’s the point of updating your ROM image if your boot sequence is going to forcibly downgrade you? It would be much better to have some metadata with the ROM image which allows the ROM programmer to check that you’ve got the right boot sequence installed prior to the update.

4. Presumably it’d be best if there were a common application for use with all possible hardware setups.

That would be good, yes. We’d need to make some changes to the HAL/OS so that the ROM programmer can determine the machine type. That way it can check that you’re programming the right ROM for your machine, and (unless the ROM programming functionality gets built into the OS) it can use the right routines to program the new ROM image. For something like OMAP, where the ROM could have come from one of several boot sources, we might also need some way for the HAL to tell the ROM programmer which boot source was used, so that the new ROM can be written to the right location.

5. Essential (IMHO) to cater for safe handling of “fallback” to previous (and/or default) image install

This could be a bit tricky, for a number of reasons. Verifying that a ROM has been programmed correctly is quite easy, and is something that I’d expect the current Iyonix ROM programmer to do. However, dealing with situations where the machine fails to boot due to a corrupt ROM is a bit trickier.

First, you’ve got to detect that something is wrong. The Iyonix ROM currently does this by checking the ROM checksum; if the checksum fails, it will flash out an old-fashioned POST fail message using the floppy drive light and wait for you to upload a new ROM image over the serial port. Not exactly user-friendly, but if you’ve got a second machine to connect to it with then it should make the Iyonix unbrickable. Of course no matter how many checksums you use, you can’t always detect that the machine is going to crash. The best way to deal with crashes is with watchdog timers. If a watchdog timer ever reaches zero, the machine will reboot, so watchdog timers are ideal for situations where the machine gets stuck in an infinite abort/interrupt loop. The ROM init code then just has to check the watchdog status to find out whether it was a normal boot or a watchdog reboot, and load the appropriate ROM image. I know the OMAP has plenty of watchdog timers, and I’d expect the IOP to have at least one or two, but I don’t think there were ever any in RiscPC hardware. So using watchdog timers might not be possible for all hardware. But even without watchdog timers, as long as you’ve got access to somewhere to write a boot status value (e.g. write it to CMOS), you should be able to detect and deal with most serious failures (e.g. critical ROM modules that fail to initialise, or data aborts during ROM initialisation)

Once you’ve detected that something’s wrong, you’ve got to work out what to do about it. As mentioned above, the Iyonix will try to get the user’s attention over the serial port, and allows them to upload and program a new ROM image. But this isn’t very user-friendly, so it would be nice if there were alternate approaches available. Storing two (or more) ROM images in flash would be one way of doing it, that way it can automatically fall back to the last known good ROM image. But AFAIK the Iyonix only uses a 4MB flash ROM, so the images would need to be compressed – and judging by the download sizes on the ROOL site, it would be a tight squeeze to fit in two Iyonix ROM images. Another alternative for the Iyonix would be to include some minimal filesystem code so it can mount the hard disc and automatically load a ROM image from there. It might be a bit tricky to extract the relevant filesystem & hardware I/O code from the OS, but it’s definitely doable. But if you’re storing ROM images on your hard disc then you’re opening yourself up to attack interference from the pesky users – you’d need some way of protecting the backup ROM image so they can’t delete it.

2. Is RiscPkg suitable and appropriate for updating ROM images?

I believe RiscPkg is designed purely for software which you can add to/remove from your hard disc. It wouldn’t be very easy for the RiscPkg package manager to handle firmware updates.

It could always install a complete updater in the utilities folder. The user would then run this manually. (Which is probably best anyway).

Another alternative for the Iyonix would be to include some minimal filesystem code so it can mount the hard disc and automatically load a ROM image from there.

Isn’t this pretty much what the beagleboard does all the time?

It wouldn’t actually have to use the OS code, would it?

It would just be a bootloader that offers a compressed image in the rest of the ROM as one option.

Isn’t this pretty much what the beagleboard does all the time?

It wouldn’t actually have to use the OS code, would it?

It would just be a bootloader that offers a compressed image in the rest of the ROM as one option.

Yes, this kind of thing would be easiest on the beagleboard. The problem is more about the platforms where it isn’t that easy, e.g. the Iyonix. If we wanted to we could try porting u-boot to the Iyonix, and use that as our bootloader. That way we’d be able to easily load backup ROM images from hard disc, or from USB drives, etc.

Does a bootloader already exist for the Iyonix? (or more correctly, the board it is based on.)

Presumably, u-boot would either need ADFS support added or RISC OS 5 would need support for partitions added.

Does a bootloader already exist for the Iyonix? (or more correctly, the board it is based on.)

Wikipedia says RedBoot is the standard bootloader used with XScale boards. The RedBoot website lists a couple of IOP321 boards supported, one being Intel’s development kit. So if Castle based the basic motherboard design around Intel’s then it probably wouldn’t be too hard to get it to run on an Iyonix.

Presumably, u-boot would either need ADFS support added or RISC OS 5 would need support for partitions added.

Yes. In a few months time when I’m finished with my current tasks I’ll probably have a go at adding support for DOS partition tables to filecore, along with finding a way of getting DOSFS to work with partitions over 2GB. Unless someone gets there before me ;)

Thanks guys – very comprehensive.

...complete updater in the utilities folder. The user would then run this manually. (Which is probably best anyway)...

Does a bootloader already exist for the Iyonix? (or more correctly, the board it is based on.)

...RedBoot... So if Castle based the basic motherboard design around Intel’s then it probably wouldn’t be too hard to get it to run on an Iyonix.

I’d initially been thinking along the lines of an updater outside of the RISC OS environment. But it’d be much better if everything could be handled from within RISC OS. If it makes managing the process simpler for new hardware too, then so much the better. Thanks.

In a few months time when I’m finished with my current tasks I’ll probably have a go at adding support for DOS partition tables to filecore, along with finding a way of getting DOSFS to work with partitions over 2GB. Unless someone gets there before me ;)

This is excellent news. But would fat32fs (http://jeffd.drobe.co.uk/) be usable for DOS partitions?

Possibly. I haven’t really checked what features DOSFS and fat32fs do/don’t have. Replacing DOSFS with fat32fs may end up being more work than just fixing DOSFS!

Support for partitions even without big DOS partition support would be a big step forward.

(Would that allow the BeagleBoard to run RISC OS from the internal SD?)

As I understand it DOSFS is an Image filesytem, which (presumably) means it treats a disk as though it is an ordinary file.

Does this mean to fix it you would have to fix the 2GB file limit and then modify filecore and DOSFS to use the new system?

...partitions… (Would that allow the BeagleBoard to run RISC OS from the internal SD?)

If so, it could be a convenient way for !Boot and the main installed apps to be kept separate from users’ data (stored on another device, e.g. flash drive, USB HDD, NAS).

Helloooo! The ChangeFSI zip file is still broken!

I finally bothered to look into this and upload a fixed version. :)