Partition support in RISCOS

As multiple partition support is common to operating systems, they also exist on exchangeable media in the wild.

To be able to use the BeagleBoard internal SD card as a bootable device for RISCOS itself but also to boot riscos from it, we need partition support too.

I am sure that there are some ideas out there how to add this to riscos in a favourable way – so please comment.

• Compatibility with media, with riscos
• Expandability, ease of use (e.g. should a partition get its own drive symbol on the iconbar given we have only 4 max.)
• handling of non supported partitions/recognition/failure of doing so
• Formatting/Progam support
• Handling of disc images
• Alternatives (e.g. Handling of pc ‘partition files’ in the old days <-> alternatives like looking for riscospartition.dat in root directory of FAT32 and use that as riscos partition file)

A brief overview of the RISC OS filesystem stack:

• At the lowest level you have the hardware drivers. E.g. ADFS, SCSIFS, RAMFS.
• These use a simple block-based API to interface with FileCore.
• FileCore provides the implementation of the disc format commonly referred to as “ADFS” (in the days of the BBC it was correct to refer to something as being ADFS formatted, but in the RISC OS world it’s more correct to say that it’s FileCore formatted)
• If FileCore doesn’t recognise a device as being FileCore formatted then it treat the entire device as a file and offers it out to any image filing systems (this is how DOSFS works with FAT formatted floppies & USB sticks)
• Then there’s FileSwitch, which is the final layer which sits ontop of FileCore (or DOSFS, or filesystems like DeviceFS which don’t rely on FileCore). FileSwitch ties everything together so that any file on any filesystem can be accessed using the same common APIs.

So to add partition support we’d need to insert a new level, somewhere near to where FileCore is currently sitting. The code may even go inside FileCore itself.

I think it’s important that we design the new system right, and in order to do that it might make sense to tackle some of the other problems with FileCore at the same time (e.g. the 4 floppy + 4 hard disc limit, or the 2GB image filing system limit). Designing a new filesystem stack which meets our needs is easy, but designing one that retains an acceptable degree of compatability with the current stack (and doesn’t require us to spend ages rewriting all of RISC OS’s filesystem code!) is likely to be a lot trickier.

Random thoughts…

• a “drive” is the only thing currently known to RISC OS that would properly map to a partition. Consequently, all previous partitioning schemes used this mapping
• mapping could be user-controlled, to be able to unmap unused/unformatted partitions
• extending RISC OS to support more than 4 (or 8) drives per FS would be infinitely more useful than trying to do a clever workaround for the “drive is a partition” scheme
• ways around the “crowded iconbar problem” must be found anyway
• achieving compatibility in filecore world is difficult (every 3rd party IDE/SCSI interface used its own partitioning scheme), but not impossible to work out. It would take probably a day or two to find out the most-used partitioning information (Simtec/RiscStation IDE, APDL/MicroDigital IDE, RapIDE, Castle SCSI, Power-tec SCSI, Cumana SCSI)
• if one of the existing schemes works (i.e. has enough headroom for the future – i.e. partitions of Exbibyte size), declare it as standard – else, roll our own
• if one of the existing schemes collides with another, write a converter

Writing a generic formatter (not the usual “roll your own HForm”) would be a good idea anyway, adding partition format for such a new utility would be easy enough.

And while you are at it, restructuring the Fileswitch/Filecore/hardware driver stuff would be good, too – as well as extending Fileswitch for 64bit filesizes of course…

I’ll probably make a complete fool of myself by even mentioning this but I just thought of the SyQuest drive on my RO 3.11 box. Since the 1 GB disc isn’t supported by the OS, the workaround was to divide it into two hdd. APDL should know how they did it if that’s any help.

• !Scrap, log, configuration etc. files possibly being located separate from !Boot
• putting the boot sequence onto the SD card may be an issue with regard to the limited number of writes to flash SSD devices, particularly if OS/user files aren’t stored elsewhere
• cross-reference to !boot – does it have to keep the same name? (partitioning on the SD card could potentially open up some further options)
• cross-reference to booting RISC OS from linux with kexec (possibly making it easier to have multi-boot setups – via a suitable boot script, as Chris Hall has already done)

if one of the existing schemes works (i.e. has enough headroom for the future – i.e. partitions of Exbibyte size), declare it as standard – else, roll our own

Well since SD cards for use with Beagleboards/etc. must have a FAT partition/partition table, the obvious solution is to do what everyone else does and adopt the use of the MBR partition table format (and the newer GUID partition table for all your exabyte/zettabyte needs).

Of course it would be nice to be able to support all the different custom partition table formats that have been used by IDE/SCSI cards, so it would be advantageous to design the system so that additional partition managers can be used. E.g. if it’s not an MBR partition table and it’s not a GUID partition table, send out a service call to see if it’s a 3rd party partition table. If that fails, check if it’s an ordinary old-style FileCore disc. If that fails, do what FileCore does and ask if any image filing systems recognise it.

Depending on how the system is implemented, and how the different 3rd party partition tables work, it may make sense to have different priority levels for the detection routines, perhaps with the priority levels specified on a per-driver or per-drive basis. E.g. Simtec’s partitioning scheme may be fundamentally incompatible with APDL’s, leading to false positives – so anyone wanting to plug a Simtec or APDL partitioned drive into their Iyonix’s motherboard would need to manually specify the partition table type (disregarding the fact that there’s currently no CMOS RAM left to allow for important settings like that to be saved)

Extending FileCore for 64-bit filesizes is (I assume) relatively simple, but compatibility will be a nightmare. I think the only option will be to implement a set of new reason codes in OS_File et al. which return length/offset/whatever in two registers (rather like what was done for FileCore_DiscOp64). I shouldn’t worry too much about legacy software – it’s managed fine up to now, and I can’t imagine more than a few specific cases requiring it anyway, and any pre-existing examples of those must already by definition have their own workarounds.

If someone reads a 64-bit file using the old API, we could choose either to mask out the high bits or just return an error.

Extending FileCore for 64-bit filesizes is (I assume) relatively simple

Relatively simple, until you remember that FileCore is 100% assembler and has been in active development for around 10 years.

Of course, the good news is that there’s no requirement for us to add support for 64 bit file pointers (or >256GB partitions) at the same time as we add partition table support.

mapping could be user-controlled, to be able to unmap unused/unformatted partitions.

RISC OS could mount only the boot partition by default, and have all the other available (fixed) partitions available via Resources:$.Discs as network drives are currently.

(Automatically mounted drives could be added via configure)

do what everyone else does and adopt the use of the MBR partition table format

The Castle Storm supported standard partitions (when 8GB was a typical limit on PCs, so it may have similar limits).

Might Castle make the sources (or specification) available for reference, if not a starting point?

so anyone wanting to plug a Simtec or APDL partitioned drive into their Iyonix’s motherboard would need to manually specify the partition table type (disregarding the fact that there’s currently no CMOS RAM left to allow for important settings like that to be saved)

A Mico partitioned disk appears to be a standard format to partial capacity. And then other partition(s) taking the rest of the space. The standard partition appears to work normally on a motherboard IDE interface.

Might it make sense to require such schemes to be explicitly enabled in configure, if they are incompatible with each other? ie by something loaded in the boot sequence.

Since a current RO 5 machine would be unable to access any extra partitions, I can’t see any loss if RISC OS were only able to boot from MBR/GUID partitions or a current format disk.

However it would make sense to be able to boot from one type of secondary partition on a system where the first partition is visible to a current machine. (Otherwise softloading RISC OS 5 on 3.x machines would be somewhat limited.)

Might it make sense to require such schemes to be explicitly enabled in configure, if they are incompatible with each other? ie by something loaded in the boot sequence.

I think you need it before the boot sequence – you can’t even see the boot sequence until you’ve worked out the disc format and partitioning.

The Iyonix has 768 bytes of CMOS RAM that are more or less unused, so that seems to me the logical place to put anything. When I’ve got the BeagleBoard CMOS RAM functional equivalent going, there will normally be huge amounts of space available.

I think you need it before the boot sequence – you can’t even see the boot sequence until you’ve worked out the disc format and partitioning.

I was only thinking of the extra partition schemes which effectively format the drive as though it is smaller and then have extra partitions in the remainder, (and then only if they were unable to be reliably identified automatically.)

The only time you would need to be able to boot from this type of partition is to get around old ADFS limits when softloading on an old 3.x machine. But any such machine would probably need to have a CMOS equivalent file for RO5 anyway.

Any new machine would probably use a standard mbr/guid for the main drive.

I have just done some tests with a memory stick, a linux amachine and my Iyonix.

It appears that an MSDOS style partition table can co-exist with a standard ADFS partition. (i.e. using SCSIform and using less cylinders.)

It took quite a few attempts, with SCSIform and gparted. But I formatted it to less than half capacity with SCSIform, and partitioned it with gparted and created fat and ext partitions beyond the halfway point.

The order it seems to need, is

1) ADFS format

2) create partiontable (gparted)

3) adfs initialise to required size

4) leave that much space (plus room for error) at the bottom of the drive when creating partitions with gparted.

With a FAT32 partition, the ADFS one is not seen on my Iyonix, but if this is hidden (eg linux fdisk to change the partition type) the ADFS partition mounts and appears to work correctly.

Hopefully this would mean that a standard PC compatible partition scheme would be able to be used.

If the first partition is ADFS, then current machines would be able to use that partition normally (and not see any others).

If RISC OS were to use flags in the partition table to choose where to boot from, it would mean that an old 3.7 machine would be able to have a 2GB old format ADFS partition, from which it could softload (or run for old software) and when RO 5 loads, it could boot from an new format partition.

What I did (in order to support dual booting RISC OS and Ubuntu off the same hard drive) was:

1. Use GParted to initialise an MSDOS partition table.

2. Created a single ext3 partition, occupying the second half of the disc, leaving the first half unallocated to any partition. Made sure I rounded it to cylinder boundaries.

3. Back in RISC OS, used some BASIC to read the MBR from the first sector.

4. Used SCSIForm to initialise the first half of the hard disc as a Filecore-formatted disc. This blanked out the MBR.

5. Used some BASIC to write the MBR back again.

This way RISC OS only knows about the Filecore-formatted part of the disc. Ubuntu only knows about the ext3 partition. Admittedly, I perhaps could have created two partitions in GParted and set the first (for subsequent use by RISC OS) to some other type of partition to make sure Linux ignored it. As it is at the moment, I cannot mount the RISC OS section of the drive from Ubuntu. Mind you, I have found the Linux support of Filecore drives not entirely reliable.

I wanted to look at getting IscaFS to mount the ext3 partition from RISC OS, but it crashed the Beagleboard beautifully. I suppose I could try on the Iyonix and see if it fares any better. IscaFS has to be compiled with its own assembler, so I don’t fancy delving into that code to fix it. Too much else to do at the moment.

P.S. This arrangement is not the scheme which was used for putting Linux partitions onto the end of a Filecore disc with ARMLinux on the RISC PC, which I still have. I think that used the official partition support in Filecore (only used by RISCiX according to the PRMs) but with a different partition type. ARMLinux must then have expected some form of partition table in the location pointed to by the FileCore partition bytes.

Also, when using SCSIForm to format the FileCore part of the drive (4. above) I had to override its interesting suggestions for heads and sectors per track and set them to 255 and 63 in order not to get confused over the cylinder calculations. SCSIForm tries to get the most out of the number of sectors reported by the drive!

I wanted to look at getting IscaFS to mount the ext3 partition from RISC OS, but it crashed the Beagleboard beautifully.

Was this v. 0.17? I’ve never tried it. If you have time to add a few details to the software compatability list it’d be very helpful.

I wanted to look at getting IscaFS to mount the ext3 partition from RISC OS, but it crashed the Beagleboard beautifully.

IscaFS predates the BeagleBoard and the ARM architecture used in the OMAP CPU, so I would imagine that it would require recompilation as a minimum.

4. Used SCSIForm to initialise the first half of the hard disc as a Filecore-formatted disc. This blanked out the MBR.

I guess what would be useful is if SCSIform were to write an MBR with an entry matching the filecore area. (There appear to be several unused partition types).

(It would also be nice if it did away with C/H/S and just gave options for size or percentage, plus some presets eg 500M 2000M (old format), 128GB and 256GB)

I wanted to look at getting IscaFS to mount the ext3 partition from RISC OS, but it crashed the Beagleboard beautifully.

I tried it on my Iyonix, it didn’t crash, but I couldn’t mount anything.

readpart display scsi 0 0 Partition table sector : 0 Partition table address : 0 No such partition.

PartTool makepseudo scsi 0 0 test

Error: Second partition is not a linux partition table

That was both with a freshly gparteded drive and the hybrid one.

I did not use PartTool to create my IscaFS image file. PartTool is expecting that the FileCore disc will have a “boot block” at &C00 with a non-ADFS partition descriptor at &DFC. This gives the partition type (it’s expecting “9”, I think) and the low and high byte of the cylinder at which the non-ADFS partition starts. I don’t know what format the partition table is then expected to be in once found!

This would have been the scheme for partitioning used when setting up ARMLinux on a RISC PC or Iyonix, but I reckon anything we adopt for the future ought to allow for partitions to be readable by an OS that can understand them, rather than having the hurdle of having to understand a peculiar FileCore partitioning system in the first place.

What I did was I created an IscaFS pseudo-image file by hand, by modifying one I had previously created on the RISC PC. The IscaFS docs explain the format: it just needs sector numbers for start and end of the partition. See !IscaFS.Docs.PseudoImg for information. “ZTS” means zero-terminated string. If entering in Zap’s Byte mode the file starts FE 68 EE 4C 65 78 74 32 (that’s the first two special words in hex).

I just tried my hand-crafted IscaFS pseudo-image file on the Iyonix with the hard drive from the BeagleBoard. Result: the Iyonix hung completely. It’s hard to know whether it was just a fault in my image file, or whether IscaFS doesn’t work on the Iyonix (I think it is supposed to). Or maybe it just doesn’t work with SCSIFS. Or it could have been working but was taking ages to open an 80GB drive? I think some more experiments with a smaller disc with expendable contents would be useful. I might also try to read the partition table from the RISC PC’s drive to see what format PartTool is expecting.

If AI’s implementation of a base/overlay aufs is related to this discussion, could a system similar to aufs be included in the RISC OS scope? Although large parts of the current !Boot structure require write access, IMHO critical aspects which set variables and start the desktop should ideally not be easily corrupted by the user. Perhaps a basic generic boot sequence could reside in the “base”, with specifics in the “overlay” (if present, and loaded by the base). As a worst case, deletion of the overlay files would then still retain a usable system.

Although large parts of the current !Boot structure require write access, IMHO critical aspects which set variables and start the desktop should ideally not be easily corrupted by the user. Perhaps a basic generic boot sequence could reside in the “base”, with specifics in the “overlay” (if present, and loaded by the base). As a worst case, deletion of the overlay files would then still retain a usable system.

That would be an extreemly good improvement to RISC OS. However, the same end result could be achieved by splitting !Boot into a read only part and user part.

Either would be a worthwhile improvement.