File system improvements - now underway

Great news that we’ve had a developer claim the “File system improvements (Step 1)” bounty. Please consider making contributions into the other bounties to help get them underway too…! :)

Let’s hope the £720 covers the coffee and paracetamol!

Anyway, great news to see work is underway to free us from the shackles of 2Gb.

I wonder (not really related, of course!) if it would be possible to add partition support to ADFS. If an SD filesystem is on the way, would it be feasible to built a multi-partition SD card akin to the Beagle “test” image (~100Mb FAT32, rest ext) only it’ll be enough FAT to load RISC OS, and the rest would be ADFS native?

Partition support is indeed in one of the future steps for the “File system improvements” bounties. Once this one is closed, we’ll open “Step 2”. ISTR we published our breakdown of what’s been put into each step somewhere on this site (I may be wrong there) but we’re currently debating the exact ordering of the millions of subtasks based upon a) their relative hardness and b) a better understanding of what depends upon what. So the truth is, we’re not 100% sure which changes and features will appear in step 2, nor exactly how many steps/bounties there will be (something like four or five).

ISTR we published our breakdown of what’s been put into each step somewhere on this site

I think I’ve found it at Filing system bounties. There’s also related discussion at File system improvements (Step 1).

This is excellent news. Does the bounty claim mean it is almost ready to go into a ROM?

Since this is quite an important step, will there be a proper anouncement with full details?

The obvious questions are;

Is removal of restrictions treated as a new format?
If not, what do old systems make of it?
Can it be softloaded onto other systems? (e.g. Select)

I think partition support is so important it justifies being step two on its own. (Obviously a partition/disk tool would be needed and clients being updated.)

Certainly a partition support (only) bounty arriving around the time of my next paycheck would get my support.

only it’ll be enough FAT to load RISC OS, and the rest would be ADFS native.

I think it would be better to have a modestly sized ADFS partition, aimed to fit everything on a 2GB card. On larger card the extra space could be use for a second ADFS partition, or used to install other operating systems.

That’s rather exciting news. Both for RISC OS and the bounty approach chosen by ROOL!

Unless it would compromise confidentiality, could you hint whether the claimant is someone who has previously worked on the RISC OS source, or have we successfully enticed someone new?

Does the bounty claim mean it is almost ready to go into a ROM?

I wouldn’t think so (although would love to be corrected). "Underway" probably means that it’s been claimed for development (rather than the bounty sum itself being claimed). Nevertheless, I agree that it’s excellent news. I hope that the developer won’t be discouraged from requesting input here as needed. Although being paid (a relatively small amount) for doing the work, IMHO this shouldn’t preclude arising issues from being discussed openly¹ if that would help accelerate/improve the implementation. This is the coding bounty test case, after all… and everyone wants it to be a success.

¹ A pseudonym login could perhaps be considered if (initial) anonymity is required, subject to ROOL’s agreement, I guess.

I think it would be better to have a modestly sized ADFS partition, aimed to fit everything on a 2GB card.

I wasn’t necessarily thinking of anything larger than ~2Gb. Maybe a 4 as it seems the smaller ones are less common these days. At any rate, RISC OS doesn’t really require the obscene drive sizes typical on <cough>other</cough> systems. Really, plugging in a .5Tb disc would (if/when actually supported) be truth in the maxim of on a clear disc, you can seek forever …

Does the bounty claim mean it is almost ready to go into a ROM?

<- what Trevor said… :)

The process is that people donate into a bounty and when it reaches a level that interests a developer, they let us know and we mark the bounty as “claimed” while they work on it. If things go well, they do the work and we give them the bounty. If not, we re-open the bounty for someone else to claim.

And no, I can’t say anything about who claimed this bounty; it’s up to the developer to decide if they want to go public.

Really, plugging in a .5Tb disc would (if/when actually supported) be truth in the maxim of on a clear disc, you can seek forever

I have a 0.5Tb and 0.3Tb disc plugged into my Raspberry Pi at the moment… (Over USB obviously!)

I have testet a two Partition Card. First adfs, second FAT. Look at the Test post. The first “Beagle live System” ;-)

The Test forum is really for experiments on the forum, not for tests of RISC OS :) but that is an interesting discovery. I think you are saying that your SD card is laid out like this:

and that U-Boot is still running from that FAT partition, even though that is not at the start of the disc. Are you sure it is not using an old copy of U-Boot from the unallocated space at the start of the FileCore partition? The easiest way to test this is to zero out the disc before you start formatting. You can do this from Linux using

sudo dd if=/dev/zero of=/dev/sd_reader_device bs=1M

ROOL has been planning to use a similar scheme for the Raspberry Pi (but not exactly the same). I wanted to make sure this was possible, so I have checked that the Raspberry Pi bootloader reads the MBR partition table to locate the FAT partition. I haven’t had time to check the Beagleboard yet, though.

I can confirm it does work on a BeagleBoard Xm. The only disadvantage is the FAT32 partition can’t be seen in RISC OS so you have to use the old cmos method/widget.

I did it a slightly different way to Raik. The trick is make sure the FAT partition is partition 1. I used ‘cfdisk’ on Ubuntu to first create a FAT32 partition at the end of a 16GiB SD card from cylinder 14001 of 793 cylinders leaving the required space for the Filecore partition at the beginning so they don’t overlap. Then I used !HForm with heads 64, Sectors/Track 32 and cylinders 14000 to exactly fit the available space.

I’ve always written that my english is very bad. Misunderstandings I would therefore have to apologize.
Otherwise, does the way I described.
I noticed earlier that the reformat of adfs cards under Win and Linux is not so easy. Apparently the information is stored in different areas. The Beagle BIOS recognizes the ADFS partition and boot from the 2nd FAT partition and SDFS recognizes only the first partion, as usual in RISCOS.
I try and create a map just made ​​sure that the two ranges do not overlap. Chris todo is similar, with a similar result.

Interesting. ROOL has been working hard on enabling the Raspberry Pi to boot without the need for external USB mass storage (it’s very important to the Raspberry Pi Foundation, and a key requirement for them distributing an official Raspberry Pi RISC OS distribution when it’s ready). It’s not such an important requirement for the Beagleboard, especially the xM where you have loads of USB slots, but your findings make it sound like it may be possible to move towards enabling booting entirely from SD on both platforms (and without needing to copy everything into RAMFS as suggested on another thread!)

Chris, I wouldn’t worry too much about accessing the CMOS file on the SD card. There’s a “Third Way” which we’ve been working on because the Raspberry Pi would have the same problem. I don’t want to say too much until I’ve seen it working, but watch this space.

The main advantage of bootable SD cards is being able to create downloadable images with the latest rom and !boot already set up. It will mean getting a fully functioning RISC OS machine up and running will just be a matter of writing the image to an SD card and plugging it in. No more ‘chicken and egg’.

Yes, I agree with you 100% on that. There are a number of practical considerations that make it tricky for ROOL to set up such downloads, especially for the many and varied OMAP3 boards, but it’s clearly the best route from the end user’s point of view.

0 … 14.5 GB FileCore partition (shown as unallocated space in the MBR partition table)

I don’t think that is very wise, it really would be sensible to allocate it with an unused partition type (there are lots, I researched it.) It would be quite easy for the ADFS partiton to get wiped if plugged into
another system.

If the FAT partition needs to be the first entry (for the loader), then the ADFS could be put in position 4.

Would it be possible for HDForm to write a partition table entry? (It could ask for 0-4 with 0 being don’t create one).
It would also be nice to have a sensible selection of preset sizes.

It will mean getting a fully functioning RISC OS machine up and running will just be a matter of writing the image to an SD card and plugging it in.

If the rom image is set to SDFS as the default filing system (rather than ADFS) then we have got this now, see other thread (Porting RISC OS/Updates on Raspberry Pi).

I did get an MBR partition type reserved for RISC OS, but whether it’s any use is out of our control. It may be that the masked ROM built into the OMAP3 and/or BCM2835 requires the first partition in the MBR table to be the one containing X-loader or start.elf, respectively. You’d need to try it to find out. The FileCore partition needs to be at the start of the disc, although FileCore doesn’t care whether it’s mentioned in the MBR partition table or not.

Having a the FileCore partition further into the disc requires proper partition support in FileCore – an amount of work which I (and other people) have scoped out, and even the best estimates mean that this wouldn’t be ready in time for the official RISC OS on Raspberry Pi launch date, so that’s a non-starter at the moment.

The order of the partition table doesn’t have to match the order of the partitions on the disk.

The 4th partition entry in the MBR could be the FC part, physically at the beginning of the disk. Hopefully the loader wouldn’t take exception to this.

What is the RISC OS partition type?

The order of the partition table doesn’t have to match the order of the partitions on the disk.

Interesting thought – obviously the partition table itself is just a sequence of words containing LBA addresses, which can in theory be in any order (or even overlap each other or hang off the end of the physical disc), but I assumed it would confuse things like GPartEd if they weren’t in order – which would negate the benefits of inserting a guard partition covering the FileCore image in the first place. It’s probably worth checking how the Windows and MacOS partition editors handle such a layout as well.

What is the RISC OS partition type?

This page seems to be the de-facto list of MBR partition types, in which you’ll see I claimed &AD for ADFS/FileCore. The maintainer of that page was very keen that we didn’t start using the partition type to control how we interpret the contents of each partition, so the value is mainly to prevent misinterpretation of the partition by any legacy software for non-RISC OS platforms.