Filing system bounties

I would definitely like to see ‘resise to fit the device’ to work for RISC OS as it does for other OS on pies with ext4 etc when you download an OS image file and write it to an sd card.

I’ve been asked and offered money to do that on more than one occasion, but the conclusion is; it is just not worth the risk. The FileCore format is so intrinsically tied to size of the allocation units when first formatted, that any significant change of size which would involve going up or down a unit involves shifting the alignment of absolutely everything on the disc and rewiring the entire map and all directories, an enormously slow process that if interrupted would result in total data loss (DiscKnight would not be able to help). It is much quicker and safer to format a new larger disc and copy everything across with normal operations.

I even looked in to the more limited scenario of providing a FileCore disc image which could be expanded on to a small range of SD card sizes. In that case as the image is not intended to be used as used as is, only after expansion, but I wasn’t happy with that any of the possible options would result in a optimal and reliable arrangement when expanded. Again creating an image for the desired size of SD card with an existing tool, was quicker, more reliable and would not result in half of the write life of the flash being used up in the process.

For the limited scope needed to bootstrap a new image, I’d be tempted to put a tiny boo,t-strap area on the disc just after the loader. On first boot, the system will “reformat” the disc, with the bootstrap area marked in the map as defects or something, but NOT update the boot block disc record. It can then either load the entire bootstrap data into RAM or read do some jigery-pokery to read it from the bootstrap bit of the disc and write it to the “full” disc. With that done, remove the defects from the map, update the disc record and reboot.

It’s still a fair chunk of work, and needs to “just work” on a system with very little on it.

That would work as long as the new disc is more than twice the size of the bootstrap stuff. It’s similar to the trick used to hide the DOS boot partition from the RISC OS on the Pi’s SD cards, but with the bonus of releasing the bootstrap area to be usable again once files have been copied on o the rest of the otherwise unmolested newly formatted disc.

I was thinking the bootstrap area could be in the hundreds of MB, so easily smaller than any half a disc you’re likely to get.

The functionality was intended to be used to expand an entire disc image on to a larger SD card, so could be quite a few GB.

If a native solution is impractical, could a Linux application be written to format a large SD card with a RISC OS format and unzip the files into it (or copy a self-extracting file onto it to be run on startup)?

The functionality was intended to be used to expand an entire disc image on to a larger SD card, so could be quite a few GB.

The Pi image is ~120MB compressed, I suspect it expands to quite a bit more but if it can be expacted onto the resized disc as long as the expansion doesn’t splat it, all should be fine.

If the ‘installation image’ is small enough, would it make sense to just ship an SD card with a FAT loader partition, which contains a file with the installation image. Then the first boot formats the remainder of the SD to the appropriate size (making sure to loop in the FAT as the !Boot.Loader file), unpacks the image into it, (optionally) deletes it from the FAT partition, and reboots? That avoids having to do any fancy Filecore footwork, it’s just formatting a partition. If it fails, you still have the installation image present on the FAT partition to fall back on.

You don’t get to reclaim the space in the FAT partition, but perhaps a few hundred MB isn’t a big deal. (technically you could borrow space from FAT back into Filecore, but best not go there). If you kept the installation image around it would be usable as a ‘system restore’ kind of feature (hold down Ctrl-Alt-Shift-Windows-Scroll Lock on boot and get the option to wipe your Filecore partition and start over).

Or, I suppose you could ship the SD with both the default-size main Filecore partition and the installation image present. Then you have options on what to do. For example, having the existing Filecore is useful if you want to preload files from another system before booting, which you wouldn’t get with just a compressed image present (although I suppose it could be a zip you could easily inspect).

Any of this gets trickier if the default image starts growing in size because more stuff is included, like on the ROD images…

The the requirement was for a very large image, featuring a lot of shovelware.

If RISC OS could be made to boot directly from a FAT formatted drive, would this make it easier?

Is there a publically available definition of the format?

FAT (including FAT32)
exFAT

Note that the current version of RaspberryPi OS handle NTFS and exFAT disks without any issues so if someone wants to add support for new disk formats on RISC OS, there probably exists some good librairies out there to help a RISC OS port.

I have noticed in bounties for FS improvements part 2 “27-Mar-2024 Technical discussions with developer”, which means that developments are restarting on that front.

Now, will there be a collaboration with RComp’s and use of its new “opensource” block driver architecture, or will we get a new developers war?

Where is the source code? Or is that ‘opensource’ like Iris? How long has it been now since that was announced and has anyone outside of the developers seen the ‘source’ for it?

exFAT

That one might require a licence even though it is open source.
https://www.microsoft.com/en-us/legal/intellectualproperty/tech-licensing/programs

or will we get a new developers war?

Developers having a war suggests that there are enough developers to be at war, rather than one guy mumbling to the bottom of his beer glass.

Where is the source code?

Aaaand…. here we go again.

has anyone outside of the developers seen the ‘source’ for it?

You know, the fact that stuff moves like treacle around here isn’t due to secrecy and conspiracies, but… one guy → beer glass.
(oh, and people (including me!) not using source repositories)

The Pi image is ~120MB compressed, I suspect it expands to quite a bit more

It expands to 1875MB, carefully smaller than 2GB but large enough to have lots of empty space and will fit onto any SD card of 2GB and above. If you have a larger card then you can use PartMgr to prepare the card with as much space for RISC OS (and the FAT partition) as you want. It will even add the firmware and the HardDisc4 image but not the shovelware that is included on the full image. You can then use !Store or PackMan to load the extra software you want.

If RISC OS could be made to boot directly from a FAT formatted drive, would this make it easier?

Pico did boot from FAT. RISC OS still looks at CMOS in Boot:Loader.CMOS and in $.CMOS so it will also boot from FAT, if the filesystem is SDFS and there is only a FAT partition on the SD card. It means that the firmware is then seen in the root directory. Not very tidy.

I see the FS bounty Step 2 was recently updated with “Technical discussions with developer”.
Before that, I think both ROOL and ROL should step forward and present their plans for FS developments.

On one end ROD’s new mysterious open source “Block driver” interface, which nobody has seen any code or spec of, could provide the implementation of partitions (which is the main topic of Step 2) in a generic way for most filing systems.

On the end user side, there is probably a good discussion to be hold on this forum on how people want the partitions to be named (disc SCISFS::0A, SCSIFS::0B, …?) and how they presented in the desktop (separate icons, menus, …).

“Block driver”

Hmm, block drivers already means something on RISC OS.

how people want the partitions to be named

Sharon, Charlotte, Floor, Simone, and Amy.

(disc SCISFS::0A, SCSIFS::0B, …?)

As good as anything else, but should only really show up when the drive doesn’t have a name.

Oh, and let SDFS::0.$ fall back to SDFS::0A.$.

and how they presented in the desktop

Icons with names, as is traditional.

Take some ideas from Simtec’s IDEFS – have an automount flag for partitions to appear at boot, the rest being hidden (no icon) until mounted. Also allow dynamic mounting.

I suspect this was to get around the baked in assumption of eight drive icons, because a 6GB drive with 512MB partitions…

Unmounted partitions were available (to be mounted) via a menu listing them. This can make the desktop tidier, like that partition with all the cat pics doesn’t need to always be present, just call it up as necessary.

Consider, also, some sort of per-partition access control like a password?