Partition Manager

they would end up with a bit of a mess.

This is why people who don’t understand how the underlying partitioning works should either image the entire media or back up the content at filesystem level.

If you’re going to start messing with the partitions without having the necessary knowledge, well, that’s like driving blindfolded. It rarely ends well.

But the necessary knowledge is very RISC OS Pi specific. More like driving a car that needs double declutching without knowing, or something odd like that.

the necessary knowledge is very RISC OS Pi specific.

Yup. And?

To continue the tortured analogy: I drive my toy car. I wouldn’t ever attempt to drive a McLaren. They’re both “cars” (on extreme ends of the definition of what makes a car a car), but the skill set, understanding, and knowledge is completely different.

Yes the analogy is poor. But the thing is, how does someone know that you need specialist knowledge for RISC OS on a Pi? When it looks the same as any other system, and all the other systems behave the same way?

I just tried an experiment. It is really easy to set up partitions in the wrong order on linux.

I set up a 128GB drive backwards, FAT either end, EXTFS and NTFS in between.

Linux gave no errors and treated things in partition table order.
Windows gave no errors and treated things (drive letter order) in partition order.
FAT32fs showed the same partitions as Linux (though I’m not sure it behaved correctly.)
The Pi successfully booted from the partition at the very far end of the drive.

The drive however had a ghost filecore that didn’t get deleted, so I’ll have to try again to get sensible results from RISC OS

you need specialist knowledge

a ghost filecore that didn’t get deleted

…when one system doesn’t use or understand partitioning at all (yet).

Think about it – on Linux and Windows, setting up a harddisc is often a two stage operation.

In the Microsoft world, it used to be FDISK to set up the partition(s) followed by FORMAT to lay out the filesystem structure. Since most people only need one partition on Windows (it doesn’t use a dedicated swap partition), the formatting “wizard” has been dumbed down somewhat.

On RISC OS, if you format a drive, well this very tool discussed in this topic is the first thing that talks about partitions on RISC OS ¹, it is otherwise an unknown concept.

The Pi successfully booted

Yes, the Pi has a really flexible bootloader.
Not all systems are that capable. The OMAP3 for example not only looks at only the first FAT partition, but the files it needs to boot have to be the first things in the directory, apparently in a specific order, though I’m not sure if I’ve ever made myself a bootable disc or just copied a RISC OS ROM onto an Angstrom image.

It’s much simpler to perform full backups by imaging the entire media, not specific partitions. Then, writing it back will work.

When it looks the same as any other system

When has that ever been true for RISC OS? Compared to the contemporaries: the desktop is alien, the command line is alien, the filesystem is alien…

About the only similarity is “turn it on and something appears on the screen that sort of vaguely looks a bit like Windows 95 used to”.
You need a lot of different knowledge for RISC OS (how to access the filing systems, where apps are, how to make menus appear…) so why should it be different when it comes to dealing with the disc layout?
RISC OS is notably different in so many ways that treating its disc structure the same as other systems is, surely, rather foolish?

I will hand you, however, that finding out about this sort of thing isn’t exactly simplicity. ;)

¹ Some podule based filing systems (Morley SCSIFS, Simtec IDEFS, …) implemented their own form of partition scheme primarily in order to work around the FileCore half gigabyte limitation (that persisted for far too long). It was the driver quietly translating the RISC OS disc address + partition into a real disc address. FileCore that managed the layout never understood any sort of partitioning.

The drive however had a ghost filecore that didn’t get deleted, so I’ll have to try again to get sensible results from RISC OS

If you Initialise the drive as MBR with Partition Manager first, it should clear the FileCore area.

I feel I need to clear some confusion from several of the posts above:

1. FileCore in the context of this thread is a “volume” format
2. Mainstream RISCOS builds do not understand partitioning, but most of the bespoke builds do and most IDEFS Podules also have bespoke partitioning that is non-standard (standard being MBR or GPT). Some SCSIFS Podules also use bespoke partitioning that is not based on either LUN/MBR/GPT
3. A “full-disk” FileCore drive is any disk with a valid FileCore DiscRec at LBA 6. This can overlap an MBR partition table (LBA 0-1) or a GPT partition table that has a maximum of 16 partition entries (LBA 2-5 and matching LBA at the end of the drive)
4. A properly constructed “overlayed” “full-disk” FileCore drive will have a “protective partition” that ranges from LBA 6 (or the first LBA after the FAT partition in the case of a Pi) to the end of the FileCore area. Note that this doesn’t match the actual FileCore “volume” which starts at LBA 0, so this structure is not valid for OS builds that do understand partitioning
5. Because of the point above, a file can be added to the FileCore volume that exactly covers the Pi’s Bootloader FAT partition, allowing it to be accessible within RISCOS as an Image filesystem
6. A properly constructed “overlayed” “full-disk” FileCore drive will “map out” the area that covers LBA 0-5 in the FileCore map, to prevent them being corrupt by files saved under RISCOS

I’ve quoted the key words/phrases as nomenclature is important here to avoid confusion:

• full-disk FileCore drive – the whole disk, or the maximum LBA FileCore can address…whichever comes first
• volume – any valid area that an OS understands as a self-contained drive
• partition – a range of LBA with a matching entry in a partition table
• protective partition – a partition table entry that doesn’t necessarily contain a valid volume, it simply blocks out a range of the disk to protect it

RISCOS on the Pi simply uses an MBR partition table, one Bootloader FAT partition, one protective partition and an overlayed FileCore volume.

Interesting discussion, even though I don’t understand a bit of it. Before I started messing with Partition Manager, I managed (unintentionally) to HForm the internal drive of my 4té2. Not a really big deal, as RISC OS runs from a removable micro SD. However, to my surprise, the reformatting had left a little germ cell of Linux untouched, that offers me the choice to install Linux again¹. It was even still there, when I reformatted the drive with PM. I now sort of-ish understand that it must be such protected partition.

My question, however, is: can I use PM to run RISC OS from the internal drive of my 4té2, rather than booting from a removable micro SD, and make that the default situation? For me, it doesn’t necessarily needs to be partitioned. The main reason for my request is that the internal (SCSI) drive is considerably faster than any micro SD that I tried.

¹ If you’re not familiar with R-Comps 4té computer: RISC OS boots and runs from a removable micro SD. If you switch the computer on without that SD, it then boots Linux from the large internal drive. Andrew has partitioned that drive, so that you have extra storage for RISC OS when not running Linux. I would like to swap the situation, so that I will have RISC OS by default, and only Linux when I insert a micro SD with Linux installed.

I would wonder about the wiseness of messing around with partitions…

RISC OS is not suitable as a general daily driver, Arm Linux almost is. Therefore dual boot is needed, if multiple setup aren’t an option. Swapping SD cards – just no. Swapping USBs, short term, OK. But a single drive system is needed. That means playing with partitions is required.

…when one system doesn’t use or understand partitioning at all (yet).

Sorry my comment was meant to point out that any RISC OS observations I made were questionable because of it.

Yes, the Pi has a really flexible bootloader. Not all systems are that capable.

I wonder if expoiting that could allow an image to be multiplatform. (That’s not a feature request, just a musing :) )

the desktop is alien, the command line is alien, the filesystem is alien…

Yes, but they generally look alien. RO Pi / !PartMan output looks totally standard, but will break doing things that would normally be expected. Whereas if the order were reversed (for example), and the FC protective partition were complete and Labelled DONT MOVE, it would be obvious something is different, and there would be a fair chance it would be clear what is going on. But this doesn’t actually need to be implemented as a feature in !PartMan, all that is needed is for it to behave sensibly with out of order partitions and be able to make the far left one into FC correctly.

bg. If you Initialise the drive as MBR with Partition Manager first, it should clear the FileCore area.

Of course, it knows it needs to do more work than a linux one which doesn’t know about FC.

I tried this on my Pi400 which I now have working (with a few glitches) on USB (SCSI 0:1) and network, and I got “There are files open on the SCSI filesystem that need to be forcibly closed Are you sure you want to proceed?”
And if I try to force it to initialise to FC it crashes the USB filesystems.

can I use PM to run RISC OS from the internal drive of my 4té2, rather than booting from a removable micro SD

Unlikely without specific support for the 4té2 as there will likely be a Bootloader somewhere on the internal drive, either in a partition or hidden at a specific LBA.

I got “There are files open on the SCSI filesystem that need to be forcibly closed Are you sure you want to proceed?”

Interesting, if you’re running a recent RISCOS you should not have received that warning as it uses the detach / attach Service calls if SCSIFS is v1.10 or newer. That warning should only be seen on legacy SCSIFS.

Could you create a debug log and let me know what version of SCSIFS is reported at the top of the debug log.

EDIT: I’ve checked the code-path. The only warning you should have received would be “There are open files on SCSI::< volume name > that need to be forcibly closed….” – which is essentially saying you’re trying to do something that results in altering a mounted drive that has open files, clicking YES will performed a *DISMOUNT on the FileCore volume before proceeding.

if I try to force it to initialise to FC it crashes the USB filesystems.

Could you confirm the SOE. You right-clicked on the drive and selected Initialise, then selected FileCore and it crashed after it had finished initialising the drive?

For what you’re trying to do you need to initialise as MBR, but regardless…it shouldn’t crash SCSIFS.

RISC OS is not suitable as a general daily driver

Depends what you need. I write my blog using RISC OS, and am writing this. Sure, there’s loads of things RISC OS can’t do, which is where my (Android) phone steps in.

I wonder if expoiting that could allow an image to be multiplatform.

If you mean disc image, then theoretically yes. I have built an image with the Beagle bootloader and the Pi bootloader… it’s just never actually been tested on the Beagle.

If you mean ROM image, then no. It took ARM devices a really long time (and an epic rant by Linus) before they started to have anything that resembled a device tree.

You see, in the PC world… maybe not so much with the complications of UEFI, but prior to that a PC would start up looking like a 286 with a crappy 16 colour VGA card. The bootloader/OS could use that to get itself going while poking and prodding certain known addresses to say “oh, it’s actually a Ryzen with 2K video” and load up the appropriate drivers to bring up the system correctly.

In the ARM world, it’s been an utter mess of partially and badly documented hardware that doesn’t have any fallback (as none was ever considered necessary) set at essentially arbitrary addresses within wildly differing memory mapping schemes. As such, while you can have a base kernel that will run on most PCs (which is why they could ship Ubuntu LiveCDs on magazines), in the ARM world each device needs its own specific one. That’s why RISC OS offers a selection of different ROM images and isn’t compatible with anything outside of that range (like the OrangePi or your old phone).

Interesting, if you’re running a recent RISCOS

I’m running a release version from earlier in the year with the beta updates merged – 5.29 (14-Jul-23) and a ROM I got from another thread (mine is the currently unsupported Pi400), all copied to replace the files on a drive Pi initialised by your app on a Pi 3+ ROMmodules reports SCSIFS 1.36 (Not sure how to get the debug you request.)

However, I just realised I was running 1.0 because I’d forgotten hopw PackMan works

And now I copied the stub in apps it works fine.

Depends what you need.

Instant messaging

If you mean disc image, then theoretically yes.

Yes this.

Instant messaging

“Media”, fine. It’s the “Social” part that I have problems with.

Or, as Wednesday puts it:

I find social media to be a soul-sucking void of meaningless affirmation.

That RISC OS can’t, I consider to be a definite positive. ;)

“Media”, fine. It’s the “Social” part that I have problems with.

It’s only IM I want. Telegram and Session are the important ones (Trying to move from the former to the latter after recent changes)

I’ve updated the current test build of Partition Manager 1.02

This post provides more detail of the changes, but for the purposes of this forum the main change is to the Pi Boot drive download code, which fixes a buffer overrun that’s causing 1.01 to fail. It also now scrapes this site and the NetSurf site to find the current software versions – which should hopefully cover 5.30 when its formally released.

As URL_GetURL doesn’t work on the NetSurf site its scraping their build server as a temporary workaround.

I’ve updated the current test build of Partition Manager 1.02, which now downloads the package sources to find the URLs for the various App packages that get pre-installed to a Pi Boot drive

I’ve updated the current test build of Partition Manager 1.02

I am confused by the “Changes” text. You refer to “Big map” and a limit of 4 GiB which I don’t understand.

Are you confusing this with “big disc” (sometimes called “large disc”, that thing signalled by the “big_flag” in the disc record), which was the change in RISC OS 3.6 Filecore (“Project Black”) where the sector offset addressing changed from byte offsets to sector offsets? This is the feature that extended Filecore max partition size from 512 MiB to 256 GiB. No changes in map handling involved.

And this is still called “E” format throughout Acorn docs AFAIK. Although the combinatorics (old/new map, old/new/big dir, no zones/1 zone/n zones, idlen length, big disc) and the floppy-driven “one letter” identifier makes for good headaches. I mean, who came up with “F”/“F+”?

I am confused by the “Changes” text. You refer to “Big map” and a limit of 4 GiB which I don’t understand.

This is where things probably get confusing unless you wade through the FileCore source code.

“Large disc” (BigDisc in the source) is set at the time the partition is formatted, based on the FileCore partition size. It’s set for all partitions larger than 512MB.

“Big map” (BigMaps in the source) increases the max idlen to 19 and consequently increases the max partition size to 4GB assuming LBA addressing.

All drives are “E” format, “+” denoting Big directories (BigDir in the source)

This is where things probably get confusing unless you wade through the FileCore source code.

You are entirely correct, but now I am even more confused.

“Big map” (BigMaps in the source) increases the max idlen to 19 and consequently increases the max partition size to 4GB assuming LBA addressing.

And there it is, the confusion. Because the partition size is also governed by LFAU. And sector size. Given a 1k LFAU, the max with sector size 512 bytes and idlen=19 should be around 8 GiB? So why pick an arbitrary combination of parameters and pretend there is a natural 4 GiB partition size limit?

The original 512 MiB partition size limit was a limit of “32 bit byte addressing, and we waste a further 3 bits for the drive number”, not an idlen/LFAU limit.

Side note: idlen can now be up to 21 (since FileCore 3.75 according to my notes), so I guess that is now called “EvenBiggerMaps” in the source? :-)

The good news is, that I have finally found refs for “Big map” in half-way official doc, and in checkmap output. At least one confusing thing less on my list.

And there it is, the confusion. Because the partition size is also governed by LFAU. And sector size. Given a 1k LFAU, the max with sector size 512 bytes and idlen=19 should be around 8 GiB? So why pick an arbitrary combination of parameters and pretend there is a natural 4 GiB partition size limit?

Sector size isn’t a factor with earlier FileCore as realistically only 512 bytes is supported by the underlying filesystem, in fact many IDEFS are hardcoded for 512 byte sectors. Technically earlier ADFS support smaller sector sizes as it also covers floppies, ST506 were 256 byte for example.

If we ignore ST506 for a minute as support was dropped in the hardware, that leaves 512 bytes as the only supported sector size on HD up until 4Kn/512e support was added…which doesn’t play a part here as “E” and “E, Big map” are for pre-RISC OS 5.

You are correct in that 4GB is an arbitrary limit, but its a fair one once you drill into the FileCore detail:

The maximum map size on earlier FileCore was 16 bits, so 4GB/(8*2^16)=8KB per map bit. Because of the minimum idlen, that equates to a minimum allocation of 128KB – which is incredibly wasteful.

I did consider reducing the partition limit (on “E, Big map”) to 2GB as an allocation size of 64KB seems somewhat more sensible, but in the end opted to follow Acorn’s advice on the subject which suggests 4GB as a practical limit.

idlen can now be up to 21 (since FileCore 3.75 according to my notes), so I guess that is now called “EvenBiggerMaps” in the source? :-)

Correct, although its still falls under “Big map” as the value was simply increased from 19 to 21 in the source. It’s a fairly recent increase under RO5, so I suppose didn’t really warrant the need to retain 19 bits as a build option.

Sector size isn’t a factor with earlier FileCore as realistically only 512 bytes is supported by the underlying filesystem, in fact many IDEFS are hardcoded for 512 byte sectors. Technically earlier ADFS support smaller sector sizes as it also covers floppies, ST506 were 256 byte for example.

And floppies (D, E) had 1024 bytes/sector. And there was some optical drives with 1024 bytes/sector – not MO (this was 512 or 2048, the latter being a real problem back in the days, but Acorn was not interested to fix it), but Phase-Change or Floptical or something like that. At least Cumana and later Power-tec SCSIFS could handle that.

I did consider reducing the partition limit (on “E, Big map”)

Isn’t “Big Map” automatically “E+”? Is “Big Map, but not Big Dir” a valid combination? And if it is, who would possibly want it? It makes no sense to want “New Dir” instead of “Big Dir”. Or does it?

What does HForm do – IIRC it won’t let you choose “Big Map” or “Big Dir”, but only “long filenames” or something like that.

I wonder if Filecore could handle a mixture of old dir, new dir and big dir on the same disc.

but in the end opted to follow Acorn’s advice on the subject which suggests 4GB as a practical limit.

The only “4 GB practical limit” recommendation by Acorn I know of was in PRM5a (“Under FileCore, the recommended maximum hard disc size is 4 GB”) and predates idlen extension. I.e. “Big Disc”, not “Big Map”.

To avoid confusion, I’ll rename the layout to something else.

The FileCore version check I’m performing is wrong anyway. idlen increased to 19 in FileCore 3.16 not 3.15 that PM is current checking against.