Partition Manager

The next one for you is a 516 MB Rust from a RiscPC600
The Pi reads, writes and verifies it OK, but PartMgr sees it as 516 MB Unallocated

I need to see the complete debug/txt file before I can investigate why they are not being detected as a FileCore drive. I’ve been doing all by FileCore testing on 3 RiscPC, so guess it’s down to differences in the version of HForm used and what it’s doing in the MBR partition table area. PM has to work around years worth of hacks and changes that aren’t necessarily compatible with GPT/MBR partitioning handlers.

From some of the debug/txt files, it’s evident that some fixing of the MBR table is going to be required before some of these drives will work when the OS supports partitions. I don’t envy the person that implements that. Hopefully we can pick them up as part of this and get the MBR’s to a standard it can support.

A gentle reminder to folk helping in this mammoth task. Please only post the debug/txt if something is wrong. When posting, I need to see the complete file and please describe the issue as best you can. As this site isn’t really suited to sharing this kind of debugging information, you can sign up to the JASPP forum and reply to the thread there with attachments. Failing that we can start a thread for this on StarDot as I will be engaging with that community in the next phase, once initialising full disk FileCore drives is coded so we can replace HForm.

Hi Jon,

0.38 works again on the Iyonix, 5.29, but only for ADFS::4.

The free space is reported as 1010MB by PartMgr but Diskinght and Freemap report it as 1011Mb so perhaps a rounding issue?

Any way all this plus debug and screen shots put on JASPP feedback thread.

I need to see the complete debug/txt file before I can investigate

Sorry. I thought I had posted sufficient. If you think it will help I can repeat one of the tests tomorrow. (The debug file has been over-written and the drives returned to the cupboard)

will we be able to specify which drive the system will boot from

There’s two issues on this point:

1. I might be wrong on this, but my understanding is that RISCOS attempts to boot from the first valid drive it finds with a !Boot file/folder.

Wrong. Filing system is read from “CMOS”, ADFS reads default drive from “CMOS”, don’t know about other device drivers. TerritoryManager will load some modules if the configured territory is not already loaded.

If it’s a FileCore partition it checks the Boot option to see if should actually run/load/exec it.

Correct.

FAT partitions are treated slightly differently in that there’s no Boot flag to check, so it just runs it anyway.

This is only the case since DOSFS 0.88, 14th April 2012 earlier versions do not load anything at this stage.

2. MBR has a bootable flag on the partitions, but GPT does not.

GPT has a bootable flag that is like the MBR, see UEFI 2.8 chapter 3 which defines the MBR boot flag as:

0×80 indicates that this is the bootable legacy partition. Other values indicate that this is not a bootable legacy partition. This field shall not be used by UEFI firmware.

And it defines the GPT boot flag as “Legacy BIOS Bootable” and states:

This bit is set aside by this specification to let systems with traditional PC-AT BIOS firmware implementations inform certain limited, special-purpose software running on these systems that a GPT partition may be bootable. For systems with firmware implementations conforming to this specification, the UEFI boot manager (see chapter 3) must ignore this bit when selecting a UEFI-compliant application, e.g., an OS loader (see 2.1.3). Therefore there is no need for this specification to define the exact meaning of this bit.

The boot flag is only only well defined for use by a program in the MBR to load execute a program in first sector of the marked partition. There are no grounds to say it is available for RISC OS to identify a Filecore partition containing !Boot rather than the bootloader to identify a FAT partition containing the RISC OS ROM image.

For GPT, either the partition GUID can imply bootable/non bootable, or the partition entry attributes can be used. There’s no formal standard, so its down to the OS maintainer to decide how

GUID + partition entry attributes are used by ChromeOS and the Systemd Discoverable Partitions Specification which is a good approach, but should be overridable by a configuration file (ie virtual “CMOS”) or command line option when RISC OS is loaded from a filing system.

will we be able to specify which drive the system will boot from

This was my question and with hindsight I asked the wrong question.

On my A410 and RPC systems, I always knew which drive would be the boot drive as it was clearly specified in CMOS and the hardware fixed the drive numbers.

On a Raspberry Pi with two SCSI drives, again I can specify that the system will boot from SCSIFS Drive 4.

But the trouble is that I have no way of specifying which USB adapter will be Drive 4.

Very nice job. Thanks for that.

@Anton Reiser, I’ve rewritten the 4K code based on your feedback, if you would like to confirm it gets your drive size correct

I’m not Anton ;-) but I use the “same” 2TB HDD created with Antons tools.
My debugfiles (not all) you find here.
At first I try…
*satadevices
Ctl Prt PMP Type Description Capacity
0 0 0 ATA Samsung SSD 850 EVO 500GB 466 Gbytes
0 0 1 ATA Samsung SSD 860 EVO 250GB 233 Gbytes
0 0 2 ATA ST2000NM0105-1YY104 1863 Gbytes
0 0 3 ATA Samsung SSD 860 EVO 500GB 466 Gbytes

PartMgr 038…

The complete size of the 2TB device is not displayed correct…

Second try: Remove the GPT SSD and plug to SCSI and plug my 1TB Backup device to SATA (FileCore+FAT partition create with Antons PartiFC).
*satadevices
Ctl Prt PMP Type Description Capacity
0 0 0 ATA Samsung SSD 850 EVO 500GB 466 Gbytes
0 0 1 ATA Samsung SSD 860 EVO 250GB 233 Gbytes
0 0 2 ATA ST2000NM0105-1YY104 1863 Gbytes
0 0 3 ATA Samsung SSD 860 EVO 1TB 932 Gbytes
0 0 15 Multiplier

The second FAT partition is ignored but works with FAT32FS without problems.

Next try: Remove the GTP device and plugin an 2GB EADSFS (PowerTec IDE partman) device from A4.
*SCSIdevices
Device Type Capacity Vendor Product Revision
0:0.0 Direct-access 2067 Mbytes FUJITSU MHA2021AT

The hdd is splited to 4×512MB. The first partiton is usable on Ti but ignored by PartMgr.

Sorry. I thought I had posted sufficient. If you think it will help I can repeat one of the tests tomorrow. (The debug file has been over-written and the drives returned to the cupboard)

No problem, you weren’t to know which bits I needed to look at. I would appreciate just one full debug/txt for the moment, that should give me the information I need.

Wrong. Filing system is read from “CMOS”, ADFS reads default drive from “CMOS”

Thanks for the correction, I’ve revised my comment. I keep forgetting about ADFS as none of my modern RO machines support it and I haven’t used it in years!

GPT has a bootable flag that is like the MBR

UEFI has a boot flag in the Boot Manager flag, GPT does not as far as I can tell. MBR is a requirement of GPT, but only for the Protective Partition so it’s boot flag should not be used.

As no RISCOS machines are UEFI (correct me if I’m wrong on this) and we have to cover a range of legacy kit, we can probably “ignore” UEFI spec.

GUID + partition entry attributes are used by ChromeOS and the Systemd Discoverable Partitions Specification which is a good approach

Yes, it seems to be the most sensible approach.

Whoever implements the Partition bounty also needs to consider softloaded OS, where a Boot FileCore partition will be needed to load the OS, before any Partition Manager can decide how to boot correctly.

@Anton Reiser, I’ve rewritten the 4K code based on your feedback, if you would like to confirm it gets your drive size correct

Not exactly.
LBA Size =2048
You have to multiply word 117/118 by two, as the value represents word, not byte.

There is a little bug in PROCdebug_block_dword: the last 3 characters in the ASCII text are missing.

Partition type 7 on my Win/Linux disc is now correctly recognized as NTFS or exFAT.

On a other disc, the partition type 0×0c (FAT32 LBA) is shown as <unknown>

*partiFC SCSIFS :4 -v
Disc size total:         465.8 GiB
512 Byte/Sector
Primary partitions:
 B Start    Sectors       Size      Typ
   00000001 1fffffff     256.0 GiB  ad  ADFS
   20000000 1a386000     209.8 GiB  0c  FAT32 LBA

add_partitions_MBR
==================
SCSI::4 checking for MBR partition table
read_bytes_SCSIFS(3,&0,&0,..,&200)
  SCSIFS_DiscOp64 drive=4 high=&0 low=&0

Address :  0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F
&000000 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000010 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000020 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000030 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000040 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000050 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000060 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000070 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000080 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000090 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&0000A0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&0000B0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&0000C0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&0000D0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&0000E0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&0000F0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000100 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000110 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000120 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000130 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000140 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000150 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000160 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000170 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000180 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&000190 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&0001A0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&0001B0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FE   ................
&0001C0 : FF FF AD FE FF FF 01 00 00 00 FF FF FF 1F 00 FE   ................
&0001D0 : FF FF 0C FE FF FF 00 00 00 20 00 60 38 1A 00 00   ......... .`8...
&0001E0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
&0001F0 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 AA   ..............U.

MBR partition table found

Partition 0
-----------
Partition_Type%=&AD
Start LBA      =&000000001
Size in LBA    =&01FFFFFFF
Checking SCSI::4 partition 0 for FileCore
Free=&3130B0000
OS Size in LBA=&01FFF5D74
Valid FileCore disc found, disc name is: BIK
SCSI::4 detected as: E+, Boot, Big map

Partition 1
-----------
Partition_Type%=&C
Start LBA      =&020000000
Size in LBA    =&01A386000
Checking SCSI::4 partition 1 for FAT
read_bytes_SCSIFS(3,&20000000,&0,..,&200)
  SCSIFS_DiscOp64 drive=4 high=&40 low=&0

The complete size of the 2TB device is not displayed correct…

Thanks for the report, I’ll grab the debug files later and investigate.

The second FAT partition is ignored but works with FAT32FS without problems.

If you’re referring to the red partition, it’s not been ignored. It correctly read the volume name, but has marked it red as DOSFS would not be able to read it fully. FAT32FS reads in a different way to work around the OS drive size limitation.

As I mentioned early on, some consideration needs to be made around FAT32FS becoming part of the OS as DOSFS can’t handle DOS partitions that FileCore can’t fully access. The alternative is DOSFS is modified to handle DOS partitions outside of a FileCore partition.

The hdd is splited to 4×512MB. The first partiton is usable on Ti but ignored by PartMgr.

Are you referring to SCSI::4 in the image, which is showing as fully unallocated? PowerTec IDE support is out of scope of this currently (would be phase 2), I’d need detailed specs on how it’s partitioning before I could support it. I would however like to see the debug/txt output for the drive so I can at least get the initial partition displaying.

I suspect what’s happened is it’s tried GPT, then MBR, found a valid MBR signature but no partition information and subsequently not tried FileCore.

You have to multiply word 117/118 by two, as the value represents word, not byte.

Thanks, I’ll modify the code. That explains Raik’s 2TB showing as 1TB.

There is a little bug in PROCdebug_block_dword: the last 3 characters in the ASCII text are missing.

It was a quick hack I put together at the last minute, I’ll correct.

On a other disc, the partition type 0×0c (FAT32 LBA) is shown as

Did it recognise it as FAT? Is it just the volume name that’s unknown?

Did it recognise it as FAT? Is it just the volume name that’s unknown?

No. ‘FAT32’ is not shown.

Confirmed, FAT32 is not shown. The box here is red with < unknown > and the disk size displayed. Its also not present in the debug text.

Should the volume list only show information for those devices that are mounted? In contrast to those devices displayed above it? Currently I see all devices in the top panel, but not in the volume list; therefore I’ve assumed the volume list is only presenting the mounted devices.

If you’re referring to the red partition, it’s not been ignored. It correctly read the volume name, but has marked it red as DOSFS would not be able to read it fully. FAT32FS reads in a different way to work around the OS drive size limitation.

I think we “speak” about differnt devices.
ADFS::4 has an Linux and a FAT partiton (Ti_Fat…fat32fs:mount -fp2 :12 { > null: })… is correct.
ADFS::7 has two partitions. ADFSBackup followed by Fat32Backup (fat32fs:mount -fp2 :15 { > null: })…

fat32fs:free … gives…
Ti_Fat
Bytes free &000000424E4C4000 = 265 Gbytes
Bytes used &000000160EC3C000 = 88 Gbytes

Fat32Backup
Bytes free &000000ABF3C78000 = 687 Gbytes
Bytes used &00000002B3138000 = 10 Gbytes

I would appreciate just one full debug/txt for the moment, that should give me the information I need.

OK. No problem.

I do not have anywhere else to put it and I am reluctant to create yet more logins, so I have posted the Pi4 5.28 debug file in Tests on this forum.

SCSI::4 is an mSATA mounted via an adapter from RISCOSbits (HForm on a Pi)
SDFS::0 is a 4GB SDcard with a 3.65MB Fat32 partition written by Windows XP
SCSI::5 is an old RiscPC600 spinning rust drive mounted via USB adapter. (Legacy HForm on RPC)
I will delete the text when you have copied it.

v0.39 available

The next one for you is a 516 MB Rust from a RiscPC600

The Pi reads, writes and verifies it OK, but PartMgr sees it as 516 MB Unallocated.

If it’s SCSI::4 in your debug/txt file – its being detected as FileCore so I’m not sure how it’s subsequently shown as unallocated.
If it’s SDFS::0 then it has an MBR partition with only one entry – a FAT partition.

The free space is reported as 1010MB by PartMgr but Diskinght and Freemap report it as 1011Mb so perhaps a rounding issue?

The free space reported by OS_FSControl 55/49 in debug/txt is &3F284000 = 1059602432 / 1024 / 1024 = exactly 1010 MB

Confirmed, FAT32 is not shown. The box here is red with < unknown > and the disk size displayed. Its also not present in the debug text.

FileCore limits? I’ve added addition debugging output which will confirm when FileCore can’t read the start and end LBA of a FAT partition.

Should the volume list only show information for those devices that are mounted?

The volume list shows volumes that are fully accessible by FileCore

ADFS::7 has two partitions. ADFSBackup followed by Fat32Backup (fat32fs:mount -fp2 :15 { > null: })

FileCore limits perhaps, try latest version to confirm.

I will delete the text when you have copied it.

I’ve copied the debug log, you can delete – thanks.

v0.39. Started without the SSD FAT partition open. Opened FAT partition, quit Partition Manager and retarted. There was delay, so here’s the debug.txt:

Log deleted.

FileCore limits? I’ve added addition debugging output which will confirm when FileCore can’t read the start and end LBA of a FAT partition.

Same here.

MBR partition table found

Partition 0
-----------
Partition_Type%=&AD
Start LBA      =&000000001
Size in LBA    =&01FFFFFFF
Checking SCSI::4 partition 0 for FileCore
Free=&3130B0000
OS Size in LBA=&01FFF5D74
Valid FileCore disc found, disc name is: BIK
SCSI::4 detected as: E+, Boot, Big map

Partition 1
-----------
Partition_Type%=&C
Start LBA      =&020000000
Size in LBA    =&01A386000
Checking SCSI::4 partition 1 for FAT
read_bytes_SCSIFS(3,&20000000,&0,..,&200)
  SCSIFS_DiscOp64 drive=4 high=&40 low=&0
FileCore unable to read partition

Hi Jon,

Tried 0.39 on my Iyonix and the free space is now correctly reported as 1011MB on ADFS::4 but still no ADFS::5.

It is also OK on the ARMBook where it sees SDFS::0 correctly.

Finally it is OK on the ARMX6 apart from not seeing SDFS::1 still.

Coming along nicely so thanks for all your efforts.

Thankyou for all the reports so far, the feedback has been invaluable. I believe I’ve resolved most of the issues and having gone back through a lot of the reports, I have the queries below. If I’ve missed anything, please let me know.

@Doug Webb, looking back through all your debug/txt on the Iyonix, I suspect my code is querying the wrong IDE device, as when you switched the drive it gave the same response. Odd thing is, the code works on my RiscPC’s. Could you confirm ADFS::5 is the Slave on the Primary IDE?

@Anton Reiser/Raik, is your 4K drive size now showing correctly?

@David Pitt, is your Titanium now displaying SCSI::1 correctly?

@Alan Adams, is your SCSI::6 still showing “< unknown >” as the FAT volume name?

Other reports:
The 1MB rounding issue on Free space appears to be a difference between values returned by OS_FSControl 55 and other more user friendly methods. Although it’s annoying, I think we can ignore it as Free space is used for display purposes and not required for partitioning.

@Bryan and others reporting full disk FileCore drives that either aren’t showing or are showing as a single FAT partition. This issue might be a quirk of the version of HForm used to format the drive. From the few reports I’ve seen so far, they either have MBR partitions that only have a FAT partition, or the MBR partition is blank.

It’s a major issue to resolve as we run the risk of breaking GPT/MBR partitioning in an effort to detect full disk FileCore drives. The sensible route is to fix the MBR, but that’s also the most impractical. The alternative is to detect full disk FileCore first, but that will then ignore the MBR partition on Pi’s etc.

Another option is to check for a full disk FileCore, when an MBR is detected that doesn’t fill the disk, but that will detect ghost FileCore partitions that have legitimately been deleted.

The final option, given we’re moving to partitioning, is to make assumptions based on the presence of a Partitioning Module or OS variable that indicates partitioning is being used. ie Everything prior to partitioning support will expect to see a full disk FileCore on every drive – which mirrors how the OS works.

For the edge cases and to cover migration from pre to post partitioning, where there’s simply no way to determine if it’s a valid MBR or a full disk FileCore, it defaults to MBR and I provide an option to manually recheck the drive for full FileCore, overriding the detection?

I’d appreciate some suggestions, as whichever route I take is not going to be perfect. I’d particularly like to hear from the Partition bounty author and coder(s) to see how they propose covering this particular issue. There’s possibly a simple solution that I’ve missed.

@Anton Reiser/Raik, is your 4K drive size now showing correctly?

Yes… Look at Free and division by 1024.

Thanks a lot.

Hi Jon,

I suspect my code is querying the wrong IDE device, as when you switched the drive it gave the same response

ADFS::4 and CDFS::0 are on the primary interface.

ADFS::5 band CDFS::1 are on the secondary interface.

Cable Select is used not primary/secondary.

CDFS::0 is the prime DVD Writer and if I remember rightly this layout was suggested as best for burning CD’s/DVD’s from data on the primary hard disc. I could be wrong on this as it was so long ago that the Iyonix was purchased/set up and it isn’t my primary machine now.

Hi Jon,

I think the only issue I am seeing now is on MBR SD or memory sticks larger than 4GB with a single FAT32 partition less than 4GB. (This enables the Pi 4 to hardware boot either from SD or USB whilst allowing RISC OS to read the device)

Everything on Partition Manager works except that the FAT 32 partition name is shown as < unknown> in both the graphic and text. RISC OS SDFSfiler correctly shows the name Pi4boot.

Correction: It is only such SD cards which have < unown> names when plugged into either the SDcard slot or via an adapter into USB. Memory sticks show the name correctly.

Debug File is shown in updated Tests topic on this forum. Once again I will delete when you have copied it

Update 2: It ocurred to me that the difference might be because one of then is plugged into the Pi USB-C socket, so I sawpped them over. It is still the SD card partition whch shows < unknown>.

@David Pitt, is your Titanium now displaying SCSI::1 correctly?

It is showing as a single unallocated space.

Debug etc is here

ADFS::5 and CDFS::1 are on the secondary interface

That confirms by suspicion that my code is checking the wrong drive. It’s checking the Slave device on the Primary controller, instead of the Master device on the Secondary controller. I’ll fix the code.

Everything on Partition Manager works except that the FAT 32 partition name is shown as < unknown> in both the graphic and text

Is the partition red? Unless it’s a 4K drive, FileCore won’t be able to address the whole partition, so my code might not be able to read RootDir – this should be visible in the logs and the partition should be red.

@Alan Adam’s SCSI::6 has a similar issue except the partition is definitely accessible by FileCore. My code is a bit of a hack though, as it doesn’t follow RootDir cluster chains and assumes the ATTR_VOLUME_ID entry is in the first cluster. I’ll revisit this at some point.

@David Pitt, is your Titanium now displaying SCSI::1 correctly?

It is showing as a single unallocated space.

1. Only discover physical drives and ignore virtual drives from PartMan
2. Read physical sectors, bypassing any partitioning

I’ll wade through the PartMan code and see how its reading physical sectors.

@Alan Adams, is your SCSI::6 still showing “< unknown >” as the FAT volume name?

It still shows as , coloured blue.

Also, SDFS::1 still doesn’t show. SDFS::0 is correctly shown.

Debug output is here: http://www.adamshome.org.uk/RiscOS/partmgr-039-debug.txt

Also, SDFS::1 still doesn’t show

Could you upload the debug/txt somewhere please. If its a Titanium – it will be the issue I mention above.