Three corrupted discs

To have a broken directory on one Titanium SSD might be considered unfortunate but then to find via DiscKnight that two different Pi’s, one SSD and the other SD, have “an object not referenced by directories, disc is bad” errors is off the scale. All are running the latest OS5.29.

WTF.

Will Test Filecore? I’m sure it wasn’t anything offensive :)

Sunspots, has to be sunspots.

Well….. https://edition.cnn.com/2023/07/14/world/solar-maximum-activity-2024-scn/index.html

No, this is RISC OS being RISC OS, i.e. very crashy, which can inconsistencies between map and directories like this.

DiscKnight should be able to fix it, just make sure:-

1. You are only running DiscKnight when you do a repair (no email programs running)
2. Reboot immediately after or RISC OS will not pick up what DiscKnight has done.

;) No. Not always- rock solid here and a joy to use!

My Kinetic RISC PC every summer had an issue where I needed to use DiscKnight- too hot! And each time, I’d forget to reboot- until I noticed I’d run out of disc space- a definite case of RTFM.

Also, there was, back in the Acorn developer days, someone who struggled to debug their program over a hot summer; it took time- before they realised their fridge was causing main spikes, which was the problem.

So there are other factors to consider. Yet, it seems plausible to be a bug based on the circumstances, but not necessarily.

Anyway, DiscKnight to the rescue- reboot after using it to fix disc errors! :)

PS. Don’t forget to try Verify- it’ll hide bad disc sectors, which might suggest problematic media.

I don’t think verify is much use on modern (flash based) media, as there’s no longer a correspondance between the sector as RISC OS sees it, and the sectors on the media itself (re. wear levelling etc).

The three disc corruptions reported had been preceded by a Broken Map event on a USB pen. After a long period of stability that was a lot grief all in the space of an hour.

The Pen was an old ARMini installation, I was just having a look at it before using it for something else. It was OK last time it was looked at. So forget that one.

Next up was the Titanium reporting a ‘Broken directory’ in NewsDir. There had been some instability on the Titanium a couple of months ago which culminated in a catastrophic disc error and weird crash. I thought that might be the SSD failing so it was unplugged after moving the disc image to another SSD. That was all good until yesterday, so it could be that the second SSD is showing its age. Replacement SSD’s have been ordered.

The two Pi’s have similar disc images, one SSD the other SD, and both showed the same fault, orphaned files. It is not known when the corruptions occurred but a buggy app had been investigated on both, or there is some other malevolence. Time will tell.

Then there is this. It is a reply to Bug 606 which is a request for the CPUSetup configuration tool to be included in the HD4 image.

… CPUSetup is a sticking plaster to help with early Pis which could emulate the old rotated load behaviour.

That’s not possible on later ARMv7 and ARMv8, so all it really does is give you a length of rope to hang yourself by enabling unaligned LDR/STR leading to unspecified crashes or odd behaviour, which isn’t something that should be offered as a user facing choice!

That’s food for thought.

My machines used to all run with Alignment Exceptions On, in strict mode. Then an app turned up that requires them to be Off. In a possibly sub-optimal decision Exceptions got turned Off on the Titanium. Exceptions remained configured On on the Pi’s and disabled by the app. The relative merits, or otherwise, of Exception On or Off has be discussed in the past and the conclusion I came at the time was ‘strict’ was best as otherwise unseen issues could occur that might bite late. I need to remind myself of that. The CPUSetup tool is now removed and if stuff does not work in strict mode then stuff does not get used.

Most importantly, a big shout out for DiscKnight.

I don’t think verify is much use on modern (flash based) media, as there’s no longer a correspondance between the sector as RISC OS sees it, and the sectors on the media itself (re. wear levelling etc).

*Verify was designed to read all the sectors on a hard disk and if any were unreadable, or required retries (command line version will indicate that, desktop only does unreadable) it would report the address, you could then use the *Defect command to get the system to ignore that dodgy part of the disc. That was useful back in the very primitive ST506 days when you might lose a few sectors, but due to the high cost of drives you would carry on using it. Modern IDE discs introduced a pool of spare sectors and if it detected one which was failing would transparently substitute a spare. So if it got as far as the OS noticing it couldn’t read part of the disc, the area of damage was probably already pretty extensive and growing, so the recommendation was to ditch the disc entirely, as they were much cheaper, and it isn’t worth the risk.

Flash media goes even further and swaps blocks around to stop any part of the disc wearing our before the other, some also hVE over provisioned so it has a proportion of spare blocks so worn blocks can be retired without decreasing the disc size. SD cards normally only have minimal wear levelling and no over provisioning, where as SSDs have sophisticated wear levelling and some amount of over provisioning, USB sticks can be anywhere in between.

What I don’t know is if any flash media will get as far as running out of good blocks and report failures to the OS (disc error something when reading) and be detectable by *Verify. My experience of SD cards and USB sticks is that when they run out of good blocks is they either go read only (decent stuff like Samsung and SanDisk) or fail entirely (cheap stuff). The third option being a *Verify could lock up or crash the system due the flash controller getting upset.

I’ve encountered an SD card that reported failures in reading certain files: this was a card formatted and used in a camera, being accessed on a (Linux) PC via a card reader. A small number of files were corrupted. As they weren’t irreplaceable, I simply copied off what I could and binned the card. It’s the only time (so far!) that I’ve had issues with an SD card.

I’ve had no partial failures on pen drives; either they’ve worked fine or failed completely (about 5 in 25 years).

DiskKnight does indeed do a nice and good job… I managed to switch the facial expression from devastation to a bright grin for some chap since we managed to get all data from the broken disk for him and put all on a new one.

I dare say if I need to use *Defect I rather get a new disc since some issue indicates the EOL of the disc is nearing.

Perhaps ROOL sould instead of amending FileCore etc. swith to use a proven disc format like ext4 or even exFAT instead – amongst other things you can then extend life of an SSD by plugging it into a modern system and issuing a trim (which implies that the OS in use understands the format) – side effect would be that big files are no problem… and if neede tons of decent recovery tools are at hand using if needed a different OS.

DiskKnight does indeed do a nice and good job…

One cannot speak highly enough of DiscKnight.

Not so long ago, a little mishap. I had my Android phone format the FAT partition. Only it didn’t. It threw that away and reformatted the entire media. Tragedy! The FileCore boot information was wiped out. All my files lost ¹. It was now a big empty FAT32 volume.

Enter DiscKnight, that looked at the disc, rummaged, rummaged some more. Then did a bit more rummaging. And finally reconstituted the boot block and drive information to point to the $ directory.

I could no longer use it as a boot device, the “special magic” that made it work was well and truly buggered, however I was able to recover everything on the FileCore side to a new SD card.

swith to use a proven disc format like ext4 or even exFAT instead

Can you provide a format that a, is not annoyingly case sensitive and b, supports the use of metadata (remember, RISC OS does not use file extensions).
It is acceptable to fudge things if you’re using an emulator on an alien OS – HostFS does this – but it has limitations ^{2 3} so it isn’t something that the OS itself should be doing in order to work with a different format.

by plugging it into a modern system and issuing a trim

One would hope that any sufficiently major upgrade to the filing system would support the use of periodic trim, at the very least.

and if neede tons of decent recovery tools are at hand

This is why I don’t put valuable things on FileCore and keep periodic SD images and copy files to other media/machines.
*CheckMap is useless, it’s only utility is in either bailing out or simply crashing if there’s a map fault (and even then it takes an order of magnitude longer than DiscKnight to scan the media). *Defect is outdated and shouldn’t be used, period.
And the only other help, should you not fancy rummaging around with a low level sector editor, is the aforementioned DiscKnight. Which is good and highly recommended… but it’s kind of lame that Acorn never created at least some sort of basic recovery tool (especially how often “Broken directory” BITD). At least MicroSoft gave us SCANDISK and later CHKDSK. Both were kind of crap ⁴, but both could fix enough problems to ‘recover’ a drive that wasn’t too badly messed up – there did tend to be issues of cross-linked files.

It’s the only time (so far!) that I’ve had issues with an SD card.

Let’s see. I’ve had one USB stick that habitually corrupted the root directory, interestingly returning different rubbish every time it was read. I’ve had another USB stick that would read about 4MB of any file (perhaps filling it’s internal buffer) and then simply die once it tried to pass that point. I’ve had one SD card that not only became unreadable but suddenly got hot enough to make burning plastic smells (and, yes, the thing was hot when I popped it out). And I’ve had one accept any data written to it, creating zero length files. And a couple that simply died.

None of these on RISC OS. Apart from the hot one, most of these things were probably killed by the PVR (imagine the number of directory/map updates to write a 3GB hour and a half recording in realtime) and utterly slaughtered by the dashcam (it dumps video in three minute chunks, around half a gigabyte per file, automatically erasing old files to make new ones, an 16GB card only holds 35-45 minutes. That, in two parts, is my daily commute.

Touch wood, I have only had two media failures under RISC OS. The first was my Pi1 which was obviously underpowered but didn’t say anything. Broken directories, broken maps, all sorts of chaos. New, better, power supply (ie not a crappy phone charger) and suddenly everything was fine. The processor will happily work with less power than the SD card will accept! [which makes me wonder, as USB ports can go down to 4.5V, or 4.35V through a bus-powered hub, and if one has an SD card in a USB reader…?]
The other failure? A soft failure. As described above. Totally my fault.

My machines used to all run with Alignment Exceptions On, in strict mode.

That’s how my setup goes. It’s a Pi3 so there’s no rotated loads. Any unaligned reads will throw an error by default.

Then an app turned up that requires them to be Off.

Behaviour depends upon what. A few of the older GCCSDK programs needed rotated unaligned loads. I think I have deleted everything that works like that, and when stuff was updated due to the SWPbomb, I think it was pretty much all purged. It’s an anachronism of the past.

More recently, the Iris browser needs modern (non-rotated) unaligned loads. I think the script interpreter or something uses Thumb code? It does explain/ask so I let it go and do it. Kinda won’t work right without.

In a possibly sub-optimal decision Exceptions got turned Off on the Titanium. Exceptions remained configured On on the Pi’s and disabled by the app.

If you have exorcised the old cruft that did rotated loads, then really there should be no difference if the exceptions are on or off.
I think most of us here have exceptions on because historically some software used rotated loads while while newer processors did not support this (I think the RisCPC, Iyonix, and Pi1 are the only RO5 machines that do), so the program would appear to work, but since the data being read in would not be the rotated load style, whatever it was trying to do would end up wrong. Switching on exceptions would crash out such software so you’d know it was failing in some way.
But, it’s 2023 and the last processor that did it the old way was the Pi1 that was released a decade ago. There oughtn’t be any old rotated load stuff floating around any more.

The CPUSetup tool is now removed and if stuff does not work in strict mode then stuff does not get used.

Make a judgement based upon what the software in question is. Like I said, Iris requires unaligned loads (exceptions off). That’s perfectly fine, and it may be that in time more software might start to make use of them when necessary.

Older software cannot use unaligned loads. If it’s sufficiently old, it’ll be expecting rotated loads. These simply do not work. The processor just doesn’t do it, and hasn’t since the ARMv7 (Beagle/Pi2). We turned exceptions on to trap this sort of thing.

But, do note, we did that a decade ago.
Software that still works like that should be few and far between.

My advice, keep exceptions on for normal use (doesn’t hurt to be paranoid, there are still plenty of links to Iyonix-era software floating around), but don’t be afraid to let Iris turn them off if it needs to.

¹ Slight exaggeration as it was a stripped down copy of the stuff on my main Pi, but then I’d need to pull everything using ShareFS and that’s not as fast… assuming that ShareFS doesn’t have one of it’s habitual breakdowns (the red hourglass routine). Copying SD to SD is much simpler.

² For example, you cannot change the type of a file that is open. Works fine on RISC OS, fails on HostFS.

³ It’s not hard to arrive at a situation where multiple files can appear on RISC OS as having the exact same name, and you can only access one of them.

⁴ All too often, a file would be “repaired” (note the scare quotes) by replacing a sector with a bunch of null bytes, which freaked out text editors and messed up executables. It might have been better if the tool saved a list of the files it had touched so you didn’t have to try to memorise stuff flashing up on the screen.

@ Rick

Can you provide a format that a, is not annoyingly case sensitive and b, supports the use of metadata (remember, RISC OS does not use file extensions).

JFYI, you CAN configure ext4 to be case-insensitive. It’s just that most Linux distro follow certain defacto standards and never enable it, but to have a a filesystem partition that is case-insensitive with ext4 it’s as easy as:

mkfs -t ext4 -O casefold /dev/vdX

Where vdX is the partition you want to FORMAT (<—- watch out!) with ext4 and have it case insensitive.

Before you do that, just make sure it your copy of ext4 has been compiled with UTF-8 support (CONFIG_UNICODE=y) and to do that with an exsisting Linux system run:

cat /sys/fs/ext4/features/casefold

It’s also a good idea to make sure the whole stack is supporting UTF-8 (which, at this point it should, but better safer than sorry) with:

cat /sys/fs/unicode/version

HTH

JFYI, you CAN configure ext4 to be case-insensitive.

Bon Jovi “…half way there”

BeFS (from BeOS) supported metadata, presumably a descendant in Haiku still does. It was case-sensitive, however.

I must admit I can’t see how you couldn’t use ext4, storing the file type information using the three-hex-digits following a comma as used by the HostFS systems on emulators. The OS could then hide this aspect from the user. File extensions aren’t an intrinsic part of UNIX-like systems, as anyone who has typed

rm *

as anyone who has typed rm * …will know

No recursion? Where’s your spirit of adventure? :)