SD Directory Error

I have Risc OS running on a Raspberry Pi model B. It was installed from RC6 image and I have installed other software from the NutPi SD. It has been running for about 3 months. Now, Checkmap reports Map inconsistent with the directory tree. If I run Checkmap on a fresh install of RC6 it reports map good. Somehow, installing software or normal use corrupts the directory tree. Is there a way to repair or rebuild it?

While I have not experienced this myself (touch wood!), I know of similar SD card failures, one of which is now recognised as a FAT volume, like the FileCore part had just “vanished”.

Can I put in request for better recovery tools than *CheckMap?

Can I put in request for better recovery tools than *CheckMap?

How about DiscKnight?

Note: DiscKnight can not check SDFS devices direct. It chrashed after you select SDFS as the filing system to check. Take the card to a cardreade and select SCSI and it will work fine.

Thanks for the suggestion to try DiscKnight. I tried it and have not been successful. My experience was the opposite of Raik. If I set filing system to SDFS it appeared to work work correctly to both check and repair, except that it came down to 2 errors which it repeatedly found but did not repair. If I put the SD in a card reader and selected SCSI the system hung permanently.

Thanks for the info. This is something that worried me given the official site talks about RISC OS 4 and…

As of version 1.49 DiscKnight has been compiled for ARMv7 compatibility so will run on systems such as Beagleboard with alignment exceptions turned on.

(my emphasis)

This makes me question if it is “current” with the new systems, results differ. Hmm!
Still, €10 for something that might work is better than “oh crap”!

Further update: If I run DiskKnight on the original SD made from the 2012-11-01-RC6 RISC OS distribution RC6-1876M, it reports errors which it cannot fix. Checkmap reports map good.

I don’t know to what extent the weird hybrid partition on the boot SD, with co-resident FAT and FileCore partitions, is misinterpreted by checkmap or by DiscKnight.

All the RPi systems that report these errors seem to otherwise work fine.

My experience was the opposite of Raik.

I have only test SDFS with the Pandora. May be it is a Pandora feature :-(

I don’t know to what extent the weird hybrid partition on the boot SD, with co-resident FAT and FileCore partitions, is misinterpreted by checkmap or by DiscKnight.

This is why it would ideally be better if DiscKnight was “current” with the new RISC OS devices. So there was an option “I know it looks like a FAT device… I know it feels like a FAT device… it isn’t”. There may be a way to say to DiscKnight that “the partition starts here” but how well would this work with the “unusual” FAT/FileCore hybrid?

Hi, despite having had RISC OS on my Raspberry Pi for several months, I haven’t run DiscKnight on it! I’ll have a look why it doesn’t work on SDFS, it should work from within the Pi on any FileCore based FS – has anyone tried the command line, rather than the front end?

It wont work straight off on the SD card when put in another machine as it doesn’t understand DOS partitions. Although DiscKnight can be told to search for a FileCore partion using the disc shape start and size parameters (it will only fix the FireCore data, not the partitioning information if that is corrupted).

But I would urge anyone using flash storage on the Pi or other boards, to rigorously backup the cards. I’ve found that unlike the very predicable failure modes of spinning rust discs, when flash goes it can be extensively scrambled by malfunctions of wear levelling, leaving a mess which DiscKnight can’t recover.

After installing firefox so that I could setup a portable router at RPi meetings, I read somewhere that a ramdisc would speed it up. So I installed “Memphis3”, but managed I supposed to cause some confusion with “Scrap”. A subsequent reboot reported that “checkmap” was necessary….

Until this thread, I was unaware of “DiskKnight”, so ran it on the SD, & here’s the result. So, is it likely DiskKnight can fix these problems?

DiscKnight 1.49 (05-Aug-2009) [32bit] CHECK ONLY

Arguments: SDFS 0

Boot Block – Defect List
Boot Block CRC is OK

Map 1 – Checksums
Zone Checks OK

• Cross check failed, is &C0, should be 0xFF
  Map 2 – Checksums
  Zone Checks OK
  Cross checks OK
• Map 1 corrupt, restoring from Map 2
  Map – Disc Record
  Map disc record matches boot record
• Zone 945 does not contain sufficent allocation for the map (is &FE000, needs &1D8C00)
• Zone 1890 disc end marker is incorrect (is &0 should be &1)
• Spurious system object found in zone 946 offset &0000 – allocating new ID &0002EE2F

Checking directory structure
………………………………………………………………………………………………………………………………………………………………………….

• Directory $.Apps.Network.!Firefox.defaults.profile size invalid (&FFFFFFFF)

• Directory $.Apps.Network.!Firefox.defaults.profile directory name length invalid (is -1) – CLEARING
  …………………………………………………………………………………………………………………………………………………………………………………………….

• 9 objects in map not referenced by directories

> Directory ?.chrome found

• Directory $.Lost+Found.chrome parent mismatch (is &0256D301, should be &05D6BF00)

> File ?.File000199F7 found

> File ?.File00019B8D found

> File ?.File0002332A found

> File ?.File000256D4 found

> File ?.File000256D6 found

> File ?.File0002EE2F found

> File ?.File000434DA found

> File ?.File000440BF found

Ok, first off I’ve not had any trouble running DiscKnight on the Pi, the front end works fine with SDFS. If anyone does have a problem, please sent a bug report to the address in the documentation – that’s the best way of getting it fixed, as I don’t read this forum very often.

Second I get the same errors on my 2GB SDCard:-

• Zone 945 does not contain sufficent allocation for the map (is &FE000, needs &1D8C00)
• Zone 1890 disc end marker is incorrect (is &0 should be &1)
• Spurious system object found in zone 946 offset &0000 – allocating new ID &0002EE2F

I suspect that the tool which creates a FileCore partition on the disc isn’t quite doing it right. The Lost+Found entries appart from 2EE2F are likely to be orphaned objects lost by not dismounting the disc properly, or a crash on the Pi.

I performed a repair a couple of times, and DiscKnight wasn’t happy with zones 945 and 946, so I’ll need to do further investigation there.

I suspect that the tool which creates a FileCore partition on the disc isn’t quite doing it right.

So does that suggest that checkmap in the ROM has been modified to compensate for the “image” trick, since it reports “Map good” on a RPi SD?

Apart from this major error, so far I’ve found RISC OS to be very tolerant of SD extraction “mid flight”, it politely asking for the SD card to be put back when necessary :)

The 2GB image was indeed created using a new tool – a prototype of a new formatter of mine that’s been in gestation for quite a while now. It does do a few things differently than HForm, deliberately relaxing some of the restrictions with an eye to future FileCore extensions for larger discs and the like. For example, it can set the bytes per allocation bit to less than a sector, which allows finer granularity of allocation than HForm does for 2GB discs.

It also, unlike HForm, permits the disc object containing the two copies of the map to straddle more than one zone; it looks from those results as though DiscKnight doesn’t think that’s a valid thing to do. I’m not so sure: it’s certainly true that HForm never did create multi-zone maps, but even if there are any such restrictions, I think they’ll have to go in order to support larger discs anyway. Fortunately, I don’t think DiscKnight’s action of allocating an alternative fragment ID for parts of the map in subsequent zones will have done any real damage.

As for the disc end marker, I may have got that wrong. Off the top of my head, I don’t think I added one, but then I wasn’t clear if it was actually mandatory. Flash-based devices tend to be an integer number of megabytes long (due to the fact that they’re made up of NAND flash pages that are often 1MB in size) so they’re relatively unlikely to have a situation where the last allocation bit contains some sectors that hang off the end of the disc, so there’s no need to mask it off with a pseudo-defect.

I’ll make a note to go back and check this stuff again as soon as possible – clearly it’s quite important that the standard disc image is valid. Though it’s obvious in hindsight, we at ROOL haven’t been checking them with DiscKnight as standard (otherwise this would have come up before!)

One thing: I can confirm that there is no special support in *CheckMap for this. The disc image is designed such that it’s a standard “super-floppy” (unpartitioned) FileCore image to a FileCore-aware OS. It just so happens that to any OS that isn’t, it also appears to be a partitioned image containing a FAT partition.

Hi Ben, only just seen your reply, but have found the same as you have described. The issue DiscKnight has is with the 2 map copies taking more than one zone, which was never allowed before, but SProw’s new FileCore docs show is supported by new FileCore versions. I’ll modify DiscKnight to cope with this, but in the mean time I would recomment not recommend using it, as I’ve managed to make one card not bootable – DiscKnight was able to retrieve all the file though.

The other issue reported with the end marker wont cause any problems, lots of third party SCSI formatters would leave it out, and FileCore was fine with it. I put in a fix for completeness, and you never know if FileCore would get more fussy in the future.

The memory card image is quite clever as you say; to both the Pi and other RISC OS machines it looks like a FileCore format 2GB disc with containing a 48MB DOSDisc image FS file (!Boot.Loader). To the Linux loader and a PC it looks like a disc containing a 48MB FAT partition and a 1.8GB empty partition. This does rely on FileCore leaving the DOSDisc at the correct fixed location on the disc, but it should never move it as auto-compactions wont work across zone. The user can muck things up by moving or deleting it though, which will stop RISC OS booting.

But in any case it should be possible to run DiscKnight (when upated) on the disc either from within the Pi, or from another RISC OS machine, so fix faults. Although please be warned that when memory cards get corrupted, they can be almost completely scrambled, so backing up is essential. Copying files using RISC OS over a network or from a card reader is very slow, a much quicker way is to use the Win32DisckImage program to read an image of the whole card, and a similarly quick to write it back again.

Just to update the topic. DiscKnight 1.50 now correctly handles the new FileCore variant, allowing files of up to 4GB, the multi-zone disc map and the special partitioning for the Raspberry Pi.

You may still see an error for a “missing disc end marker”, which was produced by some early versions of the formatting tool. If there are no other errors on the disc, it is safe to ignore this and a repair is not necessary.