Zip 250 drive provokes "Filecore in use"

A few days ago, I tried copying a large number of files to a Zip 250 drive (USB connected). The result was a “Filecore in use” error after a few hundred files.

The version of RISC OS I’m using was built on 2011 November 21.

a large number of files to a Zip 250 drive

Can you help narrow it down any further? For example

• doing it again, do you get the same result or could some rogue application have scrambled the OS workspace?
• is it something special about the files, the number of files, the size of files, or the quantity of files (for example, 1MB x 100 being different to 100MB x 1)
• does the same thing happen copying to another filecore filing system like RAMFS (ie. can the USB or SCSI stacks be eliminated)

Did FileCore recover and allow you to continue accessing your drives? The reason I ask is that I’ve just found a bug/flaw in the way that re-entrant heap operations are handled (and potentially any othe re-entrant operation). If a heap op gets interrupted by an IRQ, and the IRQ handler attempts another heap op, then the kernel will first complete the first operation and store the result ready to be returned to the foreground task. The problem is, if the SWI generates an error, it will end up using one of MessageTrans’ IRQ error buffers instead of a foreground buffer. There are only four IRQ buffers, so if the IRQ handler generates more than a couple of errors (e.g. by trying to write to disc – which would likely generate a filecore in use error if you’re busy copying files) then it could trample over the error that’s going to be returned to the foreground process. The foreground process would then fail to recognise the error and report it to the user.

In my case, I was getting lots of “SWI &4C88E not known” errors, caused by EtherUSB trying to use the SysLog module. However after modifying the kernel to make a copy of any heap errors generated by suspended foreground operations I was still getting the errors, so it looks like in my case MessageTrans is running out of foreground buffers as well. If I kill EtherUSB I also get “Unknown OS_ScreenMode reason code” errors, which puzzles me a bit since 99% of the software running on my BB was built from ROOL sources – so should know what the supported reason codes are. It also doesn’t help that I was running my heap test program from a taskwindow, so there’s ample opportunity for the MessageTrans buffers to be trampled if the task gets swapped out as soon as it returns from OS_Heap with an error.

I think we could really do with a better error handling system. It’s too late to change to a system where the caller has to provide the error buffer, but we could at least go through all the modules and make sure that any errors which don’t have substituted parameters get cached in static buffers instead of clogging up the MessageTrans buffers.

It’s too late to change to a system where the caller has to provide the error buffer

Although, we could add a new OS_CallASWI that accepts an error buffer pointer. If the shared C library was recompiled to make use of it then it could drastically reduce the number of error buffers being trampled (although the SCL may still need to buffer the errors locally, with one set of buffers per task).

Some answers. I copied all 12057 files, 212212887 bytes, in my RiscOS directory to NAS without problem. When I tried to copy them from NAS to Zip, I got the same error as the first time. The number of files was 823, which is, IIRC, identical or very close to the number of files before it fell over the first time when copying from HardDisc4 to Zip.

On restarting, I was able to copy another 176 files to the Zip before getting the same error. The first 823 files total 16528785 bytes; the next 176 total 1270095 bytes.

Must join a conference call now; I’ll post again later.

FileCore didn’t recover. The only thing I could find to do was restart RISC OS.

The entire 239 MiB surface of the Zip disc verifies OK.

There is already one other directory on the Zip disc, containing 407 files, 17954223 bytes.

In order to make sure there’s no power supply issue, I’ve connected a separately powered hub to the BBxM and I’m going to repeat the copy attempts with the Zip drive connected to that. The drive has no means other than USB to power it. The drive has a rating label on it stating 500 mA maximum consumption. The cable that came with it has, unsurprisingly, only one plug at each end, i.e. it’s not cabled to take more than one unit load of power.

I copied as many files to RAM disc as would fit (128 MiB RAM disc, 7692 files, 80543918 bytes) and copied them over to the ZIP disc, on the separately powered hub, after deleting the previous copy of the files (i.e. I’m starting at the same fill level of the target disc each time). This operation fell over at 819 files with an AODT at &FC2B8828 again.

I’m not sure where to find the *where command, but &FC2B8828 appears to be in the SCSISoftUSB module, which is based at &FC2B7244/&2002F594.

So 12000+ files from FileCore ADFS via (I assume) LanManFS to NAS is OK.
8000+ files from NAS to FileCore RAM disc is OK, until the disc was full anyway.
But either NAS or FileCore ADFS to ZIP (via SCSIFS) goes wrong after ~800 files.

Two further things to try

• Copy FileCore ADFS to FileCore RAM disc
• Copy to a USB memory stick (if you have one)

That should narrow in on something in the USB/SCSI side or a quirk of the drive itself (looked suspiciously close to 2^24 where it gave up).

I’ve also had a strange problem today with Safestore in that once it had completed it’s backup of the Beagleboard harddrive to my NAS it then immediately restarted the backup process and then eventually after about 10 attempts at trying to autostart the backup Safestore fell over with a Fatal error message.

I tried uninstalling Safestore and re-installing and also deleting all it’s data and still got the same message. I even went back to an early copy of the !Boot image just incase all with the same result.

Eventually I tried a complete reblown image of RISCOS, using SDCreate, on to the MicroSD card and rebooted and this time Safestore went through OK.
I used copies of the CMOS and ROM from the SDCard but reinstalled the U Boot and MLO files so is there a possibility of something corrupting the RISC OS image?

Doug

How do I get an ADFS drive on a BeagleBoard xM?

Don’t forget, HardDisc4 here is on USB. It never fails.

12057 files copied from HardDisc4 (USB) to 2 GB USB stick without problem.

I don’t know if it’s an issue, but the Zip disc has an IDLEN of 15 bits. The USB stick has an IDLEN of 18 bits.

How do I get an ADFS drive on a BeagleBoard xM?

That’d be tricky, your original post didn’t mention what hardware you were on nor where the files were being copied from.

That new information does add to the knowledge, so it’s either (as you say) something different about the formatting or something quirky about the Zip drive.

You could format the memory stick down to a smaller-than-it-really-is size to get idlen to match. I can’t think of a nice way to do that other than hacking HForm a bit, or just buying a smaller stick from ebay.

For Zip disc quirks, how about copying 600 files at a time, dismounting, and unplugging the drive (ie. to force a power cycle) then copying the next 600.

Does command line copy behave any differently?

The command line copy has fallen over at the same place, which is within RiscOS.OMAP3Dev.castle.RiscOS.Sources.Apps.Paint.Test – the copy has gone as far as a file called TestInsCol. The source file purports to be BASIC, but when I examine it, it looks to be seriously in error. There are 17 lines of comments, all numbered as line 0. Zap loads the file and then puts up a temporary error message over the horizontal scroll bar: “Warning: BASIC file seriously corrupted – truncated on loading”. There are several other BASIC files in the directory, all but one of which appear to be almost identical to TestInsCol. DiscKnight gives the source and ZIP discs a clean bill of health. To be clear: the files appear to be identical on source and ZIP discs, i.e. the source is damaged, the copy is a true copy. There are 48 items in the directory on the Zip disc, of which 45 are files and 3 are directories (c, h and o, of which c contains one file; o and h are empty).

Having copied 820 files onto the Zip disc, it isn’t possible to copy either the next file that should be copied (a Sprite file called True256) or an undamaged text file into that directory. Any attempt causes the AODT seen before, necessitating a reset.

I’ve used SCSIForm (V 2.55 17 May 2005) to reformat the disc; the copy falls over at the same place.

At least it’s repeatable.

The format allows long names. There isn’t any limit at 48 objects per directory, is there? And even if there were, it should be handled cleanly, shouldn’t it – a message to the effect that the directory is full, not an AODT and Filecore in use?

I’ve removed the strange BASIC files from that source directory. There are now less than 48 items in the directory. The copy proceeds to 960 files, but falls over instead at another directory that has more than 48 entries; 47 are copied, then AODT.

Today I’ve connected the Zip drive to the Iyonix, running vanilla RO 5.16, and repeated the same copy operation. It has stopped with an AODT at exactly the same point – a directory with 47 files in it, which should have a few more.

Interestingly, after the AODT, I was able to look at the Iyonix’s built in HDD (ADFS) without problem, but every attempt to look at the Zip drive (SCSIFS) resulted in “Filecore in use”.

It wouldn’t be much effort to get all the content off the ZIP disc in (e.g.) 1 MiB files (there would be approx. 239 of them) so that we could examine the content of the map, etc. without too much difficulty. Would anyone find that useful?

I did have a detailed look at disc structures a very long time ago, but I don’t know what directory entries with long file names look like.

I’ve got 239 files off the Zip disc, each 1 MiB. File names are “000” to “238”. In file “132”, I can see seven complete copies of all the filenames in the last directory written (i.e. the one it was writing when it fell over), plus one copy of the last 28 filenames. Although the Zip disc was formatted to allow long filenames, all the filenames are 10 or less characters in this directory. The other 4 files that should have been written into this directory also have names not exceeding 10 characters.

Here’s the 512-byte block of the file containing the last 28 filenames:

0005D000 : 4D 65 6E 75 43 72 65 61  74 65 0D 00 4D 65 6E 75 : MenuCreate..Menu
0005D010 : 49 6E 0D 00 4D 65 6E 75  55 74 69 6C 73 0D 00 00 : In..MenuUtils...
0005D020 : 4D 69 73 63 45 76 65 6E  74 73 0D 00 4D 69 73 63 : MiscEvents..Misc
0005D030 : 55 74 69 6C 73 0D 00 00  4D 6F 64 48 64 72 0D 00 : Utils...ModHdr..
0005D040 : 4D 6F 64 53 74 75 66 66  0D 00 00 00 4D 73 67 73 : ModStuff....Msgs
0005D050 : 49 6E 0D 00 4F 70 65 6E  0D 00 00 00 4F 70 65 6E : In..Open....Open
0005D060 : 44 69 72 0D 4F 70 74 69  6F 6E 73 0D 4F 53 43 6C : Dir.Options.OSCl
0005D070 : 69 49 6E 42 6F 78 0D 00  50 61 74 68 4D 75 6E 67 : iInBox..PathMung
0005D080 : 65 0D 00 00 52 65 63 61  63 68 65 0D 52 65 64 72 : e...Recache.Redr
0005D090 : 61 77 0D 00 52 65 73 47  65 74 73 0D 52 4D 41 6C : aw..ResGets.RMAl
0005D0A0 : 6C 6F 63 0D 53 61 76 65  44 69 73 70 6C 79 0D 00 : loc.SaveDisply..
0005D0B0 : 53 63 72 6F 6C 6C 0D 00  53 65 6C 53 74 75 66 66 : Scroll..SelStuff
0005D0C0 : 0D 00 00 00 53 65 6C 53  74 75 66 66 30 0D 00 00 : ....SelStuff0...
0005D0D0 : 53 65 6C 55 74 69 6C 73  0D 00 00 00 53 65 6E 64 : SelUtils....Send
0005D0E0 : 41 63 6B 0D 53 65 6E 64  4D 73 67 0D 53 6F 72 74 : Ack.SendMsg.Sort
0005D0F0 : 44 69 72 0D 53 70 72 55  74 69 6C 73 0D 00 00 00 : Dir.SprUtils....
0005D100 : 53 74 72 55 74 69 6C 73  0D 00 00 00 53 75 73 73 : StrUtils....Suss
0005D110 : 41 70 70 6C 69 63 0D 00  00 00 00 00 00 00 00 00 : Applic..........
0005D120 : 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 : ................
0005D130 : 00 00 00 00 00 00 00 00  00 00 00 00 05 D9 41 00 : .............ÙA.
0005D140 : 00 DA 41 00 00 DB 41 00  01 DC 41 00 0C D9 41 00 : .ÚA..ÛA..ÜA..ÙA.
0005D150 : 00 DD 41 00 00 DE 41 00  0A DC 41 00 00 DF 41 00 : .ÝA..ÞA..ÜA..ßA.
0005D160 : 01 E0 41 00 00 E1 41 00  01 E2 41 00 00 E3 41 00 : .àA..áA..âA..ãA.
0005D170 : 09 E2 41 00 01 E4 41 00  08 E4 41 00 00 E5 41 00 : .âA..äA..äA..åA.
0005D180 : 01 E6 41 00 01 E7 41 00  00 E8 41 00 09 E7 41 00 : .æA..çA..èA..çA.
0005D190 : 01 E9 41 00 00 EA 41 00  08 E9 41 00 01 EB 41 00 : .éA..êA..éA..ëA.
0005D1A0 : 00 EC 41 00 00 ED 41 00  00 EE 41 00 00 EF 41 00 : .ìA..íA..îA..ïA.
0005D1B0 : 01 F0 41 00 07 EB 41 00  01 F1 41 00 01 F2 41 00 : .ðA..ëA..ñA..òA.
0005D1C0 : 00 F3 41 00 0D E0 41 00  08 F1 41 00 00 F4 41 00 : .óA..àA..ñA..ôA.
0005D1D0 : 0C EB 41 00 00 F5 41 00  01 F6 41 00 00 F7 41 00 : .ëA..õA..öA..÷A.
0005D1E0 : 05 F6 41 00 01 F8 41 00  00 F9 41 00 09 F6 41 00 : .öA..øA..ùA..öA.
0005D1F0 : 00 FA 41 00 00 FB 41 00  6F 76 65 6E 47 00 00 F9 : .úA..ûA.ovenG..ù

It occurred to me to try generating a set of files with the same names and lengths, so I wrote a BASIC programme to create them using OS_File 10 into a directory in the root directory. Lo and behold, it falls over at the same place. It even happens just after I format (not merely initialise) the disc. It happens the same on a BBxM or an Iyonix, though clearly the addresses of the AODT are different.

So it’s repeatable, and it’s not related to the path name; this one, being only one level down, is a far shorter path than the original.

Presumably it has to be related either to packing the filenames into the directory, or packing the files into the map.

The BASIC programme is 79 lines, 3365 bytes. I’m happy to pass it on to anyone who would like to try it.