ADFS Image Filing System?

A relatively quick way to test if FileCore re-entrance is a issue, would be to modify the CRamFS code to use a file instead of RAM and see if it works.

is the re-entrance issue a global one or limited to the same instance?

My understanding was it was global, but there’s no documentation that details so I’m not sure why I came to that conclusion. By pure coincidence I’m adding HDF support to the next build of ADFFS so I’ll code it as a new filesystem and confirm if its global or instance based that triggers “FileCore in use”

Font Directory Pro is implemented as a FileCore FS (FileCore%FontFS) which holds the fonts in a D format harddisc image as a file which can live on a FileCore FS like ADFS or SCSIFS happily.

The less said about that, the better.

It’s certainly an extraordinary way to arrange and manage fonts.
Was it done as some sort of obscure copy protection?

It’s certainly an extraordinary way to arrange and manage fonts.

It seems to work all right here!

What are you actually after Simon?

Showing my ignorance, apparently! I’ve since found https://www.riscosopen.org/forum/forums/11/topics/2244 which is very informative (I particularly like the information about the magic Boot DOSFS image and fooling non-RO systems, and that everyone but me seemed to understand that the format comes from FileCore, not the individual file system).

I’m looking at trying to provide a blocking FS that will allow other tasks to continue running while data is fetched from whatever medium. I now think that CRamFS is a good starting point, just 1000 lines of code.

I’m adding HDF support to the next build of ADFFS so I’ll code it as a new filesystem and confirm if its global or instance based that triggers “FileCore in use”

I knocked up a read-only HDFS over the weekend to test this and as noted by Elesar, it works so “FileCore in use” is instance based.

Going further, I checked a selection of IDEFS implementations many of which explicitly generate disc error &A0 (“FileCore in use”) when the disc is marked as “active”. So my guess is FileCore does support reentrancy, but the instances may not.

I’m looking at trying to provide a blocking FS that will allow other tasks to continue running while data is fetched from whatever medium

I presume you mean non-blocking? I’m not really understanding what you’re after, but its starting to sound like a “background transfer” based read/write routine and not a new Filing System.

I mean blocking, as in the current task gets blocked, but other tasks get to use the processor(s). The idea is to decouple the tasking/memory management from the Wimp and have tasks that can keep working while others are performing disc or screen operations.

Ah. And this “current task”, is it something you’ve written or a generic task you’re talking about?

I’m assuming the later, otherwise you’d just split read/write operations into smaller blocks and issue Wimp_Poll between them.

Unless explicitly disabled (by setting R5=0 at FileCore_Create), all Filing Systems support background transfer. The only way I could see that being made non-blocking to other tasks is for FileCore_DiscOp to be modified to split read/write operations up into small blocks and continue issuing Wimp_Poll until the operation is complete.

It would have to pause any other tasks that attempt to read/write to that FS until the ongoing operation completes and has notified FileCore via the “background op complete” call.

I’m guessing you don’t want to go to such extremes though.

The idea is to decouple the tasking/memory management from the Wimp and have tasks that can keep working while others are performing disc or screen operations.

You’ll run into the same issue that affects any plan to spread tasks around available cores – and that is that having tasks able to run non-sequentially breaks message broadcasts, especially those which expect some sort of response (recorded messages).

Plus, in your example you mention screen operations. There’s the very obvious problem that there’s only one system VDU state, and any app that is currently active assumes that it owns that state. Normally it would only mess with graphics on a redraw, but it could also do things with fonts in redirected sprites. Suffice to say that an app doing screen operations in the background might find the state randomly changing as it is trying to do what it does.

By the way, if by screen ops you mean a long redraw… I’m not sure you can legitimately poll until you have cleared your outstanding redraw rectangles.

You’ll run into the same issue that affects any plan to spread tasks around available cores – and that is that having tasks able to run non-sequentially breaks message broadcasts, especially those which expect some sort of response (recorded messages).

Personally, I’d prevent threads using SWIs like the Wimp to simplify task management. Once a message is sent by a task, the system would wait for all active Wimp tasks to yield by calling Wimp_Poll(Idle), handle the message and replies in the current cooperative fashion, then allow the other Wimp task to work in parallel again. Actually, I’d do the same for all non-null events as a first implementation, then relax the restrictions once the rest of the system is more threads friendly.

the system would wait for all active Wimp tasks to yield by calling Wimp_Poll(Idle)

Which implies the sending task would be stalled if another task is “busy”.

The basic problem is that the current method is not designed to work in an asynchronous manner.

Probably better in terms of system management is a hybrid akin to what you suggest, but with a timeout (such as 3 centiseconds) that if a task hasn’t polled in that time, it’ll simply be skipped over (et tant pis).

My personal feeling is that if we’re going to try spreading tasks around other cores and looking at basic preemption, then it ought to be possible to register an address with the Wimp that can be called when events happen, outside of the regular polling loop, with the understanding that all events handed to a “busy” task are advisory. The task doesn’t have to do something, but it should if possible.
In this way we don’t end up with zombie tasks that fail to redraw or correctly move windows when “busy”.
It also neatly sidesteps the problem of queuing a set of events for when an app does poll…just don’t. Because events don’t carry a timestamp so there’s no way to be certain that they aren’t way out of date.

Obviously this won’t apply to BASIC…well, not until Steve has added something to Basalt. ;)

there’s only one system VDU state, and any app that is currently active assumes that it owns that state

Actually, the C kernel already has four, one for each core, and they’re easy to lock for a particular thread. Unfortunately, I seem to have broken font printing.

What I’m thinking of is having just one task to “own” the Wimp at a time, the rest being blocked in a PollIdle call, but allowing tasks to remember the rectangles they’re allowed to update and notify them they’ve got to stop and listen to the Wimp (when a task affects the window stack). A separate thread in the task can do the updating of the screen (by non-VDU actions like bitmap copies, or claiming one of the other three contexts). (I want to extract application memory management from the Wimp and make it a service the Wimp uses, rather than one it provides.)

Other tasks that don’t affect the screen at all (the reason for asking this question), I was wondering about the limits to how asynchronous file reads and writes can be. I imagine file read SWIs wait until the data is available, which could be a problem. Does FileCore cache file reads i.e. request the next few sectors of a file to be read without being asked, or is it just writes?

Progress is particularly slow at the mement because I’m on holiday.

I imagine file read SWIs wait until the data is available

Er, yeah… There’s no overlapped I/O presented to the user.

Does FileCore cache file reads i.e. request the next few sectors of a file to be read without being asked

Usually, if it’s very low cost. e.g. in ye olden days, it used to be possible to quickly get the ST506 and floppy controllers to continue reading, transferring another few sectors into the ADFS disc cache in the background, but return to the caller with the requested data transferred to them.

I understand what you’re after now, I must have missed the link to the C Kernel and multicore.

Personally I wouldn’t touch FileCore at this point, I’d leave FileCore blocking all cores and come back to it once the C kernel is stable and feature complete.

You’d need to come up with a solution that implements a lot of the following:

• is non-blocking across cores
• is non-blocking across threads. ie performs background transfers to allow other threads to continue on the core being used for HAL interaction, whilst the HAL is waiting for a physical media response
• allows parallel FileCore use, for example RAMFS can be used in parallel to SDFS
• allows threads to raise background transfer requests, with notification on completion
• supports reentrancy for all Imaging/Filing Systems or handle it in a non-fatal way. ie “FileCore in use” needs to be eliminated
• implements write-back writes/cached reads, independent of the Imaging/Filing System

That could be pretty useful if it could run on, say, Windows (XP!) 1 and allow access to the contents of SD card images.

I’m working on it…you will need a version of Java old enough to run on Windows XP, which will be Java 7 IIRC, but since I had that use case in mind from day 1, my code is compatible with Java 7.

And it took just a few more weeks…productivity ain’t what it used to be.

I now have working low-level read access to USB devices on Windows via the PHYSICALDRIVE mechanism and can also successfully enumerate the devices without invoking wmic.

Unfortunately, even for reading unmounted media (like e.g. the good old Conner 850MB IDE harddrive connected with an USB-IDE adapter which currently makes some noise right beside me), you need administrator rights on Windows 10.

Now I could go on perfecting the tool till eternity – imaging discs, export as ZIP or ISO9660, supporting the various partitioning schemes (only EADFS at the moment), integrate my StrongHelpViewer, work on my Drawfile viewer, integrate Archive reading, browse HostFS mounts, read directly from ADFS-formatted floppy discs, create that RISC OS L&F for Java Swing, perfect that filer-like FS view…or maybe I should just create and release that damned MVP ignoring all those nice-to-haves!

ignoring all those nice-to-haves!

That’s what version bumps are for.
Version 1.xx, does the job
Version 2.xx, dipped in chocolate
Version 3.xx, and a cherry on top

productivity ain’t what it used to be.

I find the biggest impediment to my getting anything done is a combination of the attention span of a gnat and that oh shiny……

Version 2.xx, dipped in chocolate

Hmmm – not always so sure it is chocolate.

@Jon Abbott
Could you provide a link to your HDFS module?

Thanks for the reminder, I nearly forgot to include it in ADFFS 2.81 – I’ll make it public soon.

Chris Johns, I’m really sorry I missed your post from back in June! I’m doing it now.

I copied HForm on to a RAM disc, it just showed a blank white screen when I ran it from the drive I wanted to format.

I started with a blank SD card, planning to copy the RISC OS files from the working one using an SD – to – USB converter.

The Linux machine of course doesn’t notice the newly formatted card, but put_loader works regardless.

claim_frags assumes the loader is called Loader. add_loader completes, apparently OK.

The Pi now boots, the disc reports 29Gbytes, but trying to open Loader just seems to go to the hourglass, needing control-break to get out.

I’ll copy the files from my working system over later on, and see if that helps.

Thanks for writing that. https://github.com/c-jo/adfs-tools.git

Could you provide a link to your HDFS module?

I’ve made it available as a stand-alone Module which you can download here

It should hopefully be ROM/RAM safe, 26/32bit neutral and work on RISC OS 3.1 through to current.