Floppy driver board

Ref the floppy driver board that Rick pointed to – is there any chance of RO driver software -like for the USB DOS floppy.

https://retroready.one/products/greaseweazle-v4-usb-floppy-adapter-flux-reader-writer-amiga-atari-pc

[Edit]

That’s a flux reader, which isn’t going to much use without a Floppy Drive Controller. I suppose you could write a 1772 floppy emulator and feed it through that.

Handling flux without an FDC is problematic at best. You need to know the format ahead of time and compensate for the data rate – FDC’s handle all that for you.

Provided they’ve documented the USB endpoints, you could in theory read a floppy in via BASIC, translate the flux to a data stream and then feed that into a 1772 emulator.

You could surely do “an ADFFS thing” and just replace the floppy part of ADFS with something that reads on a sector level from Greaseweazle assuming a “regular” RISC OS floppy format like D or E?

That would be sufficient for accessing old RISC OS floppy disc content, notwithstanding copy-protected game floppies, but then that content is catered for by other things.

I will soon (really!) start prototyping a PC-Java-based solution for USB Greaseweazle access in such a way (assuming plain old D or E floppy disc format), will be interesting to see how easy/hard that will be.

I doubt Greaseweazle will work at sector level, its more likely to be Track based as that doesn’t require knowledge about the format. If its anything like Kryoflux you’ll be able to tell it to read a track in, then you’re on your own.

From looking at the picture above it looks like it has an embedded ARM, so the obvious route would be to implement a 1772 FDC on the Greaseweazle itself and send 1772 commands/data via USB.

Colin: do you actually need to read one or more floppy discs – and do you need to write one/them – or is this just a technical challenge?

I doubt Greaseweazle will work at sector level, its more likely to be Track based as that doesn’t require knowledge about the format. If its anything like Kryoflux you’ll be able to tell it to read a track in, then you’re on your own.

I don’t understand why “track-based” will pose problems. You expect and assume D or E format, so you KNOW which track from which head you need to read to find the right block/sector that Filecore needs.

From looking at the picture above it looks like it has an embedded ARM, so the obvious route would be to implement a 1772 FDC on the Greaseweazle itself and send 1772 commands/data via USB.

It might be the obvious route if you intend to use ADFS-as-is (and add some USB stuff to it), but it also looks like a very complicated and work-intensive route. Much better would be a USB Mass Storage interface which could just read blocks and lets you interrogate the floppy disc format to work out block size and which sector corresponds to which logical block. If you do it correctly, RISC OS USB might be able to directly access ADFS-written floppy discs just like it manages for USB FAT floppy discs.

I don’t understand why “track-based” will pose problems

You won’t be able to seek directly to sectors easily. The host would need to cache the whole track and either pre-convert the flux to raw data so the 1772 emulator can seek within the cached copy, or take up large chunks of memory and cache the flux data which would need to pass through a state machine before hitting the 1772 emulator.

looks like a very complicated and work-intensive route

The codebase would be near identical on either side, the only difference is which side is handling the FDC command translation and flux conversion.

Handling flux data on the RISC OS side should be avoided at all cost as its a real CPU/RAM hog to process.

Handling flux data on the RISC OS side should be avoided at all cost as its a real CPU/RAM hog to process.

Greaseweazle Python code, when extracting data, seems to rip through the floppy disc with the speed of “as fast as the disc drive allows”, so I am not sure if decoding the flux stream of a track on the RISC OS side can possibly be a problem. Of course you will create an in-memory representation of the data decoded so far (the whole 1.6 MiB of it), so the currently transferred-and-to-be-processed flux data will only be the representation of one track at most. In the face of the floppy and the USB-I/O, it would be very surprising if CPU usage on the host would be the problem. And looking at the weak MCU on the Greaseweazle device, why could it possibly help to do the decoding on the other side?

But this is all theory, I will start experimenting with this stuff real soon now (free-time permitting, which seems to be mostly non-existent atm). The code for handling the USB-I/O and flux data is all there in Keir’s GitHub repos (main Greaseweazle with Python code, and disk-utilities with C code for further processing, everything with the most liberal license of ’em all).

Given how slow floppies are, you are effectively reading the whole track while you wait for the index hole to go past anyway. It wouldn’t take a lot to grab the whole track and process it offline, effectively building up a memory cache of the floppy. You could then answer a lot of requests directly from the memory cache rather than waiting for a new index pulse. Writing might have to go directly to the medium, invalidating the cache – I’m not sure what the hardware does about sector write timing, but there must be a way to write a partial track.

You’d need some code to handle flux decoding and sector gaps etc, but I don’t see why you’d need an ADFS and a 1772 emulation – just go straight from DiscOp to flux-over-USB.

You won’t be able to seek directly to sectors easily.

Are we not perhaps overthinking all of this? It worked perfectly well with a 1772 and an 8MHz ARM, and before that a 1770 (IIRC?) and a 2MHz 6502 with barely any spare memory.

If anything, a gigahertz processor and ****loads of memory should make things easier, we have the flexibility to grab the entire disc and parse it in software. Which, okay, won’t be fast, but probably faster than working out how to attach an actual 1772 (if, even, such things exist).

I’m not sure what the hardware does about sector write timing,

I think that’s why there are sector gaps. The disc speed is a stepper motor which should rotate at a specific speed (more or less, that’s where the problems lie), so the FDC needs to simply time how long since the index mark until it reaches the sector of interest, but note that it’ll be reading in the sector ID markers along the way in order to be sure, at which point it starts dumping data to the drive. T Those sector gaps allow for a bit of slop either side so nothing gets overwritten as cheap mass produced floppy drives are not identical.

Indeed, knowing the format and the disc layout, it’s possible to completely ignore the index signal and read the sectors from the embedded markers. That’s what some copy protection systems did, using duff nonstandard markers so that regular disc reading software (like the filesystem) would reject the disc as broken.

effectively building up a memory cache of the floppy

That seems to be what typical USB floppies do (with FAT). If you look at the disc lamp, it chunders through the disc for a few seconds at the start, and then can blink quickly a few times like a harddisc as it serves up what it has read into memory. It only touches the disc again when writing or when reading something not in the cache.

Turns out the Greaseweazle just presents itself as a USB serial device to the OS. Quick, someone knock up a simple BBC BASIC prototype…

Handling flux data on the RISC OS side should be avoided at all cost as its a real CPU/RAM hog to process.

Something of a pain that on a lot of current machines we have spare cores doing sod all and something like this could utilise them, if only…

Are we not perhaps overthinking all of this? It worked perfectly well with a 1772 and an 8MHz ARM, and before that a 1770 (IIRC?) and a 2MHz 6502 with barely any spare memory.

If anything, a gigahertz processor and ****loads of memory should make things easier, we have the flexibility to grab the entire disc and parse it in software. Which, okay, won’t be fast, but probably faster than working out how to attach an actual 1772 (if, even, such things exist).

The trouble here depends on the interface timing. The tricks with doing things on the fly work when you have a one-byte FIFO to your 2MHz FDC and you know how long things take (eg instruction timings). With a USB serial port the latency is more difficult, and jitter may be much worse, so you can’t do these tricks directly from your OS without danger. However there’s an MCU at the other end, so I imagine the timing sensitive stuff happens in there and USB just tells it what to do, in which case the host software is effectively outside the control loop.

so I am not sure if decoding the flux stream of a track on the RISC OS side can possibly be a problem

Once you start working with flux data, you’ll understand why I recommend running the flux through a 1772 emulation layer.

A DD MFM floppy produces over 500,000 flux transitions per second. That is going to be several MB/sec of data to transfer and process. If Greaseweazle is anything like Kryoflux, you’ll get “real” flux and timing values. These need passing through a PLL to handle jitter and clock synchronisation and the MFM clock and data streams need separating out.

Once you have a synced stream, you then need to sync to the first Index mark before you start processing the flux into a byte stream.

A good starting point would be to look at Danielj’s Java source from kryolib – it implements the FDC emulation and FM/MFM decoding, so should be sufficient once translated to C.

The issues you’re likely to come across are the need to read the track several times, so you can average out the flux levels and have enough data to perform retries. You’ll also find you get sync issues, where the bits shift due to clock drift and jitter. Western Digital haven’t documented how their PLL works in the 1772, so syncing the flux stream is a bit of guess work.

So long as you implement the 1772 emulation layer accurately enough, both on the flux side and the data side, you should get a fairly accurate track data stream.

Sometime ago I came across and Open Source product call FluxEngine. According to the documentation it can read almost any type of floppy copying the contents to an image file.

“BMI”.http://cowlark.com/fluxengine/index.html.

The system uses a cheap (£10.00) development FPGA to do all the heavy lifting. If anyone attempts to build one of these items I would be very interested in your views – I have around RISCOS 100 floppies the contents of which I would like to recover.

I’ve not built one, but the PSoC5 is a nice chip for this – it’s an ARM microcontroller with a really tiny amount of FPGA logic, but that’s just enough to wrap some of the awkward timing in hard logic, rather than relying on software. It seems the STM32 in the Greaseweazle has a way to set up the DMA controller to do a similar thing, but the PSoC is more flexible. (my GW has a Chinese STM32 clone, but presumably the DMA works the same). The PSoC can also do some analogue stuff which I don’t think the STM32 can.

So either would work really. The PSoC compiler is Windows-only, though.

I have a KryoFlux and a GreaseWeazle and everything to build a FluxEngine and Slamy’s STM32 Floppy Controller (and I am procrastinating building those two since probably two or three years). All I can say up to now is that both the KryoFlux and the GreaseWeazle are absolutely fine for reading RISC OS floppies, but so is any old PC with a true floppy interface and the help of e.g. OmniFlop – I usually use an old Windows XP laptop for this purpose.

FluxEngine software also now works with GreaseWeazle hardware btw which might be helpful for the more esoteric formats that FluxEngine supports.

When I bought the KryoFlux, there were no other alternatives available, but today, I would never again buy one – the guys are very non-open about their stuff, while GreaseWeazle is a completely open project.

But this is all theory, I will start experimenting with this stuff real soon now

Didn’t get very far, and that won’t change for the forseeable future due to lack of free time. I dusted off the hardware setup (GreaseWeazle, PSU, Sony floppy drive), cloned Keir’s GitHub repo and did a few test runs of the Python code from the IDE to grab SCP flux streams as well as DD/HD DOS floppies and DD/HD ADFS floppies. Only started to look into how the Python code talks via the serial port to the GreaseWeazle hardware, and managed to get some Java code to do serial comms stuff. But I don’t even know yet which baud rate GreaseWeazle supports, just that it presents itself as “USB 1.1/12 MBit/s serial device”.

it presents itself as “USB 1.1/12 MBit/s serial device

That would imply the data might be compressed/reduced before transmission as a DD floppy should produce around 12MBit/sec if single floats were used for the flux level/transition timings.

When I looked the other week, I couldn’t find any documentation for the GreaseWeazle so its probably is a case of reverse engineering their code to figure out the flux data structure.

As you’re using Java, you should be able to pass the flux stream through kryolib and avoid having to code a large chunk of flux interpretation code. If you initially produce an APD style output, you can check it’s validity with either Arculator or ADFFS.

APD is three constant streams of flux data converted to binary (SD/DD/QD) so probably the easiest format for initial testing, kryolib can handle that, and if you need further detail, I documented the APD file structure in !ADFFS.structure; which also contains the 1772 data structure if you need to eyeball the data.

Once you’re confident the flux data can be interpreted, my initial thought would be to merge portions of Arculator into ADFS4, as ADFS4 is C++ and a clean slate so far as floppies go. With some minor changes, the Arculator APD code could be used to request the data directly from the GreaseWeazle instead of from a file.

Although it’s additional work, it might be worth getting this all working natively in Arculator on a PC before attempting to port across to RISC OS as that would only require modifying the source of the bitstream and implementing the flux to bitstream conversion.

Once you’re confident the flux data can be interpreted, my initial thought would be to merge portions of Arculator into ADFS4, as ADFS4 is C++ and a clean slate so far as floppies go. With some minor changes, the Arculator APD code could be used to request the data directly from the GreaseWeazle instead of from a file.

Is ADFS4 GPL? Because Arculator is, and I’m not relicensing it.

RISC OS is modular so GPL is not really relevant.

Um, it kind of is if you’re proposing using GPL code. If the ADFS4 module is not licensed under the GPL then you can’t use GPL code in it.

I suspect what he meant is that as RISC OS uses modules, then it could be possible to have no GPL code in the ADFS4 module (as is the case with the rest of RISC OS), but instead provide a set of hooks that a separate module (loaded from disc) could attach itself to.

This might not be a terrible idea, as older machines (such as the RiscPC with an actual floppy drive) could instead have a hardware floppy driver instead of the flux reader.

Absolutely. It just needs jump tables rather than be embedded in the rom. No reason why it has to be loaded from disc, it could be loaded from romfs or Resourcefs.

A jump table is dynamic linking. That’s still not GPL-compatible. (But is LGPL-compatible.) Can we please stop discussing how to pirate Sarah’s software now?