USB changed in an incompatible way

Since one or two weeks my FTDI driver is no longer working properly wiht actual RISC OS 5.19 USB Driver version.

I assume that someone changed the behaviour of DeviceFS_CallDevice 1<<31 + 6 Transfer Info in an incompatible way so that my program is no longer able to detect running requests or not.

It is really frustrating that the own programs are only working some months or weeks and must be afterwards rewritten due to incompatible changes.

By the way what are CoSprites? I am only aware that they are missing since the same time. Never heared about them before.

By the way what are CoSprites? I am only aware that they are missing since the same time. Never heared about them before

CoSprites are the icons used by !Configure. The error message is appearing due to a small bug, if you update ! Boot.RO500Hook.Res.!Configure from the disc image the message should go away.

Thanks for clarification regarding CoSprites.

A couple of weeks ago I fixed a bug in the USB drivers where it was possible for a circular reference to be created in the devices transfer queue if the RX wakeup/TX wakeup device calls were made at the right time. E.g. if a USB transfer is in the process of completing, and you call the RX wakeup device call from within the buffer filling upcall, then USBDriver would examine the transfer, see that its status is set to “complete”, and add the transfer back into the queue. But at that point the NetBSD layer isn’t finished with the transfer (e.g. it hasn’t removed the transfer from the queue, or called the transfer complete callback), and so the same transfer gets inserted twice. I only found out about it this bug because I was getting infinite loops inside usbd_dump_queue() when debugging is enabled. I think it may have also been responsible for some of the instability of the DWC driver on the Pi.

The way I fixed the bug was to add a couple of flags to the devstream struct that the DeviceFS code uses: One flag for whether the transfer is busy, and one flag for whether RX wakeup was called while the transfer was busy. This allows the driver to automatically start a new RX transfer once the current one completes, mimicing the behaviour of the old code. For writes, no such flag is necessary, as the TX transfer is always kept active as long as there weren’t any errors.

Looking at the code now, it looks like there are two ways the transfer status call can be fixed:

1. Make get_transfer_info return a status of 0 if str→xfer_busy is true. This is the “correct” way of fixing it, but will mean that the transfer will never be reported as complete from within the buffer filling upcall, since that upcall is made before the NetBSD driver calls the transfer completion callback within the DeviceFS code.
2. Make get_transfer_info return a status of 0 if str→xfer→status is USBD_NORMAL_COMPLETION and str→delayed_read is true. This should mean that it will now report the transfer as being complete from within the buffer filling upcall, and then report it as in progress if you go on to request a new read via RX wakeup. Note that you can’t simply check str→delayed_read, as that flag isn’t cleared after an error.

Alternatively, we could allow programs to register a callback function with each DeviceFS stream, so that the programs can receive proper notifications for when transfers are complete instead of relying on buffer upcalls and guesswork.

Any strange things the same reason?
1. My xM is wayting any secons until the HD is ready to boot from. Boot from stick or use a old ROM it is not so.
2. Any USBtoSATA or USBtoIDE adapter not work with the newer ROMs.
3. Pandora not boot from stick. With ROM from 26.06. it works. The new ROMs from jul are not enable the USB Host by start riscos after power on. If I start Linux and enable the USB Host and restart the machine without power off and got to riscos, it works.

For me that with the buffer upcalls is ok. Of course there might be callbacks useful but that is “nice to have” and has the disadvantage that it is not working with simple Tasks but only with a piece of module code.

For my FTDI code it would be better that a transfer is ready or aborted in the moment at which the last Byte is transmitted. Sounds more like possibility 2. If choosing possibility 1 I see the danger that all Bytes are received but you have to check this yourself as a special state. Because no further data will arrive you can’t rely on any extern action triggering the code. May occur seldom and is difficult to debug. If implementing this in that way at least a remark inside the documentation would be required.

So I personally would prefer way 2.
However I don’t know whether I am the only one which uses this SWI in the moment.

With the today ROM the Pandore boot from Stick without activate the USB Host by using Linux.
Power on – right sholder button – boot from SD – B or X – all things fine ;-)
Many thanks.

On the one hand, I’m happy that it’s working. On the other hand, I haven’t changed anything yet!

Unless anyone has any more thoughts, I’ll go with option 2 for fixing get_transfer_info.

The get_transfer_info fix is now checked in.

I just checked the lastest ROM version from yesterday.

Instead getting the right data in loop back from FTDI but with unwanted random transmission I am now getting simple data at all.

Seems that I have to rewrite the driver entirely for the third time in the hope that it may work one or two weeks until the next incompatible USB change is done.

I am meanwhile really disappointed about the incompatibility.

Will your driver work with almost any FTDI device? I can buy one and find out what the problem is.

In theory it shall do with serial devices, not BitBang/Parallel mode. I only checked it with my OBD devices. 9600 Baud as default would be fine.

To ensure that a device is working it would be better to use ROM some weeks ago before the changes as I am still doing and afterwards switching to the new ROM.

However overall the basic problem stays. I was glad to see that some weeks after I made the driver working again a new ROM release was launched for the Iyonix. So it should have been possible to use the driver on both machines (I didn’t find time to upgrade my Iyonix until now).

If now changing the driver it is likely that it will again only work on the BB xM. Iyonix users will have to wait another year.

Taking into accounting that the Castle stack is from around 2001 is it really disappointing that it is not a stable base for software development. Every change seems to have unpredictable side effects even the API looks ok since a while. May be it is due to the BSD/LINUX base.

@Jeffrey
The latest ROM starts with USB power off. May be it is a side effect from other changes e.g. SDFS. I don’t no how I can switch USB power on under RISC OS. I have send a question to notaz. To start RISC OS there is no problem. I change the card in the left slot after RISC OS ROM is loading but I can’t use other USB devices (UMTS Stick, Cardreader…)

Taking into accounting that the Castle stack is from around 2001 is it really disappointing that it is not a stable base for software development. Every change seems to have unpredictable side effects even the API looks ok since a while. May be it is due to the BSD/LINUX base.

The USB drivers are quite complex, and there doesn’t seem to be any documentation on how everything is meant to work, and under what situations it’s safe to make certain function calls. It also doesn’t help that the NetBSD code was designed to be run on an OS that supports threading, and so has had to be adapted to run under RISC OS. Plus we’re trying to use a stream-based interface (DeviceFS) to interact with something which is very much packet-based (USB).

I did make a couple more tweaks to the code last night, which might fix things if you’re lucky, but it doesn’t look like the autobuilder has built any new ROMs yet. I’m going to buy an FTDI serial device, so if the fix I checked in last night doesn’t work then it shouldn’t be too long until I can find what’s causing the problems.

Raik: I believe there are under 12 people between me and the front of the Pandora upgrade queue. Once it arrives I should be able to fix all the issues you’re having pretty quickly. I just don’t want to debug things remotely, as it’s a very time consuming and inefficient way of doing things.

I’m surprised but I fear I’m the only one at this time using the Pandora with RISC OS. I know you can do all the things step by step and the Pandora is not the important thing. It should be only a information. I’m very thankful to all the involved people.
The problem isn’t a stable, I note today. I do the same if I start the Pandora but I can not say if it start with or without powered USB host :-( I hope I become a answer from Notaz and I understand this. I will make a tool to enable/disable the USB host under RISC OS. If I boot via SDFS I don’t need the USB host.
Many thanks.

May be the Pandora (?) u-boot command usbinit fix the problem. I insert this in the boot.scr (via !SDCreate and !MKImage) and the USB host is powered.

it doesn’t look like the autobuilder has built any new ROMs yet

My fault, I forgot to restart our cloud server yesterday. The autobuilds should happen as normal tonight (tomorrow morning).

Can you double check that server please Steve?

Can you double check that server please Steve?

I have and it all looks fine.

Note: the upload (for a given build) only happens if the sources for that build have changed since the last time it was built. And the uploads don’t happen until some time between 7am and 8am UK time (currently UTC + 1). This is because the process doesn’t start until around 2:30am to give all of the midnight oil workers time to commit stuff.

Thanks Steve, I must have posted my previous message before the uploads had started.

Sorry for answering so late but I was involved in wallpapering.

Whatever Jeffrey did, it bring back the old behaviour. The FTDI device now reports permanently 1 Byte even there was no transmission. But it also reports typed characters correctly.

wallpapering

Have you found a suitable theme manager for your walls, or was it a manual upgrade? (Sorry, couldn’t resist that.)

The FTDI device now reports permanently 1 Byte even there was no transmission.

Now that I’m not preoccupied with getting things ready for the Raspberry Pi release I’ve been able to take a look at this. By introducing the delayed_read flag to the USBDriver code I’d introduced a bug where one filesystem-level read would result in two USB-level transfers being performed – one read triggered by the ‘RX wakeup’ DeviceFS call that’s made on every FS-level read, and a second read triggered by the ‘resume’ DeviceFS call that happens once the devices buffer becomes empty. I’ve now added some extra logic to make sure that the RX wakeup & resume calls are handled properly: ‘RX wakeup’ calls only start new transfers, and ‘resume’ calls only resumes existing ones. With these fixes the FTDI driver now reports the correct amount of receive data. The fixes should show up in tomorrow’s ROMs as USBDriver 0.62.

If you read this… yes. !ComCentre works with my UMTSStick stable with this fix.

Thank you.

It seems that Jeffreys last fix cured the problem.
FTDI seems to work (as far as I can cehck without car). A new problem occurred at COMCentre however the stack behaviour changed yes but not in an incorrect way. The COMCentre problem with the new stack was on my side as I didn’t handle all transmissions possibilities properly inside a BASIC library. Due to the new stack behaviour this weakness became a problem. But as mentioned this was my fault.