How to do unpredictable USB input

Ok this is the latest version of my USB drivers. I would appreciate if any one interested would at least read the readme file and let me know if they have any USB programming problems that it doesn’t cover – or any other thoughts for that matter.

Thanks.

Ok this is the latest version of my USB drivers.

I have it on a USB key, I’ll try it on the Pi real soon now. :-)

if they have any USB programming problems that it doesn’t cover

Yes – if your non-blocking works, and the Pi’s evidently doesn’t want to – how do I tell for absolute certain that the USB driver in use is your version of the module?
I would like my MIDI module to not hang on waiting for data that isn’t there, as there is no sensible way to abort from it; which may mean requiring your module be present and in use…

Colin,

A long long time ago, you said:

The only way to get all options to work is to put them in your special field in the same order that they appear in the validation string.

Do you happen to have handy a copy of the validation string so I can mess around with the order I use in my module to see if that’ll make it work?

[if you wonder why I’m asking instead of looking, I’m using the PC right now and I’m not sure what part of the CVS to be looking in…..I really ought to download a tarball – which isn’t so airy-fairy now I am going to have a crack at rolling my own RISC OS (more “for the hell of it” than anything else ☺)]

Line 462. and line 2327 This isn’t necessary with the new modules. You"ll know they are loaded because after running the program *modules will show them at the end – and it’ll be a ram address.

There’s no need to try them – I feel I’m pressurising you now. If its inconvenient no problems. I’m surprised the pi is blocking. Really must get one. I’d be interested to see your code. You can check noblock in basic with my modules with a simple OPENOUT followed by an OS_GBPB from your device if noblock works r3 returned from OS_GBPB will be unchanged. You should also use nopad.

Line 462. and line 2327

Ah. nopad is in one place, noblock is in the other. As I am passing one string with both, is there a specific order they should be in?

You"ll know they are loaded because after running the program *modules will show them at the end – and it’ll be a ram address.

Fair enough – I ought to be able to get that from OS_Module.

There’s no need to try them – I feel I’m pressurising you now.

I would have tried them ages ago if my memory was any good… Ultimately we both want the same thing – a reliable easy to use USB system. So if this helps to that end, then you should pressure me. Come over here and stick post-it notes to everything – something like “Don’t forget Colin’s USB, dumbass!”. ;-)

I’m surprised the pi is blocking.

So was I. There may be an error in my programming (though I do call OS_Args 9 as well!); however sadly this never showed up before for two reasons:

1. The previous program was a single-tasking BASIC program
2. I was connected to my Yamaha keyboard, which – when it is on – is always spewing out a load of data (MIDI clock ticks and ActiveSense messages).

What brought this to my attention was the USB-MIDI dongle. I didn’t need an actual keyboard to be hooked up as when the dongle is plugged in, that in itself is enough for it to be recognised…
…but there’s no data flowing.
Oops.

What’s worth pointing out is that if you recall my earlier messages in this respect¹, the Pi and the Beagle are markedly different in behaviour. Remember one will happily return with a GBPB request of 4 bytes, but the other one kept failing with a data read error until you suggested I try a block of 256 bytes.

Really must get one.

It is… strange… to think that (if we ignore the Beagle) the scant few square inches of the Pi is more memory and oomph than all of my previous RISC OS machines added together. In fact, it’s my fourth most powerful “computer” machine (eeePC=~1.6GHz/2Gb; NASbox=1.1GHz/1Gb; BeaglexM=1GHz/512Mb; Pi=~700MHz/256Mb). It is a little further down the list if we include Android phones, but they don’t count until they can run RISC OS. ;-)

It is also priced such that you can set yourself up with something for a fairly minimal outlay. Most of the stuff attached to mine is recycled from elsewhere.

Oh, and possibly useful for development work – when you get your Pi you’ll see a two pin header next to the HDMI connector. I think it is labelled P6 but don’t quote me on it (it’s in the other room). Shorting the pins with a screwdriver, breakfast spoon, glasses frame, soft drink can, or anything else metal you have handy² – will invoke a reset. For those times when you do silly stuff in module code³ and you’ve killed RISC OS enough that Ctrl-Break just isn’t going to cut it.

I’d be interested to see your code.

It is currently on the Pi’s SD card. I want to play around with it some more; see if changing the order of things alters the behaviour at all?

You can check noblock in basic with my modules with a simple OPENOUT followed by an OS_GBPB from your device if noblock works r3 returned from OS_GBPB will be unchanged. You should also use nopad.

Run it in a TaskWindow, see if the machine grinds to a halt awaiting data. ;-)

¹ See? Useless stuff like that, I remember. I can tell you the printf() format specifiers for floating point numbers yet I rarely even use such – but can I tell you what I had for breakfast? … Did I even have breakfast today? ⁴

² Uhhh… Get the feeling I’ve done this a few times?

³ What d’you mean you have to stack R14 in SVC mode before you call a SWI!? 8-)

⁴ Thinking about that makes me hungry. I’m going to go have a “Clemenville” (some sort of knock-off clementine) and then go offline for a while. It’s 02h25 here, any more’o this and I’ll be a big walking Data Abort!

The noblock string goes first (that string applies to all devicefs devices).

What’s worth pointing out is that if you recall my earlier messages in this respect1, the Pi and the Beagle are markedly different in behaviour. Remember one will happily return with a GBPB request of 4 bytes, but the other one kept failing with a data read error until you suggested I try a block of 256 bytes.

My update should fix this, you can fetch any size you like.

The only difference between the various machines is the usb controller driver back ends. Transfers are requested from them using a function call and callback system – blocking is done at the devicefs level. I havent looked at the pi controller driver but It seems inconceivable that the pi backend blocks.

The first problem to overcome with my update is whether the softload scheme will work on the pi. I don’t know if the sd card interface uses devicefs – which gets changed in the update. I’d run the update from RAM disk and remove your sd card first. I’d then check if a spare sd card was working ok – don’t want to lose your programme :-).

Regarding the pi I’d only want one to test that I haven’t broken something, it hasn’t reached iyonix replacement level for my purposes.

Edit. I should add that the update has debugging enabled – see readme file.

Is it possible to have a flag at the beginning of the module to indicate debug enabled? So that you could change it with something like !Zap – from debug/non debug.

There may be a way with the USBDebug command (help usb.) I’ve never used it so can’t help further I’m afraid. This is only available in the debug version as the the debug instructions are not there in the non debug version of the module. The debug data goes to memory so you don’t really notice it. Debugging is only enabled for the usbdriver module.

On thing I’ve discovered this morning is that the debug version, despite outputting debug data, copies jpegs from a memory stick to an iyonix HDD or RAM Disc 3 times faster than the non debug version (or the version in the 5.18 rom)

There’s an interesting discovery hmm…

Anyone care to duplicate that finding?

Test code

T%=TIME
FOR A%=1 TO 10
   OSCLI "copy sourcefile destinationfile ~CF"
NEXT
PRINT TIME-T%

There’s an interesting discovery hmm…

Turns out that the memory stick is attached to the OHCI device on startup but after softloading the new modules, and all the devices are rediscovered, it is attached to the EHCI Device.

Not so interesting after all.

Rick:

It is currently on the Pi’s SD card

And your backup copies are where?

Someone once said to me until you have your data or program in three places you don’t have it!
As I’ve more than once overwritten a backup with a new version when I should have kept incremental copies, three places sounds good to me.

And your backup copies arewhere?

Of the code for the version I released? There is no backup. Got dark, got tired, threw together the release archive. Didn’t bother with the source.

Of the previous version? The USB key, the emulator, and the SD card inside Azumi¹. A version-before is on the Beagle, but I’ve done most of the development on the Pi as it’ll run off Azumi’s USB port, the Beagle is somewhat more power hungry.

¹ The eeePC.

Ok this is the latest version of my USB drivers.
I have it on a USB key, I’ll try it on the Pi real soon now. :-)

Nothing to report as yet. Ran the modules on the Pi. A taskwindow popped open, then the machine froze for a few seconds, then everything continued as before.

I have not experimented with the additional facilities because I’m hoping the MIDI module will work on the widest range of hardware (and thank you – putting the parameters in the right order, it works now!).
Suffice to say, on the Pi, it’s a thumbs up. I’ll try it on the Beagle (xM) when I next power it up, unless somebody else here beats me to it?

Okay, I’m off to watch ep 5 of Nobuta wo Produce. :-)

Good – first hurdle overcome (I haven’t stiffed your machine :-) )

I’m hoping the MIDI module will work on the widest range of hardware

The special field entry order fix was included from RO 5.19 and will affect all machines.

At this stage I’m not interested in someone using them to program, I’d like to know if something stops working.

In order to maybe give someone else more confidence to try the new modules – I’m using them compiled into the ROM on an iyonix – did you try:

1 reading/writing the sdcard
2 reading/writing to a memory stick
3 accessing the internet

I’d also like to know if anyone is using the FTDI device driver whether it caused problems with that.

I’ve tried to ensure that existing drivers aren’t affected but we all know the difference between theory and practice.

I’m hoping the MIDI module will work on the widest range of hardware

Okay. For now, just assume the MIDI module does not work on a Beagle. On my (older) firmware, it blocks. I don’t plan to do any development work on the Beagle at this time as I spent an hour trying to find DVI settings that the HDMI→VGA adaptor would be happy with. I did, albeit the Beagle thinking it was outputting 1280×1024 and RISC OS thinking it was in 1024×768 (both at 60Hz). But, then, I rebooted and all was lost. Back to looking at a black screen with the exact same settings. Gah! I don’t have time for that…

The special field entry order fix was included from RO 5.19 and will affect all machines.

Because ROOL’s version numbering is the polar opposite to Firefox, “5.19” covers a hell of a lot of territory. ;-)

I’d like to know if something stops working.

The experience was a lot less positive on the Beagle. As there is no special loader SD card thingy for it, the entire ADFS filesystem is on a USB stick. I dismounted that to run your modules from RAMdisc. It kept complaining about “Harddisc0” not being found – I’m guessing the OS is trying to evaluate Run$Path?
So I went and Obey’d the file from the command line. Losts of stuff on-screen, and the re-init took an insanely long time to do (I didn’t time it, but I’d guess 5-8 seconds).
Back in the desktop, things seemed okay.
Until I plugged the USB key back in.

Something crashed.
No drive icon.
Trying to look at the drive reported “Drive empty”.

So it would appear that your next challenge is how to get this to work around the root filesystem being on USB. <cough> Good luck!

In order to maybe give someone else more confidence to try the new modules – I’m using them compiled into the ROM on an iyonix – did you try:

1 reading/writing the sdcard
2 reading/writing to a memory stick
3 accessing the internet

I left the SD card in the whole time. It communicates directly with the SoC doesn’t it? I don’t imagine that would be affected unless the USB driver kills the entire filesystem, but I figure you might have noticed that lurking gotcha! ;-)

I copied a build of the MIDI module to a FAT32 stick.

I didn’t access the internet. Apart from one test (for the blog entry on setting up static IP networking on the Pi), I have not been on-line with the Pi at all. I’m hoping one day a WiFi driver might appear from the ether.

However, MIDI module compiled and tested both with and without your module loaded.

Would I have confidence to use your module? Yes and no.

No – if my root filesystem was on a USB stick. I don’t imagine the problem is your module specifically, I just think removing and reinserting the main filesystem device really annoys the hell out of RISC OS. ;-)

Yes – if the root filesystem is on an SD card or the like. I ran/rebooted/tested your module a number of times on the Pi yesterday and from a behavioural perspective, it appeared to operate identically whether your module was present or not; no unusual behaviour observed, peripherals kept on working, etc etc.

I’ve tried to ensure that existing drivers aren’t affected but we all know the difference between theory and practice.

Yeah.

We know.

<looks sternly at the Beagle…>

I would suggest if somebody else wants to try Colin’s USB driver on a Beagle (or machine with USB filesystem), you might be able to do it by hacking the early boot sequence.
I won’t be trying this due to the gross vagarities in getting the HDMI→VGA adaptor to output anything on the Beagle¹; however the process I would consider is this:

• Check for Boot$Begun
• If NOT set:
• Create a RAMdisc
• Copy USB driver (and script) to it
• Obey the startup script on RAMdisc
• The RAMdisc script will:
• Dismount the boot device (Obey module should now be looking at RAMdisc, right? or can you nest Obey files?)
• Kill the bits and pieces
• Load the module
• Reinit the bits and pieces
• Set Boot$Begun
• Call $.!Boot to kick off the boot process
• As !Boot will now have Boot$Begun defined, boot will continue as normal, though hopefully this time with the new USB driver.

I’ll await Jeffrey or nemo saying “Noooo! You can’t do that because….”. ;-)

¹ It seems that if you tell the Pi to use a specific mode on the digital output, it will use exactly that and fit everything else around it. The Beagle, however, appears to override what was specified in the “dvimode” setting to output whatever was requested directly – much of which the HDMI→USB adaptor doesn’t appear to like. It doesn’t help working blind…

There is a program that takes a group of Modules and makes them into one FF8 Application.

Something like this might work with the softloaded USB modules.

Or indeed this might work loading the newer ‘Filecore’ module on an older OS.
Might have to make a simple module to give a few *commands and then quit.

I’ll await Jeffrey or nemo saying “Noooo! You can’t do that because….”. ;-)

Nope, that’s probably pretty much the only way of doing it. Unless you switch to a Pi-style dual partition SD card or something. Or maybe set to boot from SD, with a simple !Boot script which just loads the new USB modules and then starts the boot sequence that’s on USB? RISC OS does now support booting from DOSFS, so that’s probably an easier approach than modifying your main boot sequence or going for a dual partition setup.

Also note that you’ll probably need some code in there to wait for the USB drive to be mounted again – e.g. by looking for “drive empty” errors when it tries to restart the boot sequence.

It seems that if you tell the Pi to use a specific mode on the digital output, it will use exactly that and fit everything else around it. The Beagle, however, appears to override what was specified in the “dvimode” setting to output whatever was requested directly – much of which the HDMI→USB adaptor doesn’t appear to like. It doesn’t help working blind…

Which dvimode setting? We don’t pay attention to any Linux kernel command line arguments. Or any u-boot environment variables. So yeah, you’ll be getting whatever’s in your MDF.

Unless you switch to a Pi-style dual partition SD card or something.

Any possibility of the dual format formatter being available? It would seem as if the Beagle is fussier about file locations, which could preclude tweaking a Pi image?

Which dvimode setting?

Not set as a kernel option; just specifying dvimode “1280×1024MR-32@60” (I’m in the car so that might not be entirely correct) as an option to u-boot. Hmm, I’ll need to fart around with the MDF/config…

Rick.

Shame about problems with the beagle. I knew one of the machines was usb everything. Are all the various beagle boards all usb?

I would have liked to consolidate the changes in the usbdriver into RISC OS (assuming they want them) before I go any further. The changes in the other modules are already in the latest RISC OS. It tidies up the existing interface which has to stay to maintain backward compatability.

The interface is OK for control, bulk and interrupt end points, the only thing that is missing is a transfer_complete callback for use inside modules. You do have Upcall_BufferFilling and UpCall_BufferEmptying but its not quite the same. I was toying with adding that at this stage something like USBDriver_RegisterCallBackWithStream but decided that I need to to do some experimentation and that I’d rather do that after I know the existing changes are accepted.

Your problem with your program working ie not blocking on the pi but blocking on the beagle with the existing usbdriver has to be down to the controller driver for the beagle. So I don’t think you can fix it by modifying your program.

It may be easier to create a new rom and softload. Or does a beagle already softload in which case can you softload on a softload :-) Anyway if the softload has to be at boot time it may be an idea to ask if you want to do it.

Shame about problems with the beagle. I knew one of the machines was usb everything. Are all the various beagle boards all usb?

BeagleBoard and PandaBoard (and all derivatives) are no more USB than a Pi is. Only difference is that the Pi has readily-available dual partition SD card images, while with the BeagleBoard/etc. you either have to stick to USB booting or try and roll your own.

Or does a beagle already softload in which case can you softload on a softload :-)

The ROM is held on the SD card, same as with the Pi.

I’ll just quickly inject the fact that we (R-Comp) now offer the “magic” dual partition SD cards for Beagleboard as well as Panda and Pi, as part of our Support Schemes. I have spent today burning special 5.20 micro SDs for ARMini users (Beagle), in 16 and 32gb sizes. We offer 32 and 64 for Panda, and 16 for Pi, although we can do 32Gb or 64Gb for Pi if people wish.

Because of the changes in 5.20, we’re actively trying to move Beagle users away from USB booting, and onto SD cards, in line with our Pi and Panda offerings. That way things become more unified, and we can further standardise our updating utilities to make it even easier for people to upgrade their systems.

you either have to stick to USB booting or try and roll your own.

Given the level of wizardry necessary to roll your own, you can see why I’ve stuck with USB booting. All I need to do is pull the RISC OS image off the uSD card, then *CONFIGURE the filesystem and drive to point the OS to the USB stick. Simple. Of course, it does mean there’s about 4Gb of SD space doing nothing (well, it is holding a perpetually broken¹ version of Angstrom but from a RISC OS perspective, that’s “doing nothing”).

Colin: it would be quite possible to supply a ROM image with your module. All you need to do is build the entire ROM… for an OMAP, mind you. ;-) Probably better to ask somebody with one of the R-Comp dohickeys, at least they won’t be depending upon USB!

¹ I define “broken” as all attempts to configure the thing replying “The configuration could not be loaded; An unknown error occurred.” [second picture down, here] Note that it also does this with a freshly downloaded image burned onto a new uSD card. The only thing my Beagle lacks is the clock battery, but surely the settings point to various files, not some sort of NVRAM?

it would be quite possible to supply a ROM image with your module. All you need to do is build the entire ROM… for an OMAP, mind you. ;-)

Compared to working your way through the USB modules building an omap rom is a doddle.

It would be handy for the RO 5.21 series of ROM’s to have the various Debug modules installed.

It it possible to connect serial mice to the new machines like the Pi/Beagle etc?

It should make replacing various USB modules easier.
This worked with my Iyonix – plus using a software keyboard.