How to do USB audio

There’s been a lot of correspondence on this recently. With my interest in USB and my past career in audio, I can’t help but be interested :-) Colin has made considerable progress in improving USB recently, and Jeffrey has done sterling work porting RISC OS, so we have at least two experts on the RISC OS USB stack.

What do we need to get USB audio going in RISC OS?

In no particular order…

I think it would be helpful if we can create more detailed documentation on the structure and internal operation of the stack. That might make it possible for more people to contribute code.

Output is probably easier than input, and probably useful to more people, too, so it seems to me that’s what we should aim for first.

We need to be able to open an isochronous endpoint for output.

I’m not yet clear on what implications the various synchronisation methods have, if any, on the way the endpoint is opened and maintained.

There needs to be a buffer linked to the endpoint or stream. The OS has to take care of emptying the buffer into the USB stream. We really don’t want to force applications to do that.

The process has to know how many bytes to output per frame. Note that, for 44100 Hz, the number of bytes is not the same for all frames.

Applications do, on the other hand, have to keep the buffer full enough. There have to be hooks of some sort to signal to an application that the buffer needs filling.

Something has to control the USB device. Most commonly, endpoints have some alternate settings, each of which represents a combination of sample rate, word length, and number of channels. One of the alternates (zero) is “off”. I guess this means we have to have an API to permit setting a device appropriately; this also means reading back from the device to enumerate what controls it has and what settings are acceptable to it. Devices typically also have mute and volume controls; without appropriate settings for those values, we could be outputting bytes but hear nothing!

I’ll be interested to hear anyone’s thoughts.

I just received a cheap pair of USB loudspeakers yesterday. I have several USB audio devices now. I’ve written some experimental application code to read their descriptors, and make some elementary settings.

I’m happy to volunteer for some of the tasks above: the documentation (although I will need to talk a lot to some key people to acquire some of their knowledge), and work in the area of the descriptors and API.

China has a lot to answer for, I recently sent for a USB soundcard (microphone in, headphone out) a whole 2 quid so definitely out of Jim Lesurf’s league :-) but the principle is the same. I can do plenty more reading while I’m waiting.

I have started documenting the USB interface as it is now – thats how I discovered that my copy of Techwriter wasn’t working. It’s a useful way of working out what’s missing. I could post a copy for you to look at if you like. I haven’t got that far (I just do a bit at a time) but it may be interesting. I can only post as a Techwriter file or postscript – I haven’t got pdf printing set up and daren’t try in case I get sidetracked on that too :-) Choose your format.

I’m not happy with the DeviceFS API when writing for incoming interrupt and isochronous endpoints you want immediate attention for these and a call and callback system would be much better at that especially as audio quality will depend on keeping the data moving in a metronomic manner. My initial thoughts are that jitter may be a problem. With interrupt endpoints yes you can use DeviceFS but its so cumbersome its like using a sledgehammer to crack a nut. All you want to do usually is read a couple of bytes and act on them immediately. In application space you get to know about them too late.

The current system of selecting alternates is to close the device and reopen it. The OS_SeviceCall will list all connected devices and all of their configuration endpoints.

I managed to print it out (PDFFile.pdf) – on the wrong sized paper. Its not much but its a start.

That’s worthy of going into the PRMs, Colin. I’m most impressed.

Anyone who wants to know how to programme USB should use Colin’s document.

Do you have any notes giving an overview (at least) of what the the code in the various source files does? That’s what interests me. Anyone who wants to contribute code to the RISC OS USB stack needs that information. I’ve made several attempts to work it out, with hardly any success.

If you or Jeffrey have any notes, I’d be glad to turn them into another wiki page on this site.

The only notes I have are already on the wiki.

https://www.riscosopen.org/wiki/documentation/show/RISC%20OS%205%20USB%20stack%20overview

Some things are easier to hold in your head than to write down.

If you are interested in the dirty end where the black art of controller device drivers lurk then it starts in mixed.riscos.sources.HWSupport.NetBSD.build and all revolves around the usbdriver – helpfully called usbmodule – in the c directory. The controller modules EHCI,OHCI etc register themselves with usbmodule any devices that appear/disappear on a controller are registered with usbmodule.

The usbmodule accesses these devices via the NETBSD usbdi interface (mixed.riscos.sources.HWSupport.NetBSD.Dev.USB.c)

This module has direct access to the usbdi interface (usbd.h) which is a virtual usb controller interface and the main NetBSD interface and the one through which all usb calls go.

If you look at device_initialise in usbmodule this is called when an endpoint is opened. If you search for usbd_ in the function you will see the extent of the usb calls – not many

Transfers (xfers) boil down to

usbd_open_pipe – which opens the endpoint.

usbd_setup_xfer – which sets up a usbd_xfer structure for a single transfer.

usbd_transfer – which initiates the transfer using the usbd_xfer structure as its argument – this structure includes a callback for transfer complete (write_cb or read_cb in usbmodule)

usbd_close_pipe

The biggest problem is trying to work out what all the abbreviations in naming functions variables etc mean. I mean things like the structure associated this a usbd_transfer is a usbd_xfer. why not usbd_transfer_t too easy. another on that stumped me for ages was a something_softc. Turns out to mean the software controller structure and is the interface by which udbdi accesses the controllers.

Another thing intially confusing to me were the likes of

STAILQ_HEAD(…)
SIMPLEQ_ENTRY(…)
TAILQ_FOREACH(…)

They are macros for different types of linked lists found queue.h in TcpIpLibs – easier to google for queue.h. Quite useful when you know about them.

Scouring the internet for usb on FREE/NET Bsd was the biggest help if only to work out what structures do.

To understand controller drivers its best to look at something like the OHCI or EHCI driver dev.usb.c.ohci. ohci is a well documented standard and there as specifications for it.

Some help for in there the structures defined in the ohci specs and which are attached to the device are in dev.usb.h.ohcireg

ohci_ed_t (endpoint data type) is in item inserted in a linked list of ohci_ed_t’s when an endpoint is opened. So the controller has a linked list of open pipes attached to it. When a transfer is requested with usbd_transfer a ohci_td_t (transfer data type) is attached to the ohci_ed_t belonging to the transfers pipe. The controller will transfer any ohci_td_t structure attached to a ohci_ep_t structure. When the transfer is complete an the ohci_ep_t is marked complete by the controller an interrupt occurs and and you can remove the ohci_td_t and call the callback function stored in the usbd_xfer structure used by your initial usbd_transfer call.

I’m having difficulty writing in this letterbox now – I’ll have to post it to read it back through. I don’t know if this helps or not there’s mountains of it hehe. If it doesn’t help if you can be a bit more specific.

If you are producing a long tale :-) best to use something like !StrongEd – and then drag the end result into the letterbox.

Where does the words ‘endpoint’ and ‘pipe’ come from – something from the PC world from dealing with the rs232 serial port!

With ref to a usb serial port – to change the status of the pins out of the serial plug – do you poke and peek the ‘control endpoint’?

Do you receive a signal/interupt – to say something has changed at the input of the plug?

Has anyone produced a Serial port driver – so other progs can use the usb/serial adaptor?

Any news on the updated USB stack?

Just to comment:

I can’t help with any of the programming involved here as I’m too dim. But…

1) I can extend or increase my old ‘Bounty’ on this just to express my wish to encourage the work.

2) I’m about to write a column for Hi Fi News which will publicise RO hardware to audio enthusiasts. Really just to draw attention to hardware like the ARMiniX and get some awareness of RO as a possible ‘smaller and lighter’ alternative.

3) I can test developments and report back.

4) At present on the ‘documentation’ front I’m concentrating on writing a clearer and more up-to-date explanation of the ROSS (RO Sound System) and how audio is played. Once that is done I may be able to help with writing other documents if that were any use.

So far as I know, some of the cheaper USB DACs do actually work nicely. So should be fine as a device for test and development as well as sounding good. If people adopt the open method akin to that Linux has added it should work fine with many USB DACs. I assume people can examine the Linux code as an aid here – given they are far better programmers than me! :-)

Jim

Colin.

If you look at the introduction the the pdf I posted earlier it may help.

How to explain it.

Endpoint and pipes are all over USB documentation. pipes are just software pathways for transfering data from here to there – like a hose pipe carries water from the tap to the garden. So when you open a file to save data and get a file handle you are opening a pipe between your program (here) and a file on the hard disc (there). pipe is the generic term for the connection.

If the device is a memory mapped device you access it by peeking and poking registers. You would normally write a driver for the device to give a software interface for other programmers to use. Other programmers don’t have direct access to the hardware and so using the software interface can be thought of as using a pipe between your program and the hardware.

But USB devices don’t have any memory mapped registers – well the controllers do but thats another story – so to control USB devices you have to pipe data to and from endpoints. USBDriver sends the data to the USB controller and the USB controller passes it on to the device. The device recognises the endpoint number and acts upon the data associated with it (data is transmitted to the device in packets which contains the information required for a device to recognise it as belonging to them). The role of the controller is to multiplex the data from several sources across a single usb wire.

All devices have an endpoint 0. This is the control endpoint and is generally used to setup a device or request information from it.

*usbdevinfo 1 and *usbconfinfo 1

shows some information you can read from endpoint 0 of device 1 (you can get the device numbers with *USBDevices)

For the serial port you have to use DeviceFS_CallDevice &80000000,"USBn",... to read and write to the control endpoint.

USBDrivers issue a module service call when a device is plugged in and also sets up some system variables.

Serial port devices I’ve come across tend to be unique unlike ‘mass storage devices’ or ‘audio devices’ for example so you’d need a driver for your specific device.

Serial port devices I’ve come across tend to be unique unlike ‘mass storage devices’ or ‘audio devices’ for example so you’d need a driver for your specific device.

It may help here that many of the USB DACs use the same iso/asynch process which is fairly generic and ‘driverless’ (in windows terms) and can ‘just work’ with a reasonably up-to-date Linux kernel. Lokking at what the Linux people have done may give a pointer to this. (But is waaay over my head!)

Jim

“China has a lot to answer for, I recently sent for a USB soundcard (microphone in, headphone out”

I’ve just bought one of these with the fanciful idea of using it to create a configurable delay on audio with a Raspberry Pi. There is a program written in Perl for Linux which is only a few lines long which will delay audio. I suspect the Linux route may, at the moment, be easier than the RISC OS route? However, I know nothing!

I’ve been decoding the descriptors for the various USB audio devices I have. It’s interesting, albeit I find it very challenging.

I bought a combined mouse and telephone handset adaptor a few months back. I knew it was going to be rubbish, but I just couldn’t help buying such a stupid product, just to laugh at it. It was cheap.

Telephone handset adaptor, ha? That’ll be for VoIP, won’t it, so it will support a sample rate of 8 kHz, won’t it?

No. 48 kHz only, in both directions.

Still, if you have a big x86 processor there, you can afford to do some sample rate conversion. At least the factor is an integer.

Some specific questions about how the stack works.

1. What source files determine whether EHCI or OHCI is in use?
2. Do the EHCI and OHCI drivers act purely as servers, or do they call out to any other bits of code?
3. Am I correct in thinking that EHCI and OHCI drivers are there to present a unified and consistent view of the host adaptor, abstracted from the specific hardware?
4. What source file calculates the schedule for sending isochronous packets?
5. What source file moves bytes from an application’s output buffer to the EHCI/OHCI drivers?
6. What makes EHCI and OHCI appear as a single entity?

1. What source files determine whether EHCI or OHCI is in use?

When a device is plugged in an interrupt occurs – which ehcimodule passes on to dev.c.ehci_intr this decides if it can handle it or not. EHCI is given preference – not sure how yet – but its supposed to and it does – There is a bug however that because both ehci and ohci modules are not present at the same time during rom initialisation (switch on) ohci can pick up ehci devices on an iyonix. This is easily fixed by reversing the EHCI OHCI module order in rom but we just need someone to do it :-) Once both modules are initialised unpluging and reinserting devices picks the correct controller regardless of module order.

2. Do the EHCI and OHCI drivers act purely as servers, or do they call out to any other bits of code?

ohcidriver (aka ohcimodule) and ehcidriver (aka ehcimodule) are just a front end for dev.c.ehci/ohci

The controller drivers are as low as it gets they are the files that access the hardware directly.

3. Am I correct in thinking that EHCI and OHCI drivers are there to present a unified and consistent view of the host adaptor, abstracted from the specific hardware?

For the host adapter to be EHCI or OHCI the hardware has to conform to the EHCI/OHCI specifications at a register level.

4. What source file calculates the schedule for sending isochronous packets?

Thats down to the individual controller and which usbd_pipe_methods it choses for the various endpoint types – see below. for EHCI its dev.usb.c.ehci

5. What source file moves bytes from an application’s output buffer to the EHCI/OHCI drivers?

build.c.usbmodule (creates a modules called USBDriver)

start_read and start_write which call usbd_transfer (in usbdi.h). usbmodule contains bufins(..) and bufrem(..) which are hooks added to riscossify usbdi.c

This module uses calls from usbdi.h which is the NETBsd user interface. usbdi functions are generic and for this to work each controller driver has to supply struct usbd_bus_methods and struct usbd_pipe_methods for devices which are plugged into it. see dev.usb.h.usbdivar for the structures. Its possible for a controller to supply different usbd_pipe_methods for specific enpoint types – which is what ehci/ohci does.

note usbd_bus_methods are methods specific to the controller ie controller = bus.

6. What makes EHCI and OHCI appear as a single entity?

?

The ? means I don’t understand the question

What makes EHCI and OHCI appear as a single entity?

Applications have no idea about EHCI or OHCI, but one of them (i.e. either EHCI or OCHI) drives the USB chip. Something decides which of them to use, which means to me that, above that level, EHCI and OHCI appear as a single entity.

I think this may be a hardware thing there’s something about it in the EHCI specification i seem to remember. It may be decided on chip once both controllers are initialised – not sure though. I haven’t found a software link between OHCI and EHCI.

I’ve also been pondering about another issue that may end up being relevant here. DigitalRender. This bypasses SharedSound anyway and goes direct to SoundDMA. Would perhaps that make a route for a *nix based port of some of the relevant USB audio code? Or is this just a daft idea based on my sheer ignorance of USB?

Jim

Here’s an interesting one :-) IsoOut.zip

Well I’m underwhelmed at the response. The file is the first isochronous recording I’ve made on an iyonix. First on RISC OS ever?

Maybe the response would have been greater if you recorded “This file is the first isochronous recording I’ve made on an iyonix. First on RISC OS ever?” instead of “Mary had a little lamb” :)

It would help if you explained in more detail what you actually did.

I’ve used !AudioIn to read in audio (at 48k) with the Iyonix, and used this as the basis of the !IyoScope application. But at present I have no idea how you produced the wave file inside the zip you linked above.

Jim

It was produced from the microphone on a USB webcam. I’ve had to modify the usb drivers do it. Well it was a Eureka moment for me :-) hence the tribute to Edison

It was produced from the microphone on a USB webcam. I’ve had to modify the usb drivers do it. Well it was a Eureka moment for me :-)

Ah! Yes, that is quite interesting! The next question will be if it should work on other hardware – e.g. ARMiniX – and/or other USB audio devices? :-)

So far as I can tell, at present the only working ‘audio input’ to the ARMiniX is the, erm, CD drive. The audio chip does have other inputs but I don’t think they are currently useable.

If I could get it working here I could also then do some tests on the timing, etc. Although as things stand I have no USB audio input devices at all. That could be ‘rectified’ if I had a sensible reason, though…

Presumably this may also give you a handle on USB output as well?

Jim

The webcam I have uses the OHCIDriver (USB 1 full speed device) via USBDriver to actually talk to the hardware. The recording shows that input works for isochronous transfers and I expect to get output to work as well – input and output share the same code at the controller driver level.

I don’t know if ARMiniX will work – does it use the OHCIDriver module? In any case if I get in and out isochronous working on an Iyonix I’ll know the USBDriver is working and if isochronous doesn’t work on another machine the most likely explanation is a problem with isochronous support, or lack thereof, in that machines USB Controller driver.