How to do USB audio

I disagree with your comparison of built in and USB audio devices. The only reason the built in devices present a ‘simple’ interface is that they have all been squeezed into the RISC OS API. They are no more or less complicated than a usb device. There is nothing stopping USB devices being squeezed into the same interface – or for that matter built in devices appearing as a USB device as a way of making all features available. Presenting them as usbdevices does have the advantage of only having to deal with usb devices for all audio.

All audio devices will have a control interface and a streaming interface. Under riscos this is mirrored with SoundDMA and SoundCtrl – although as I understand it SoundDMA does some processing so isn’t low level enough.
For USB think SoundCtrl = Audio Control Class and SoundDMA = Audio Streaming Class.

So I think you should be writing an Audio module not a USB Audio module and hiding all the usb stuff in it.

USB shouldn’t use the DeviceFS interface we need to be able to daisychain buffer filling routines – each adding different facilities as required – all driven by a single clock and want them to be as lightwight as possible. What is required from an audio streaming module is similar to SoundDMA now – I say similar because I think SoundDMA is too restrictive.

So if you want SoundDMA to be more flexible than it is now we need an API more basic than SoundDMA

Designing any API is a trade off of convenience v flexibility.

If all you want is USBAudio to work with RISCOS applications then you have to squeeze the USB device into the RISC OS api and forget about what other things the device can do. Thats what happens now with built in devices. If you want a high quality audio system then you have to write a new audio system that doesn’t fix data formats/rates between source and sink.

So if you want SoundDMA to be more flexible than it is now we need an API more basic than SoundDMA

That seems sensible to me. Even ignoring USB as such, at some future point SoundDMA is almost certain to need changing/updating/expanding/replacing anyway to cover things like 24 bit audio and more than two channels. (And of course, formats like ‘5.1’ have one channel at a different sample rate to the others as well!

So in the end I suspect the existing SoundDMA/SoundControl may end up looking like a basic ‘internal fixed hardware subset’ of something larger that can also cover user-pluggable USB devices.

In the short term, though, I presume something more of a ‘direct bypass’ is needed to have audio working via USB. But it looks sensible if the API allows for the future development.

All cheerfully said on the basis that I have no real clue about the USB. Hence I formally reserve my right to be wrong. :-)

Jim

They are no more or less complicated than a usb device.

In a way you’re right, of course, except that USB introduces the complications of devices coming and going – you can no longer rely on a known number and type of interfaces being built into a machine, so you can forget the HAL. And you don’t know in advance how to control them either.

So I think you should be writing an Audio module not a USB Audio module and hiding all the usb stuff in it.

I agree that we should have an Audio module of some kind, but I think that a USBAudio module is a useful layer that any Audio module should call on. It’s a bad thing in general to put lots of stuff into a single layer. The result can easily become incomprehensible.

USB shouldn’t use the DeviceFS interface we need to be able to daisychain buffer filling routines – each adding different facilities as required

If there is a better way than the DeviceFS way, I’m all for it.

They are no more or less complicated than a usb device.

In a way you’re right, of course, except that USB introduces the complications of devices coming and going – you can no longer rely on a known number and type of interfaces being built into a machine, so you can forget the HAL. And you don’t know in advance how to control them either.

This is already partially handled by SoundDMA. On the Pi the sound hardware driver is provided by the BCMSound module. In order to allow the module to be killed safely I updated SoundDMA to add support for detecting when the HAL audio device is being removed from the system, and to put the sound system into a safe idle state (audio output will stop, and once a new sound device comes along SoundDMA will automatically try to re-enable sound at whatever sample rate & buffer size is closest to the old settings). The only things it won’t deal with at the moment are the audio device being missing on startup (the module will refuse to start), and situations where there are multiple devices available (there’s no way to specify which device gets used, and if the active device vanishes, it’ll still go into the idle state and sit there until something new comes along).

The complications of the permanence of USB is a secondary problem that doesn’t affect the overall solution.

Problem 1 is having an audio device tell you what it is capable of. For USB thats easy read the descriptors. For built in devices there is no mechanism for this. I’m using USB descriptors for this discussion because its a fully described interface and not limited to USB. So in an interface based on USB. Each machine if it wanted to use the internal sound hardware would write a module converting it to a usb device. Note this doesn’t mean the data is passed through the USB controller it just means that you use the USB API to control/use it. Once one such module was written it could be used as a template for other machines. You can then find out what it does by reading the configuration descriptors. It would work exactly as if it was a USB device plugged into the machine.

Problem 2. You now have the ability to enumerate all devices and their capabilities and an interface to use them. I envisage the user being presented with a source sink split ie a list of ‘microphones’ and a list of ‘speakers’. You need to be able to connect any ‘microphone’ to any ‘speaker’. Note disc drives act as either ‘microphone’ (when they source data) or ‘speaker’ (when they store data)

Problem 3 ‘microphones’ and ‘speakers’ may not speak the same language so we need a universal translator module.

Problem 4 getting the sound from multiple ‘microphones’ mix to 1 ‘speaker’

Problem 5 broadcasting 1 ‘microphone’ to multiple ‘speakers’.

So in the end you can end up with being able to do

mic - speaker

or

mic1 -           \                    / translate - speaker 1
mic2 - translate  - mix - broadcast -   speaker 2
mic3 - translate /                    \ speaker 3

where mic1 and speaker3 could be a hard disc, speaker 1 headphones, speaker2 speakers mic2 an actual microphone mic 3 an internet stream.

That is just the streaming interface in top of that there is the control interface where every ‘microphone’ or ‘speaker’ can be adjusted for volume.

With the above system the RISC OS would claim a speaker add a mixer and all programs wanting to use the ‘system speaker’ would do so via the mixer. Similarly a system ‘microphone’ could be accessed via a broadcast unit.

So thats the pie in the sky out of the way. That I think is my ideal system.

On top of all of that there is a problem with USB in that you can’t identify uniquely an audio device. You need to be able to identify a device to store configuration settings. The only solutions I see is either assume no-one will use identical devices or use where the device is plugged in to distinguish identical devices which would fix those devices to certain ports. Removing the device isn’t a problem you just make it work like a null ‘speaker’ in the same way that a null: file sinks data.

When I see “Problem 1” and the idea of a generic audio devices API module based on the USB API and descriptors, how much of it is tied to audio? So I say drop the “audio”. I think you need an API somewhat like:

• device_id = Register(card_id, slot_id, descriptor list, interface_list) where
  • device_id is an internal device number for use the the rest of the API, maybe an ever increasing number to ensure that no one reuses unregistered devices.
  • card_id defines the card (internal USB, USB in PCI slot x, internal IDE, …),
  • slot_id identifies the physical connection (dunno how it translates under USB),
  • descriptor list, an USB style descriptor list
  • api_list a list of pairs (type, pointer to a list of functions or maybe a SWI number) where type identifies the a RISC OS standardized API/version to access this kind of device. There would be standard APIs for reading/controlling settings of the device as well specialized APIs for reading/writing data for harddiscs, CD/DVD, audio interfaces, generic devices.

• Unregister(device_id) to unplug a device

• device_id list = List(search parameters) to retrieve a list of devices matching certain criteria

• descriptor_list = getDescriptor(device_id)

• api = findAPI(device_id, type) to retrieve a given type of API from the device.

• Subscribe(event, device_id or 0, notification_interface) so that modules can register their interest in the plugging or unplugging of devices.

When I see “Problem 1” and the idea of a generic audio devices API module based on the USB API and descriptors, how much of it is tied to audio? So I say drop the “audio”. I think you need an API somewhat like:

The HAL device API is almost exactly what you’ve described. The docs could do with a bit of a tidy up, but each device is described by a standard header giving the device type, API version, “location”, etc. Then the header is followed by any device-specific stuff (function pointers for API calls, etc.). OS_Hardware is used to add, remove and enumerate devices, and Service_Hardware is used to notify of device addition/removal. It’s also possible for the HAL to add devices.

So you could potentially make it so that the USBAudio module identifies any USB audio devices that are attached to the system and then creates HAL devices for them in order to allow other systems to detect and interact with them.

There are already HAL device APIs defined for audio, which is what SoundDMA and SoundControl use to talk to the hardware (on most platforms, at least). But whether it makes sense to use and extend those APIs or define new ones for highly-functional USB devices remains to be seen. E.g. the audio controller assumes that a simple double-buffering approach is in use, using buffers that were allocated by the user, and has no way of specifying the expected data format (other than the order of the two stereo channels).

Whilst not advocating USB, with USB I can find out:

• Sample rates supported and whether they are fixed or a contiguous range
• Sample format supported, PCM, MPEG etc – there’s a document dedicated to just formats frmts10.pdf
• whether the device requires data pushing to it or whether it will pull data.
• what controls the device has on the hardware volume/mute/balance graphic equalizers etc
• what features a device has mixer, processor etc.

Not only can you find this out in your program but you can list the descriptors at the desktop.

The application programmer can decide the features they want to support eg hardware mpeg streaming and all audio devices which have hardware mpeg streaming will work.

All the streaming is done via a fill buffer and callback when a transfer is complete interface the implementation of that is left up to the device.

On the off chance I’ve made USB sound easy it is not but its not easy to use more advanced capabilities of a built in device. What it does do is remove you from the hardware and make all audio hardware, for example, have the same interface so for audio class devices you don’t need individual device datasheets the descriptor list will tell you the capabilities and the audio class specification how to use them.

Regarding the location of a device on the computer. That is not really relevant to USB devices. Whilst there is a command in castles api to find the port a device is plugged in to it is of limited use as where its plugged in doesn’t matter.

I’ve been a bit ‘distracted’ in recent days by trying to write an analysis program for DVB transport streams. However I’ve just noticed that keene.co.uk do a USB dac that works up to 192k/24bit that only costs 40 quid. Don’t know much beyond an advert so far. But I may get one to play… erm test. Occured to me that others interested might find one useful. I’ll see if I can find out more.

Jim

Another little update. I’ve been away on holiday recently, which has of course impeded progress. Anyway, SAUG was at the Portsmouth show and demonstrating USB audio to visitors – also to Ben, who didn’t realise how far advanced it was!

I have working USBAudio module code to return the parameters necessary to open an isochronous stream to or from a device. The same code can, of course, be used to allow a stream to be opened directly by the module.

I also now have code that gets the unit ID of the Feature Unit associated with an interface.

I have to work out a suitable way to return the relevant parameters of volume and mute controls that are part of the Feature Unit so that they can be used directly to control volume and mute. Once I’ve done that, it should be possible to have user applications open streams and do the basic control settings without needing to know the specific device, and without needing to understand and parse USB audio device descriptors.

Another little update. …

Thanks. I was thinking it had gone eerily quiet… :-)

Any guess wrt when there might be something I could test myself with the USB DACs I have? 8-]

FWIW I’ve been trying to work on the ROSS document. But allowed myself to be distracted by finding that various items like the Britten War Requiem and Jimmy Smith’s The Cat are now available as ‘high resolution’ downloads. Excellent stuff! :-))

Jim

Any guess wrt when there might be something I could test myself with the USB DACs I have? 8-]

Could be within days. Which USB Audio release do your DACs conform to? Anything that uses release 2 is unlikely to work.

The DACs I use can be set to either ‘Class 1’ or ‘Class 2’.

In practice I normally have them set to work as USB1 / Class 1 because I don’t make much use of a rate above 96k/24bit. To me 192k seems needless in normal listening. Only of value for some technical test purposes at present.

So I’d expect/hope what you’ve done to work IIUC.

Although some ‘cutting edge’ audiophiles will be wanting to use above 96k/24, most hifi fans will be quite happy with the far more common 96k/24 and probably be using a DAC similar to the ones I have.

Jim

The DACs I use can be set to either ‘Class 1’ or ‘Class 2’.

Interesting. How is the setting made? A mechanical switch, or something else?

I’ve been away on holiday recently

snap

Could be within days.

Unfortunately Jim won’t be able to test Class 1 Audio as there is no support for usb1 isochronous over usb2. There is a remote chance that a Class 2 Audio device – which uses usb2 natively may work. However if Jim does have a class 2 device it will be asyncronous (else Jim wouldn’t have bought it :-)) and I haven’t implemented that yet – should get sound but timing of buffer fills will be wrong resulting in crackles probably.

The reason I haven’t implemented asynchronous audio yet is because I’m not sure that it belongs in the USBDriver. In fact I think it doesn’t but am trying to avoid too much work.

The problem with modifying the USB DeviceFS interface in USBDriver to implement audio isochronous is that isochronous becomes audio specific. Other classes of isochronous devices may have different buffer filling requirements and having to modify USBDriver everytime is silly.

A better solution I think is an Audio Class Module – more work :-(. I’ve added a lower level interface to the USBDriver so that you don’t have to use the DeviceFS interface to write a driver which means you have tight control over when and how a buffer is filled so all audio devices could be handled by a separate module without polluting the USBDriver with audio class specifics.

So the question is what API do we need for audio. I don’t know anything about audio on riscos – don’t particularly want to be involved with it. So if anyone has an idea how they would do it if they knew how to program USB this is your chance pipe up now :-)

Do we want a DeviceFS type interface? As this is audio specific you could have:

OPENOUT "audio1#samplerate=44100;samplesize=4:"

Problem with audio is you want it to work in the background. So a DeviceFS type interface may not be the best – I’m not a DeviceFS fan.

Any thoughts, however trivial, welcome.

Unfortunately Jim won’t be able to test Class 1 Audio as there is no support for usb1 isochronous over usb2.

I’m confused here.

USB Audio Class release 1 is all about the class specific device descriptors, and pretty much nothing else. It has nothing to do (AFAIK) with the USB class (1.1 or 2).

The audio devices I have are all full speed. They are also all USB Audio Class release 1.00, which means I don’t have the opportunity to test operation with devices that conform to USB Audio Class release 2. They all work with the Iyonix and the modified USB stack you made available. Assuming Jim has an Iyonix and at least one USB audio device conforming to release 1, he should be able to get successful replay.

I presume Audio 2 spec is backward compatible with Audio 1 but the main reason to use it is the use of High Speed USB

From the usb audio 2.0 spec

Many of the changes introduced in Version 2.0 of the USB Specification for Audio Devices take advantage
of the new features provided in the USB 2.0 Specification. With the additional bandwidth made available,
high speed USB operation allows the transport of multiple channels of high bit rate audio.

I seem to remember Jim is using an Armini which only has USB2 – botched board design, USB2 ehci devices should pass USB1 devices to a USB1 controller. I also think he has a device which can be switched to use Audio 2 probably for a sample rate too high for Full Speed USB. This will use High Speed (USB2) which the EHCI controller software does support. But if he also has an Iyonix he will be able to test.

The DACs I use can be set to either ‘Class 1’ or ‘Class 2’.

Interesting. How is the setting made? A mechanical switch, or something else?

It varies. For the DACMagic Plus which I’d find most convenient for tests using my ARMiniX it is a process where you power up the DAC whilst holding down one of its buttons. This puts it into special mode so you can then select/change which mode it is in.

By default it works as a USB1 ‘Class 1’ device. But can be changed to USB2 ‘Class 2’.

The main difference from the user’s POV is that the ‘1’ mode can’t cope with sample rates above 96k/24bit.

AIUI the whole point from the maker’s POV is to allow it to work with ‘older’ hardware that can only provide USB1. But I confess I don’t know enough about this area. I guess we’ll only know when this is put to a test.

Jim

Jim is using an Armini which only has USB2 – botched board design

Do you mean ARMiniX or Armini? I’m using an ARMiniX and I think they are fairly different.

Jim

Jim: for your testing, you will need the modified USB drivers from Colin. These can be soft-loaded onto an Iyonix with RO 5.20 or later. It may be possible to load them onto an ARMiniX; I don’t know, Colin is your man. I am sure that you will always need at least RO 5.20, though, whatever you’re soft-loading the drivers on. And of course RO 5.20 requires a new !Boot, which in turn requires you to follow the instructions – my apologies of you already know all this.

I’m sceptical about the need for an Audio Class module to make a special case out of isochronous transfer. All that’s necessary is to specify the correct parameters when opening the DeviceFS stream. There are a couple or so more that can’t be defaulted when opening isochronous streams than other stream types, and certainly one more when opening a stream that requires different payload lengths, e.g. CD audio requiring 176 or 180 bytes.

There is one thing I can think of that could be better joined up: replaying from, or recording to, a file. In these cases it’s highly desirable that the copy between buffers and writing to the output stream are handled in the background. The recipes to do that are always the same, and the complexity makes it worth our considering whether we can automate it in some way, IMHO.

Jim: for your testing, you will need the modified USB drivers from Colin. These can be soft-loaded onto an Iyonix with RO 5.20 or later.

That raises two matters.

A) Can I now download Colin’s drivers from the net, or are they something Colin would have to email to me?

B) At present I’m using 5.19 and I don’t think Andrew has released a version of 5.20 targetted on the ARMiniX. I’m reluctant to use something that he can’t support/recommend as he adds a number of tweaks to the package.

I’ll ask Andrew about (B). Also I do have a ‘spare’ SD card I could use for non-standard testing if necessary. But is it that there is/are some specific module(s) in 5.20 I’d need? If so for test purposes maybe I could softload them when testing?

I’m sceptical about the need for an Audio Class module to make a special case out of isochronous transfer.

My limited understanding is that in essence is much the same as saving to a file on a connected ‘hard disc’ or memory device. The device tells the main system and sending process when to send another chunk of data, and the size of the chunk, etc. I’m puzzled if this can’t work via USB1 since I’d though that was the protocols I’ve using on my Linux boxes anyway. But I realise there may be some specific problem with the RO system or hardware. I don’t know enough about this.

Jim

and I don’t think Andrew has released a version of 5.20 targetted on the ARMiniX

He has released 5.21 targetted on the ARMiniX. I have a 32Gbyte SD card with all 32Gbytes available to SDFS. Quite fast too on a class 10 45Mbyte/s Sandisk Extreme card – I am getting 15-20Mbyte/s on block read/writes

Dave

I’m sceptical about the need for an Audio Class module to make a special case out of isochronous transfer.

Not for a special case of isochronous transfer but for an audio management and audio usage API

If you think of usb classes USB Mass Storage has its own module and provides an SCSI interface. EtherUSB has its own module and hooks into the sockets interface. HID devices are not in their own module because they don’t use the DeviceFS interface and there is no other available outside the USBDriver module but they hook into the PointerV and KeyV. With my new USB library these could now be written in their own module – but thats not the point I’m trying to make. My point is that the user API for these devices doesn’t use the USB API. ie for SCSISoftUSB you use the filing system and for EtherUSB you use sockets. I think the USB stack should be layed out like this

With asynchronous OUT isochronous audio devices you have 1 isochronous OUT stream endpoint and 1 isochronous IN feedback endpoint.

If in another module I OPENOUT "usboutstream" and OPENIN "usbfeedback" thats useless because I can’t use the usbfeedback to modify the usboutstream data rate – well not very accurately. So to get DeviceFS USB to work I have to make opening an audio isochronous endpoint also open a isochronous feedback endpoint (which sometimes could be an interrupt feedback endpoint) I can do all that no problem but to do it I have to do all the Audio class parsing which I believe you are looking at. Effectively I build part of the the audio class driver into the USBDriver.

As an aside the feedback problem is not just a problem with isochronous. Interrupt IN endpoints in USB serial devices are not streams. They are more akin to a status register.

You also have to wonder does Video work exactly the same way as audio or any other Class that uses isochronous for that matter. You don’t want to be messing around adding what would be part of the Video class to the USBDriver to get video working, for example. You want to write a self contained module and not risk that you may break the USBDriver.

My view is that USBDriver should do the minimum required to access USB hardware and device classes and vendor specific drivers should be in their own modules. For that you need a USB API to do it and USB DeviceFS isn’t it. You know yourself you are always trying to work around it.

I don’t know what you are doing Dave but I presume you are doing an Audio management module ie you are enumerating usb ids of audio devices and providing an interface to control them. Why shouldn’t the audio streaming interface be in the same module? The audio streaming API doesn’t have to be a DeviceFS interface.

I’ll stop there before I write an ebook :-)

Jim. It can work like that. What I’m trying to avoid is a monolithic USBDriver which makes it difficult to replace parts. Instead of OPENOUT "USB8:" you could have OPENOUT"SOUND1:" if that is what was wanted but equally you could have the current RISC OS sound interface.

Edit: I should add that linux has pre-emptive multitasking which means you can fill the buffer in the foreground from a thread and have some guarantee that it will be filled. This won’t work in RISC OS.