Audio Recording API

One important feature is missing in RISCOS. That is a general call to record or input of audio.
This might be by the AUDIO IN socket or by USB devices like a webcam.

As part of our RISCOS Cloverleaf project we are currently programming a chat messenger !ChatBox. First public version might be avaiable in about 2 to 3 weeks. For this application we need a possibility to get an audio feed by a headset or a Webcam to offer voice call.

So RISCOS must have a general “Audio in” support. A source must be selectable eg “Onboard microphone”, “Audio in”, “Webcam”. Standard “Audio in” calls must be implemented.

Jeffrey Lee gave me this first info:

“Audio input is a topic that’s come up a few times over the years. As you
probably know, the OS doesn’t have any built-in support for it, so we’ve
got two problems: there’s no common API which applications can use, and
there are no drivers which implement that API.

Someone did create an “!AudioIn” app for the Iyonix which would allow
programs to record audio by reading from a DeviceFS stream. Unfortunately
that app is no longer available (the author’s site went offline a few
years ago). There’s also a USB audio driver from Dave Higton which
includes audio input support (and also uses DeviceFS for buffering the
data, but uses a different API for configuration). As far as I know those
are the only two examples of audio input drivers for RISC OS 5.

I think using DeviceFS for audio input is the right idea (it’s nice and easy for
applications to deal with), but I don’t think either AudioIn or USBAudio
have got the API right (AudioIn was missing some key bits of
functionality, and USBAudio is tied a bit too closely to USB). So my main
thoughts are that if/when we do create an official audio input API, we’d
probably want to design a new API. But that’s about as much work as I’ve
put into it!"

This topic has the aim to collect information and define the API calls that are needed,
A) I would like to have more feedback about possible available drivers.
B) Does someone know the programmer that did the driver !AudioIn" app for the Iyonix?
C) What calls should be supported with what parameters?
D) Should the “Audio In” be part of the HAL ?
Any usefull comments and feedback is welcome.

I had a quick look at my Iyonix archives.

The author of !AudioIn was Christian Ludlam.

I don’t know him personally, but the !Help file for AudioIn did contain his email address (which I’m not going to publish here as it’s from 2004 and may have changed hands since).

I figured you might get more responses to your query if people knew you wanted to contact “Christian Ludlam” rather than “the author of AudioIn”…

The wayback machine has an archive of his site (looks like it vanished 10 years ago?). No archived downloads.

http://web.archive.org/web/20081005015438/http://sudden.recoil.org/others/index.php

There has never been a unified API for audio recording. Instead sampling has always been tightly bound to proprietary APIs for various sampling hardware.

Some of the earliest devices were from Armadillo – The A488b rings a bell. They had software and hardware solutions for Arthur. HighNote I think it was called, I still have the podule somewhere.

So that came with a module that provided sampling SWIs. Other systems were similar. Computer Concepts tried to provide a generalised API with the AudioManager module but who else implemented the &8004E service call it used?

So no, there isn’t a single API. AudioManager came nearest.

[And AudioManager had a load of frivolous and extraneous stuff too, so I certainly wouldn’t hold it up as a model API]

I would have expected an audio/video input possibility via Replay – what did (24)i16 and Videodesk by Irlam use? They also captured audio after all. Did ProSound/StudioSound provide direct audio recording?

I don’t think ProSound had any sampling aspect.

Replay was a playback format, not an API (there is no Replay module, just a huge pile of codecs in a directory). The Replay Encoder was just a stand-alone program.

I spoke to the ProSound guy about developing a suitable API when he updated ProSound, but he didn’t want that level of responsibility/work.

I think there is an opportunity here – there’s plenty of hardware with audio-in capability now, and with programmers available to write applications using audio-in (Cloverleaf’s chat program was demo’d on our stand at the show), it really just needs someone who knows a bit about such APIs (eg. what does Windows/Linux offer) to define things a bit.

There are plenty of USB audio in/out devices available; Chris Gransden added isochronous support to the RISC OS 5 USB stack; I created a USB audio control module (available from my web site here allong with the API documentation). So you’d think almost everything was in place.

There have been a couple of demo apps to record and replay audio. Some more programming effort would be helpful, to add the desirable control features.

But there may be a difficulty with communication apps – I don’t know…

Most USB audio devices only operate at sampling rates of 48kHz and/or 44.1kHz, or even higher. That’s fine for hi-fi, but communications apps normally AFAIK expect much lower sampling rates (most commonly 8kHz) and/or compression. We don’t have the real time signal processing power to change the sampling rate properly. (If you try dropping or repeating samples, you’ll immediately become aware why this is not good.)

I was lucky enough to come by a headset that properly supports sample rates from below 8kHz to above 48kHz, but it hasn’t been available for many years now. (Its ear cushions have long ago disintegrated into powder.)

Where does this leave us?

If anyone wants to use USB audio devices directly, and can cope with the available sampling rates, all the rest of the code and documentation is available.

If someone wants to take the available code and docs, and make available a higher level API, that is possible AFAICS.

I’d be interested to hear what happens, even though I’m very unlikely to take an active part in it other than supporting the module that I wrote. I’ve already got at least enough other things to do.

I’d like to be proved wrong about the availability of inexpensive USB audio devices suitable for communications use.

Yeah, the API should be interface agnostic, obviously, supporting existing hardware on legacy devices and also virtual devices – for example a loop-back so that the output of one program can be recorded by another.

I’m wondering if SharedSound is already half of the interface.

By which I mean that SharedSound is basically a Mixer and Sample Rate Converter, and though its “Handler” and “Driver” terminology is not very helpful, it simply mixes a number of inputs together and delivers buffers to a recipient… which is normally SoundDMA of course. So it’s a bus.

One could theoretically instantiate SS to create a new bus, connect required sampler(s) to it and receive the results at the required samplerate (which may not be the same as the sampler’s natural rate). Equally, if there are multiple output devices one could instantiate another bus to drive them.

Having said that, there is some overlap between SoundDMA and SharedSound here, especially if one wants the system sound to come out of multiple devices simultaneously – SharedSound is a many-to-one mixer, and if different output devices needed different samplerates…

Hmm, perhaps SharedSound’s output (to SoundDMA) is itself a SharedSound Handler (do I mean Driver, I forget) so that it can route the output of one bus as input into another, thereby allowing output at different sample rates simultaneously.

If you’re starting off with ‘low resolution/rate’ material I’d assume that simple linear interpolation up to 44.1k/48k would be OK. That isn’t particularly hard to do. Better forms of interpolation are slightly harder, but you can find some examples on my website. Albeit for going from 44.1/48 to double that rate. :-)

However for recording purposes it makes sense to use a decent USB ADC. And if you must, then use something like sox to get a different rate. And the cheap Behringer devices will run at less than 44.1k/48k anyway, for DAC and ADC use. Again, based on Colin and Dave’s work I wrote simple programmes to do this ages ago. Even added a nice PPM display for the recording process so the user can ensure they get the levels right.

The big problem is that the gap between USB audio devices and the stuck-in-the-past RO sound system hasn’t be bridged. Denying people the ability to simply choose whatever USB audio device they fancy, and use with all the usual RO software.

Agreed, except I would want the API to be interface-agnostic (and definitely not limited to USB devices).

Yes. What’s really needed is something equivalent to the way we can now, with appropriate systems, set the output to go either to the onboard hardware or to HDMI. This should also include USB, and input. Then all existing players, etc, should be ‘hardware agnostic’ at least in terms of type.

The potential snag for USB is more likely to be the wish to use 24bit because the standard ‘ROSS’ is only 16bit. However the reality is that those bottom bits are almost all noise anyway. So in practice it wouldn’t matter if 24 bit values were dithered to 16. Might offend the pure-in-heart but unlikely to bother most people. And we could still write specialist replay/record software that does 24bit and goes ‘directy’ as t’were for the fusspots like me. :-)

As it is, simply playing a file of any type which !CDC can understand rules out sending it to a decent USB DAC. Given how often now people play mp4 files, etc, on other platforms/devices this is a lack that will baffle many.

I’m sure some would want floating point.

The most recent ‘fanatic trends’ have tended to be DSD (which USB can trandfer via making it look like LPCM) and ‘MQA’. However it looks like MQA may be dying on its feet. If so, I won’t weep. For most ‘serious’ audiophile (nuts) the main wishes are for rates > 48k and 24bit LPCM. Measurements generally show that this adds about 3/4rs of what injuneers call ‘SFA’, though one you get above 96k rate. 8-]

Uggh DSD. Never mind the quality, ffffffffeeeeeeeeeeeeeelllllllll tttttttthhhhhhheeeeeee wwwwwwwiiiiiiiiddddddddtttttttthhhhhhhhhh.

Hadn’t heard of MQA, thanks for that.

ffffffffeeeeeeeeeeeeeelllllllll tttttttthhhhhhheeeeeee wwwwwwwiiiiiiiiddddddddtttttttthhhhhhhhhh.

Meh. When my dad was younger than I am now, the cool “in” thing was quadraphonic sound with eight track tapes. Who needs wwwwwwwiiiiiiiiddddddddtttttttthhhhhhhhhh when you have speakers at each corner of the room and an amp that goes up to eleven?

I well remember the 8 track, and I was an early adopter of ProLogic Surround, so I’ve long hankered after a proper soundfield installation, but once Dolby bought Imm Sound to bury Ambisonics in favour of their steerable ATMOS system I’ve given up hope of good mass-market channel-less audio. Now I just sit quietly in the corner with my BeyerDynamics. :’-(

Thanks for all replies about this matter. What I can see is that Dave Hidgon has the most evolved and only available solution for this problem.

So I would suggest that we continue discuss on his Sound API proposal and evolve that if needed.
Question if his API support also USB Sound in or just hardware AUDIO In. Also we should incluse general AUDIO support
Also interesting looks his SIP Phone application but I guess there is some SWI calls supplied by the IYONICS to support the Audio IN right?
So what we also must establish is a general collection of SWI call that offers the hardware support for each board. Or is that part of a full implemented HAL?

A related task might be the RISCOS sound system with 16 bit instead of 24 bits (as mentioned from Jim Lesurf). And general RISCOS settings managing like select a sound output (Internal speakers, AUDIO OUT, USB, Bluetooth speakers).

Following the link to Dave Higton’s Sound API.
I would like comments and suggestions on his existing Sound API so that we can decide over a complete Sound API as standard for RISCOS.

Link SOUND API Dave Higton

I am strongly opposed to a USB-specific API. That’s as far from the philosophy of RISC OS as you can get.

I am strongly opposed to a USB-specific API.

Agreed.
You guys do realise that the Beagle devices have an audio input, right?

Sorry I oversaw that Dave Higton’s API was based on USB Audio stuff.
But an universal Audio IN API must also cover USB devices like Webcam or how they should be managed?
Any user application (for example our Chat program !ChatBox) that needs an audio input should get a list of Audio devices that are avaiable.
So that should list the Audio IN socket (for headset) and the Webcam with integrated microphone.

What I mean is if we delete the “USB” from the API declaration. Don’t we have then what we need to have?

The USB Audio class driver I did was intended to provide one layer of control for USB audio devices. I envisaged that it would be called as necessary by a generalised sound API – certainly not that it would in any way become the new sound API.

I am strongly opposed to a USB-specific API.

For audio, so am I.

Any times ago I have play around with any USB devices und Colins Isoc-stuff.
Was working with any of my USB devices e.g. webcam (record audio only) and headset(record and play audio).
At this time I play around with an IP-Cam for streaming (A/V). Is working and the stream is usable with mplayer and ffplay in RISC OS.