Multiple audio devices
Colin (478) 2433 posts |
As I see it you aren’t registering the device, you are registering the device with a specific interface. The same device may have a audio input stream that would be a different interface so the device would be registered twice once in devices:$.audio.out.pcm and another in audio.in.pcm maybe an entry for hardware volume controls audio.out.pcm.volume which has a standard volume control interface. |
Jeffrey Lee (213) 6048 posts |
Colin: System sound is an audio out PCM player so all pcm audio out devices register with DeviceFS as Ah, yes. I’d forgotten that DeviceFS supports directories (and like you I’m also puzzled as to why serial and USB don’t use the feature) Using the directory structure to organise the devices definitely seems like the right way of solving that particular problem. Having an IOCTL interface to access the double buffer, and having the device read from the stream when the IOCTL interface isn’t in use, also seems like a sensible proposition. So I guess the remaining question is – how high level do we want the DeviceFS devices to be? If we’re keeping them low-level, and the first implementation only supports the IOCTL interface (no FIFO streaming) then I’d be fairly happy to implement that now. But if we want devices to support features not natively supported by the hardware, or we want them to be capable of mixing multiple streams together, then it might make sense to think things through a bit more first. From my point of view, I’d be happy if all the mixing/format conversion/etc. was handled somewhere centrally (i.e. SharedSound), with the idea being that most applications will talk to SharedSound in order to output sound. But that may prevent your use case of being able to simply pipe audio to a DeviceFS entry from within an application (especially if SharedSound claims exclusive use of the devices). James: Regarding hardware device registration, I seem to remember that HAL devices were to be used for that. A USB audio module would register devices with OS_Hardware 2. Correct. Most RISC OS 5 machines use a version of SoundDMA which talks to an audio controller HAL device, using a fairly generic API (although the API does keep evolving as drivers for more platforms are developed). And with my original proposal, having the USB audio device create a series of HAL devices (which then get detected by SoundDMA) is the way I would have expected things to work. However I’d consider the use of HAL devices for audio to be an “implementation detail” and not something that we’d want to expose to “users” of the sound system – the HAL devices are a bit too low-level for it to make sense for people to use them directly from within their software. We may also end up introducing new revisions of the interface to cope with specific needs of hardware – something that SoundDMA should be able to hide from the user by providing support for all the API revisions. Also for those wondering about the Pi – the problem there is that the GPU firmware (unless it’s changed recently) only allows the use of one audio output at a time. RISC OS doesn’t currently understand this concept (we don’t – yet – support multiple devices, and we can’t deal with it in the mixer because SoundControl doesn’t understand “switch” type devices), so currently the OS tells the GPU to use whatever output it thinks is best. At one point I did have a prototype audio driver that drove the PWM output directly (bypassing the GPU), so if I resurrected that then we could use that to get HDMI + analogue at the same time… but I’m not sure how well it would work in reality (CPU load, buffering, etc.) |
Colin (478) 2433 posts |
IOCTL would require you to enable the streaming interface as it uses the stream handle to access it. I’d use DeviceFS_CallDevice which is even easier to use as it takes the devicefs name.
Whilst it would be nice to enable the streaming interface just using DeviceFS_CallDevice would be a good start. If you have open and close in DeviceFS_CallDevice interface you give the sound system unique access to the stream – if you want it to have. Even just implementing that you could get direct access to any device even if there is only one – you could just give the sound system a null device and take control of the device yourself mixers would just be another audio.out.pcm.device the only difference is that you would be able to open several streams to it instead of just one. If the user chooses the mixer device for the sound system then any programs not using system sound can also send data to the device. It has the potential for the developement of new sound systems without having to modify the device modules The beauty of the system is it’s user extendable. You just write a module and register your device in the appropriate folder and programs can automatically use it. If the interface needed changing for a new class of audio streams you introduce a new root folder Audio1 and devices in that tree use the new interface and programs aware of the change can use the new interface. When you think about it all devices should do this. If all the usb devices were in a USB folder that would give you the list of devices with a USB interface. If you want a new usb stack just add a USB1 folder with its different interface. Similarly with Serial devices. People wouldn’t have to register device names if they just want to write something that complies with the interface of an existing directory. |
Colin (478) 2433 posts |
Oh and DeviceFS_DeviceCall can broadcast messages to all devices by using a zero handle you may be able to use that to tell all devices to sleep or something like that. |
James Peacock (318) 129 posts |
Yes, I certainly didn’t mean to imply that anything else other than the core sound modules should be talking to the HAL devices directly. When I was first experimenting with USB ethernet, I found the DeviceFS API really useful as I could use BASIC or C in a taskwindow. I did at one point experiment with a module which provided a DeviceFS device for SharedSound for outputting audio directly from an application. However once I started on EtherUSB, I found DeviceFS cumbersome and unpleasant to work with. What I ended up with was a messy combination of OS_Find, DeviceFS, BufferManager, Upcalls, etc; each one of which used a different handle for the same stream. |
Colin (478) 2433 posts |
I’ve figured out a good reason why USB doesn’t use the devicefs tree structure. If you are using a streaming interface on devicefs and have told devicefs to create a path variable when registering, it creates a path variable with a full stop in it which doesn’t work. So I think we will have to use an underscore instead of a full stop as a path separator which is a shame as we lose the directories in the devicefs window. The only other alternative I can think of is to stop using path variables for new device types |
Jeffrey Lee (213) 6048 posts |
So, here’s a revised plan of how a DeviceFS solution might work:
Does this sound sensible to everyone? I think the only remaining question to answer is what naming convention should be used for the DeviceFS entries. My proposal is: Audio.<direction>.<type>.<module>.<device> Where ‘direction’ is ‘in’ or ‘out’, ‘type’ is ‘raw’ or ‘mixer’, ‘module’ is the module name (to give each module full control over its device names without worrying about name clashes), and ‘device’ is whatever the module decides it should be. However I’m wondering whether we should even bother to make the distinction between ‘raw’ and ‘mixer’. Note that I don’t think there’s much point separating devices by API level – unless a massive breaking change is required, things should generally be backwards-compatible, and clients which need specific features can always use CallDevice to check if a device supports the required functionality (we could even make that part of the spec, e.g. have a CallDevice entry which checks if a certain CallDevice reason code is supported) Also note that I’m assuming that nobody really cares about DeviceFS’s path variable support, which as Colin points out won’t work with devices which are organised in directories. If we do go with this plan, I think it means that the initial implementation to fulfill my ‘basic support for multiple devices’ goal would be:
|
Sprow (202) 1155 posts |
DeviceFS entries are an allocated quantity so you’ll need to ask for a reservation (of the very top directory I assume).
Why not lump that into SoundControl if it’s something it already (almost) does? SoundControl is a generic enough module name for it to make sense that it’s the keeper of *AudioDevices too. |
Colin (478) 2433 posts |
I’ll answer bits at a time an effort to make it readable.
yes
Direct callback has to use Calldevice. IOCTL requires you to open a file stream – see next point.
No. I’ve been writing a USBSerial module to try out writing a file streaming DeviceFS frontend to a file streaming DeviceFS backend and it is very inefficient. When a program uses OS_GBPB to read a block of data for example this is what happens. 1) USB dma’s the data to a block of memory The problem is that devices don’t stream bytes they stream blocks and the circular buffer deconstructs those blocks squeezing them through a byte pipe and the other end has to try and reconstruct them. The threshold for the filling/emptying callbacks is not very useful as data is moved in and out of the circular buffer in blocks and the block size tends to be the size of the buffer. The USB interface, for example, should as a minimum give direct access to the blocks of memory used for DMA then a usb audio device could work as efficiently as a Hal device. There are other USB inefficiencies such as effectively single buffering which also slow down USB through put. The ‘callback’ interface needs to be able to cope with a device that does triple buffering ie each callback specifies the size and location of the buffer. Then it would then be possible for usb to queue one block while the device processes another and the software fills another ensuring usb is always transferring data. |
Colin (478) 2433 posts |
I thought I’d outline the overall vision as I see it. I’ve kept it as simple as possible so as not to clutter it up with details at this stage. I’ll define
Terminals are characterised as only having one end. In the simplest audio system possible Output terminals are plugged into Input terminals. So an Input terminal needs to be told which output terminal to use – it has a ‘UseOutput’ function. So to get an overview of the problem what do we need for the following to work
we need the user to know that they are both audio devices, that microphone is an Input Terminal and speaker is an Output Terminal and that the data format coming out of microphone is readable by speaker. So a devicefs entry for microphone could be
and the speaker
where bps is bytes per sample. So a user can look through the audio tree and can connect the 2 together with
It’s not enough for a device type to be raw as you don’t know which raw format. In this case the type is pcm.2ch.16bit.4bps. Simultaneous audio streams to different terminals are trivial the user just matches up input and output terminals. Now a terminal doesn’t have to be an audio device. A piece of software generating audio.in.pcm.2ch.4bps audio data is an Input Terminal and can ‘UseOutput’ any audio.out.pcm.2ch.4bps Output terminal in the list. And that is what is being proposed as a first step here. We treat the entire existing RISC OS sound system as an Input Terminal. All that is needed from the DeviceFS naming is the ability to match the Input Terminal format to the Output Terminal format. In the future we add ‘Units’ a unit is a pipe with an Output terminal at one end and an Input Terminal at the other. So we have
where the microphone ’UseOutput’s the Unit which ’UseOutput’s the speaker. An example of a Unit would be a mixer – which has multiple Output Terminals and a single Input Terminal and the input format matches the output format, a format converter which takes a single input in one format and outputs it in another. There’s no need for a switcher using because ‘UseOutput’ provides that function. With these building blocks you should be able to have a system as complex or as simple as you want. |
Colin (478) 2433 posts |
You’d also need to reserve the paths. The paths are the class of audio device and the leaf is notionally the device. so you would need the device allocated to the correct folder. Which brings us to the knotty problem of device allocation and removable devices. Presumably USBn has been reserved and the USB stack can allocate any number for the stack which is ok as the user can enumerate USB* to enumerate the list of USB devices. Devices are numbered 1 higher than the last usbnumber registered. My question is why do I need allocation for a device like ‘Serialn’ why can’t I just use the last serial device number registered to devicefs + 1. An iyonix has fixed 2 serial ports serial and serial1. If I added a pci serial card I’d have to register serial3 and serial4. If I then add usb serial devices you have to allocate a name root so I can add anything after this to give a handle to the device. My suggestion is why not just require allocation for the root of the device name (which is the device class) and then anyone can use a device name + number 1 greater then the last registered device. So all serial devices can be identified as Serialnnn. The Iyonix hal hardware would still get serial and serial1 as they will be registered first in the module list A pci device module loaded before a usb serial module would always be Serial2 and Serial3 at switch on. If you want to use something like ‘serial_CP2102_’ then you would need that allocated and numbers could be added to the end of it using the same scheme but all serial devices can still be enumerated using the ‘Serial’ root. |
Colin (478) 2433 posts |
You can register the device as unbuffered. But you would also have the problem of a user attempting to write to the device by using OS_GBPB. Don’t know about the unbuffered mode but the buffered mode locks up nicely if you don’t use data entered with OS_GBPB so going the filestream route would require you to deal with this situation. |
Jeffrey Lee (213) 6048 posts |
You’re not making this easy for me, are you? ;-) * A stream-based approach using standard file IO If we aren’t supporting standard file operations, then is there any point in using DeviceFS? What advantages will it give compared to using a bespoke API? (E.g. the original proposal, or maybe some evolution of the current audio HAL device API). So a devicefs entry for microphone could be Perhaps I’m interpreting this a bit too literally, but if a user – i.e. human – has to navigate through a directory structure 6 levels deep then it doesn’t seem very user-friendly. Also if you’re going to have one DeviceFS entry per supported sample format (data type, channel count, etc.) then you could easily end up a large number of devices being created for just one piece of hardware. E.g. HDMI can support both LPCM and many different compressed audio formats. There can be between 1 and 8 channels (or is 2 channel the minimum? I’m not actually sure). For LPCM the data can be 16, 20 or 24bit. Then you’ve got the different packing options for 20bit & 24bit. And if you’re trying to represent everything important using just the file name then you’ll need some way of including the channel ordering in there as well – which for 8 channels would work out as a lot of different permutations. I think it would be much better to keep all of that out of the device name and either use the special field to specify the parameters (when the user is configuring things manually) or have the software auto-negotiate to try and find an acceptable format.
Specifically for serial devices, it looks like the requirement to register names was made as a conscious decision. The spec doc for the IOCtl interface says:
|
Colin (478) 2433 posts |
I’m surprised you are still discussing it I was expecting a prototype working over the weekend :-) The other 16 Jeffrey Lees must be on holiday
Yes to register the audio devices as a device So we have a central point to see what is attached, a standard way of enumerating devices and a standard way of accessing devices. The alternative is to write an ‘audio devices module’ which does the same thing in a non standard way. If you are feeling adventurous the streaming interface could be hijacked and registering your device with version 2 would replace the circular buffer with the callback system. It seems to me that just opening a filestream for the sake of a handle is fraught with danger Re device name complexity. You are right about device name complexity and maybe the device name isn’t the place to show it. The trouble with audio is when you spell it out it is complex especially if you want a versatile system. Whether you spell out a devices capabilities in the DeviceFS directory tree or in a DeviceCall you still have to do it. You can’t just plugin an output device and hope that it works there has to be a negotiation between the output and input device. To that end each device needs a capabilities function – basically a list of audio formats it supports. We then go back to keeping it simple and have AudioOut and AudioIn DeviceFS names. An audio application can enumerate the AudioOut devices and a DeviceFS_DeviceCall Info – all devices, not just audio, should have this but that is another story. Info would be a block of text. You may also require ShortInfo for a possible menu entry name. Then you have the problems of internationalising info and Shortinfo – the list goes on until you decide it’s not worth it. Anyway the complexities of writing a configure plugin to select the system sound backend are not of immediate concern with only one or two to choose from for a particular machine it can be ignored for the moment in the same way that a user can’t find out if Serial1 is the one nearest the edge of the case. I think you just need to implement something simple to get it working with a view to future expansion and if nothing more gets done at least it will be better than what we have now. A basic standard audiodevice api with noconversions in the device will open up a lot of possibilities.
Yes I’ve read that and my question is (A) why, when once Serialn has been registered Serial devices enumerate naturally from module initialisation, and (B) I can’t register a device name as my module will pick up any serial devices it recognises so could be 1 or 10 serial devices so what do I do? |
Jeffrey Lee (213) 6048 posts |
Cool. In that case I suspect I’ll ignore DeviceFS for now and go with my original proposal (i.e. the cut-down version that’s summarised here). Getting it working will only be a small amount of effort, and if/when we switch to DeviceFS in the future there shouldn’t be any awkward bits of API left over from the old system. E.g. the ‘enumerate audio devices’ SWI could easily be retconned to be an ‘enumerate SoundDMA compatible audio devices’ SWI. |
Colin (478) 2433 posts |
An odd conclusion given your comment on device identification on the other thread. A DeviceFS module which disables the filestream part when registering is just as easy to write as a module for swis. The entrypoint passed when registering the device is the equivalent of the modules swi handler. Where you gain is you don’t have to write another module to manage the devices. But you’re writing it. If ROOL were interested in the device identification scheme I outlined in the other thread or anyone came up with a better idea I would implement it for them if they didn’t have time. I could also add it to usb devices. No point in me doing anything if they are not interested though. |
Jeffrey Lee (213) 6048 posts |
Not really. You said to do something simple, and the simplest thing to do (within the context of allowing one of many devices to be selected) is to stick with having SoundDMA use HAL devices directly. If we can come to a conclusion on how a DeviceFS based system should work within the next few weeks/months then I can try and find the time to switch sound over to using it in time for RISC OS 5.24. But I don’t really want the DeviceFS discussions to hold up getting this feature done and checked in (with any luck, there’s only about a week or two of work left to do – and getting it in before Christmas would be good because once the new year rolls around I suspect I’ll be in for a whole new world of zero page related pain). |
Colin (478) 2433 posts |
Fair enough. Regarding zero pain it may be a good idea to keep a copy of the last non relocated vector version of riscos on the web site. I have a version of TechWriter Pro which fails with zeropain and I don’t want to buy a new version for the number of times I use it. There may be others like me. |
Mike Freestone (2564) 131 posts |
I bet Martin W has spent thousands of hours improving techwriter and has to earn a living. You’ve posted over 1000 times, keep a jar by the keyboard and put 10p in each time you post and you’ll soon have enough to upgrade, to keep the platform growing |
Rick Murray (539) 13806 posts |
Good luck if that works for you. I have a plastic pig (!) that counts coins for my “go to Japan” fund. Three1 years later I might have enough to get to Orly to look at a plane going there…without me on it.2 1 More like twice that, but that’s depressing so I tell myself it’s three. 2 Not that flying over the Arab lands is a terribly bright idea right now. |
Richard Walker (2090) 431 posts |
That sounds a bit harsh, Mike. I do not see any link between post frequency and the investment in an application. And I certainly value Colin’s contributions. Back on-topic, doesn’t the ZeroPain module allow such apps to continue to function? Or have I misunderstood its purpose? |
Steffen Huber (91) 1949 posts |
I suggest instead that every time a user reads one of Colin’s postings here and finds it helpful, he should make a mental note. After a few days, we count those notes, 10p a piece, and buy Colin the latest TechWriter version. |
Jeffrey Lee (213) 6048 posts |
I managed to get this code checked in last night – or at least the core changes that were needed for SoundDMA & SoundCtrl. Documentation updates on the wiki should be available in the next few days, time permitting. But the main reason I’m posting this here now is that while implementing the changes I’ve realised that requiring the drivers to be implemented as HAL devices may stifle creativity – the device IDs need to be centrally allocated (and ROOL currently don’t seem to have a policy for that with regards to external developers), so to develop a new driver or try out something new there’s a bit more effort involved than just (e.g.) throwing something into DeviceFS. So in the next few weeks/months I expect I’ll try and put some more effort into coming up with a new interface (and one which will be designed to cope with bigger things like different sample formats and multi-channel audio). In the short term, I’m hoping to find the time to retire Sound0Trid (Iyonix version of SoundDMA) and switch it over to using the ‘HAL’ version instead. The audio HAL device + Sound0HAL have now matured to the point where there aren’t any technical reasons left for why it can’t be done, so it’s just a case of ripping all the Iyonix-specific logic out of Sound0Trid and putting it in a HAL device instead. Sadly the same isn’t true of the IOMD version of SoundDMA, where we have to retain compatibility with systems which lack 16bit sound hardware.
Technically the ZeroPain discussion is off-topic since this thread is about the sound system ;-) But since I brought it up I should probably respond! The purpose of ZeroPain is to allow old apps to continue to function, but (a) it’s not perfect (so some software will fail), (b) Colin hasn’t given any reports for how TechWriter fails (so we can’t improve ZeroPain to make it cope with it even if we wanted to), and © ZeroPain (in its current form, at least) is only meant to be a temporary solution – the final solution is yet to be determined. Which reminds me that I should probably raise the subject with ROOL so that I can work out what I need to be focusing on (options are likely to be an improved version of ZeroPain, a patcher app which will be able to patch stuff on load, or nothing at all – in which case buggy software will just stop working). |
Colin (478) 2433 posts |
It doesn’t produce a zero pain report it just aborts on data transfer. edit:
edit2:
doesn’t look like a bug – more a feature |
Jon Abbott (1421) 2641 posts |
Colin, what Module is it that’s crashing in you debug dump? Is it sound related? It’s reading from PageZero so will need modifying to use a legal method. Might be worth creating a new thread, TechWrites sounds off topic to me.
I’ve yet to test, but the wording here scares the living daylights out of me from a compatibility perspective!! Does 8bit sound still exist in the revised SoundDMA? I’ve not misinterpreted as now being 16bit only post IOMD? Or have you simply moved SoundDMA over to the HAL model where possible, leaving IOMD as-is. |