problem using _kernel_oscli

Apologies. I probably posted this in the wrong forum earler, and this one is more appropriate!

I’m trying to write a program that can make use of the ‘flac’ port to read chunks of data from a flac file and then analyse them. However I’m stopped by something I don’t understand about _kernel_oscli (Norcrof/Acorn compiler). So I hope someone can explain and suggest a solution… The code I’m using is as follows

(Sorry, as usual, the forum text engine ‘interprets’ text and I can’t get it to appear correctly. Curiously, the same text comes with different ‘effects’ here than on the other forum.)

The flac command gets acted upon and the output file of raw values appears. But the program (being run in a TaskWindow) halts. I don’t see the “temp file made” text or anything that follows to tell me the size of the saved data file.

So far as I can see, running the flac command with _kernel_oscli terminates the entire program which issued the command, so I can’t then examine the results with following code or proceed.

Can someone explain how I can ensure the process continues OK?

Failing that, anyone have code I can actually understand and use to DIY the flac decoding? I need this to be accurate (i.e. give the same results as flac and sox) and work with 16 and 24 bit flac files of any standard sample rate.

Jim

Damn! One minute after re-posting the request for help I burrowed though my old Acorn ‘C’ book and decided to try the stdlib.h ‘system()’ instead. That works! So good news, even though I’m now looking a dimwit for asking twice just before twigging!

Still would like to know why system works but the kernel oscli doesn’t. The oscli method seems fine for other commands I’ve used it for. Is it because they weren’t ‘added’ commands I have in my local library directory?

Oh well, fingers crossed I’ll now be able to do the flac file health check program OK.

Jim

Your oscli command actually loads and run the “flac” program at adress &8000 thus overriding your own code.
I presume that is caused by the fact that the kernel is a remnant of the BBC micro and has no concept of tasks and so can only work in the current environment.

I have read somewhere (could be in this forum or maybe on Gerph’s rambles) that C’s system() starts by copying your program somewhere else in memory on entry and restore’s it on exit to avoid this problem.

Still would like to know why system works but the kernel oscli doesn’t.

Speaking from experience with gcc, system() will be using a fork() and exec() in other words it runs the command in a new process taking with it a copy of the existing environment. This needs a another chunk of memory to do so, and you can sometimes see this reflected in the tasks view, if it runs for long enough.
The alternative is to use the libraries/API’s and integrate required parts into your program.

I have read somewhere (could be in this forum or maybe on Gerph’s rambles) that C’s system() starts by copying your program somewhere else in memory on entry and restore’s it on exit to avoid this problem.

Yes, but it is fairly basic. Just better hope there’s enough space and the application being run is small enough. ;-)

https://www.riscosopen.org/viewer/view/castle/RiscOS/Sources/Lib/RISC_OSLib/kernel/s/k_body?rev=4.28#l2226

What had caused me to sleepwalk into this was that using oscli for simply system commands worked fine and the program that issued the call continued. I did try giving the application program a much bigger wimpslot without it curing the abrupt exit.

So far I’ve left the wimpslot set to 20 megs. I’ve not checked but as a Linux port I expect that flac wants a fair bit of space. I’m wary as I’m using the current ARMiniX version of RO which will assign a large slot when needed, and in the past I fell into the trap of not setting a large wimpslot for a program – which then duly failed when people ran it on older versions of the OS.

Is there a way of getting a call like system() to tell you what amount of space it needs/used? Otherwise I’ll be trying to spot brief changes in the tasks display list.

The alternative is to use the libraries/API’s and integrate required parts into your program.

Yes. Ideally, I’d prefer to do that. But I fear that learning how to include the required flac decoding, etc, into my program would take me far more effort than simply piggy-backing onto the flac port. FWIW I did wonder about using sox instead so the program would accept more types of input audio file. But most of the relevant files people will want to examine will be flac or wav. I doubt that 128kbps mp3 users will be bothered about the details my programs aim to reveal. 8-]

Jim

Yes, but it is fairly basic. Just better hope there’s enough space and the application being run is small enough. ;-)

Justin seemed less than enthusiastic about the original implementation (the one that I assume is still in RO5) when he looked at it: http://gerph.org/riscos/ramble/environment.html#Environmenthandlers

I’ve not checked but as a Linux port I expect that flac wants a fair bit of space.

I found flac 1.3.1 needed 4096K to run in. I understand what you are saying, a lot depends on the complexity, documentation or sometimes examples can give a hand up.

But I fear that learning how to include the required flac decoding, etc, into my program would take me far more effort than simply piggy-backing onto the flac port.

Are you running this as a Wimp task? If so, what you want to be doing is launching flac as a separate process in a TaskWindow. You can allocate as much or as little memory as you wish, and then watch for it terminating. You’ll find an example (for launching GhostScript) in PrintPDF’s convert.c/h files.

http://gerph.org/riscos/ramble/environment.html#Environmenthandlers

Wow. Thinking too hard about that is enough to make me a bed wetter.

Maybe the first step to sane multi-anything is to teach the kernel about the concept of having more than one application at once? After all, why so much messing around when it may be possible for system() to ask the kernel to switch out the application (entirely) and start up the requested one?

Is there a way of getting a call like system() to tell you what amount of space it needs/used? Otherwise I’ll be trying to spot brief changes in the tasks display list.

You won’t be able to spot any changes anyway – the way that system() works means that the child task can’t grow or shrink the wimpslot.

And no, there’s no way of finding out how much is used. I guess you could run flac manually from within a task window and see what the wimpslot has grown to once it’s finished. Then make your program try and make sure there’s at least that much memory spare when it tries to run it (not sure what the best approach for that would be – I guess make it first try and shrink the wimpslot as low as it will go, then grow it by the required size)

Maybe the first step to sane multi-anything is to teach the kernel about the concept of having more than one application at once? After all, why so much messing around when it may be possible for system() to ask the kernel to switch out the application (entirely) and start up the requested one?

Agreed.

Are you running this as a Wimp task? If so, what you want to be doing is launching flac as a separate process in a TaskWindow.

I’m running my main program in a TaskWindow, then having it call flac – now by using system() – to issue the command string for flac. It does work, but it a but of a kludge.

I’m doing this on a ‘chunked’ basis. This be because some flac files would expand into very large LPCM files. I don’t want to set a strict maximum limit or have the process run out of free space, and I want to keep up the processing speed. So I’m using flac to decode one 5-second chunk at a time into a raw lpcm file on ramdisc, read those and then go on to the next chunk. Means I can have the program handle very long high rez flac files. (I now have the program working for high rate and 24 bit flac as well as the more common ones people will rip from audio CDs.)

In use, the user sees a file appear on ramdisc that is big enough for 5 secs of audio at the file’s rate and bit depth. That seems to stay, but is regularly being replaced by a ‘new’ file of the same name and size as a temp store for the data I’m getting using flac and then reading back and examining.

Many of the files I have are about a gigabyte when in lpcm form. So not very practical to have to decode the entire file and put the results somewhere to then examine.

As the French almost say; It’s not magnificent, it’s a railway station. :-) But then that sums up most of my programs, anyway!

And no, there’s no way of finding out how much is used. I guess you could run flac manually from within a task window and see what the wimpslot has grown to once it’s finished.

I may experiment. Or just chicken out and leave the WimpSlot with a daftly large value for safety. 8-] At present I don’t even know if the slot flac needs grows with the size of the input file.

With luck I’ll have a version that is safe to release today or tomorrow, and then people can try it and point out my errors.

Jim

I’m running my main program in a TaskWindow, then having it call flac – now by using system() – to issue the command string for flac. It does work, but it a but of a kludge.

From what I’m learning here about this version of system(), it doesn’t use a new process, So your program may run from the F12 CLI for example. I dont think memory has to be specified there if you have doubts about the amount you have allocated.
Steve’s suggestion/example of using the Taskwindow /command/ to run flac would allow you to specify a wimpslot for that operation only, and shouldn’t affect the base program’s requirements. Using the Taskwindow command always makes a new process (with or without a display) compared to CLI or Obey which runs the command, before continuing. (similar to system() by the sounds of it).
Generally you dont need a new process, as in your case. A process thoretically would allow your main program to report that the command process was unsuccessful and continue in some way. In practice, if the process command has caused a crash, things may be unstable after that, and sometimes it pays to reboot to be sure you are working with a clean slate (the system is the full quid).
It is interesting to know what can or can’t be done with system(). I think the way you are using flac in short measured bursts it could be predictable and be reliable perhaps? Seems to be a lot of doubt around using it though.

It is interesting to know what can or can’t be done with system(). I think the way you are using flac in short measured bursts it could be predictable and be reliable perhaps? Seems to be a lot of doubt around using it though.

Dunno. It seems OK here for the purpose. But we’ll know more when I finally release a version into the wild! 8-]

I’m hesitant of using ‘child’ taskwindows as the process currently runs though a number of 5-sec chunks per sec. And, say, a five min file takes about 60 of them. Doing it ‘one new taskwindow per chunk’ presumably would mean a really strange effect, or an accumulation of taskwindows.

In the end, I guess I need to learn how to do the flac file reading and decoding for myself. If the current approach works well enough that people show they find it useful, that’ll be worth a go. But if no-one uses the program or says they need it, then I can just leave it ‘as is’.

FWIW As things stand I want to at least find out how to read the flac file header to read file content details like the playing duration.

BTW One other question about something I should know, but don’t.

Since flac is required to be distributed with its source code, etc, I’ve included instructions in the !Help telling people how to get that and to put a copy of the flac absolute into their ‘library’ directory. However when I do that to check I get a problem I’ve seen before but don’t know how to avoid.

Having the filer ‘see’ the !Flac distribution in the zip, when I run my program it fails because it can’t find ‘flac’ in the library. I’ve noticed the same behaviour if whilst I’m working I make a directory the ‘current’ one using the ‘set directory’ filer menu option.

Is there a quick way of clearing that? I’ve simply been rebooting if it happens, purely because I don’t know how to actually clear the problem. And it happening may confuse or deter other users even though I can include a note about it in the !Help.

Jim

Just to say that I’ve now released an initial version of !Flac_HealthCheck and it can be obtained from my software page at

http://www.audiomisc.co.uk/software/index.html

More info on the ‘Announcements’ forum.

All being well, I’ll improve it once I have spent some time studying flac file header details..

Jim

or an accumulation of taskwindows.

That shouldn’t happen if using —totally-silent
In my flac2usb player I chose to leave windows displaying because if things do go wonky, at least you can close down the tasks without referring to the tasks view. StrongEd can re-use windows for subsequent tasks, which makes things a bit tidier. But unlike you I had to use the secondary process’s for pipe: to work.

FWIW As things stand I want to at least find out how to read the flac file header to read file content details like the playing duration.

Having the filer ‘see’ the !Flac distribution in the zip, when I run my program it fails because it can’t find ‘flac’ in the library.

I’ve now been reading the flac format details from xiph. I think I can now write some code to find out the length of the flac files. Fingers crossed, that’ll let me fix the current flaw. But no doubt there’ll be other flaws I haven’t yet spotted! 8-]

Yes it is standard procedure for the packages to set an alias to their own package directory.

What I’m puzzled by is why setting the alias causes the call to fail. I’m using the old approach of having a copy of flac in the library directory so I can just say ‘flac’ to find it. I’d have assumed that having ‘flac’ aliased to point elsewhere should still work if there is a copy of flac at that location. However I’m wondering if my problem was that the location was in the zip which I’d closed.

And I’m more puzzled by having the call fail if I, say, set my ramdisc as the ‘current directory’. Does that also make it the library directory? I’m using full pathnames for the input and output files I’m telling ‘flac’ to use.

I’ll try the ‘%’ trick and see if that dodges the bullet OK. If so, I’ll make that the standard for a future release when I’ve fixed the ‘misses the last part of some files’ flaw.

Is the code for your flac2usb available? It might be useful for me to see how flac decoding is handled. Or are you using the same sneaky trick of calling ‘flac’? :-)

Jim

I’ll try the ‘%’ trick and see if that dodges the bullet OK. If so, I’ll make that the standard for a future release when I’ve fixed the ‘misses the last part of some files’ flaw.

Is the code for your flac2usb available?

I think using Unset alias$flac with an explanation why in your !Run file might be the least drastic solution. If you really did need an alias you wouldn’t ever be able to use it if you use %flac in your code.
You could as an alternative, have people put the flac package in your application directory and filerboot it in the location <obey$dir>.!Flac or whatever.

I am doing that so Colins IsocPlayer can be found if it hasn’t already been clicked.

I thought you had already tried my player, I put a link to it in your previous thread.
Today I got mpg2USB working in a similar way.
They are not pretty, but can play flacs and mp3’s. Be careful to have your USB audio device plugged in before running, or you could get a system crash.

Flac finds the wav filesize and puts it in the header, there may be a problem with the mp3 players doing this, for both madplay and mpg123(raw) I used a default setting of x00 x00 x00 x24 which is about 600MB.
raw123 is limited to 16 bit 44100 , Madplay is supposed to be able to do more, I haven’t tried it yet. You’d think the output settings would have to be a reasonable match for the rate or the resolution of the mp3.

I thought you had already tried my player

Apologies. I can’t recall if I did or not! Afraid my memory isn’t good as I tend to forget about some things as I do others. :-/

Just had a quick look at your zip of flac2usb and you seem to use a newer build of flac than I have. The version I’ve been using is 1.2.1. Yours is 1.3.1. Did you build this? I’m wondering where it’s available from as it may be good for me to tell people to get the latest version that’s available. I’ve been directing them to

http://www.riscos.info/packages/AudioDetails.html

which only provides 1.2.1

Jim

which only provides 1.2.1

It is possible to alter the autobuilder to fetch from a URL instead of the debian repositories, Somebody would have to do so. Couldn’t you just include the one you got from me in your app’s subdirectory and make it easier all round?

Couldn’t you just include the one you got from me in your app’s subdirectory and make it easier all round?

I’d find that convenient, and I think users would also. I did initially think of including the 1.2.1 absolute executable file, but wondered if that meant I needed to also include all the source files, etc. Not sure what is permitted.

However if its OK, I’ll add the 1.3.1 as an item in the ‘Extras’ subdirectory of my program for a future release. If nothing else it bypasses the problem of having to get it from !Flac and deal with the aliasing that causes. :-)

Jim

The problem with the likes of the GPL (and I think the EUPL as well) is that the user must be able to access a copy of the source to build a copy of the supplied executable. It is good in practice to point people to the official repository if you are supplying an unmodified executable, it saves you from the issue of hosting the sources yourself, but where this falls down is when the official version is updated – at that point technically the executable you are supplying and the source tarball for it will no longer match, until you include the newer executable within your software.

So – this leaves two options. Point to the official and just shrug if the two come out of sync or host a copy of the exact source to help the world fill up with outdated source codes. Between you and me, I would point at the original source code. It isn’t your problem if the user wanting the exact source needs to play with SVN/CVS/etc to get at it… ;-)

I’d supply the sources to your compiled code with your distribution – its a much less complicated option. Then when people ask you for the sources you can say I already gave you them and they have no comeback. If instead you use a thirdparty site for the sources and they remove them for whatever reason you are still responsible for supplying the sources for the version of the code you distributed.

The above shows why I ducked this and just gave a URL for where I’d found a RO version of flac! The 1.2.1 version does seem to work OK, but it would probably be desirable for people to have access to later versions if they can.

Since I’m currently not supplying flac I can avoid being nominally responsible for providing sources on request. My program uses flac, not rebuilds or modifies it. But the situation is clearly a PITA for users who don’t already have utilities like flac, sox, etc, ready to call. The riscos.info pages are useful, but lack the ability of an apt-get or synaptic to automate dependencies, etc.

I don’t know how riscos.info get their packages updated. Can we get them to update to the version Ronald has? Seems a pity if they drift out of date if that can be avoided.

Jim

So – this leaves two options. Point to the official and just shrug if the two come out of sync

Perhaps avoiding github links and including the link to the tarballs such as http://downloads.xiph.org/releases/flac/ would meet your requirement. I think someone who is about to compile the source probably doesn’t need too much hand-holding.
Millions of people are installing flac in Linux and if my machines are an indication, the only source code I can find in root are the kernel headers, which I had to install manually for a tv card.
As you say, modified source code is a different kettle of fish. The autobuilder is quite helpful in that case, you can view patches for changes and pick up quickly on previous work.
For Norcroft stuff it has been normal to include the source code, I guess. There are diff and patch utilities.