D4R issues

I haven’t got a clue where riscos stores it’s equivalent of resolv.conf!

Inet$Resolvers?

It would be a bit better if the SWI definition was replaced with

I normally include “SWIs.h” in my projects, but since that’s a DDE thing I don’t think I should distribute it. If anybody is interested, it is lurking in one of the old examples. Perhaps CModule or something that interworks C and assembler. Though, it will need modernisation, I think the original is RO3.5 era.
It is exactly as you describe, a big list of SWIs.

These days you’re probably supposed to link into parts of the RISC OS build environment for that sort of thing, but that’s not a lot of good for build systems and methods set up prior to this being available (plus given that I develop on three different machines, I prefer to keep my projects fairly self-contained with no dependencies other than DeskLib and CLib).

RISC OS is a nightmare for things like this. Sockets etc. seem to be the worst for trashing things badly.
I managed to trash the NIC driver a few times when I was testing newsuk in the alpha versions.

Strange. I am writing a server, the back end is a module and the front end is an application loaded on demand. The sockets part hasn’t caused me too much difficulty.
I am using TCPIPLib and not doing anything unusual.

A base module isn’t my problem, debugging it is. (another abort on data transfer/filecore in use…)

Either something is going very badly wrong with your memory accesses, or you are running from a callback/IRQ and you are trying to do “complicated things” when RISC OS is threaded.
In my server module, I use a ticker to check the socket state, and if there is anything worth noting, I set up a callback and let RISC OS get back to me when it is “safe” to do things. I cannot do much on the Call[After|Every] ticker, for attempting to touch a file will report FileCore in use (it isn’t reentrant).
I don’t think many people quite realise the quirks weirdness limitations of CallAfter / CallEvery versus Callback.

As for debugging the code, there are two approaches. I’ve used both.

The first is to write, and check, that basic sockets (etc) code works as an app. If it does not, you can put tracing code in to switch to single task and “press a key to continue” from one part to the next. I would say “use the debugger” but it seems that the debugger is itself somewhat prone to dying horribly on newer hardware. Useful. ;-)

The second, for things that have to be designed and tested as module code, is to drop in as much trace information as is necessary to output to DADebug. I have a version of DADebug that spits output to the serial port. This information is then picked up by the PC running HyperTerm (via a 3.3V serial to USB adaptor). Even if the RISC OS machine stiffs, the debug info is still on my other screen so I can try to work out what went wrong. It is rather old tech (akin to PRINT blah% in days of old) but sometimes the old ways are the simplest to get a job done.

I’ll take a look and try and do some DNS work tonight.

I see you made a DNS client. How come you didn’t just call the Resolver module?

Debugging via a serial console isn’t my idea of fun ;-)
But you are quite right about writing and checking the code in userland, this is exactly why I’m writing D4R.
I noticed when writing newsuk, that the calls to resolver (from C) are blocking calls. That’s not really acceptable.
Also when I was at Wakefield last year I had a chat with one of the netsurf guys, he said they wrote their own because of deficiencies in the RISC OS one. Also Steve has had a good whinge about the Resolver module.
From what I remember it’s closed source as well, so there’s no chance of improving it.
So it maybe rewriting the wheel, but it sounds like it’s a square wheel that’s nearly falling off ;-)
Also it’s better than wasting time on pre-emption and threads. (I’ll stop there)
But thankfully that library does the trick and was almost no effort!
So the next step is to integrate it into my socket library.

I noticed when writing newsuk, that the calls to resolver (from C) are blocking calls. That’s not really acceptable.
Also when I was at Wakefield last year I had a chat with one of the netsurf guys, he said they wrote their own because of deficiencies in the RISC OS one. Also Steve has had a good whinge about the Resolver module.

I do harp on a bit, but it is a bit of an irritation that the networking aspect has languished so long.

Have a look at AntiSpam – it contains a partial implementation done by Frank d.B. because the Acorn version plain doesn’t work for reverse lookup as used by DNS blacklist. You could usefully swap notes with Frank.

I noticed when writing newsuk, that the calls to resolver (from C) are blocking calls. That’s not really acceptable.

Sure you’re calling the right SWI? The StrongHelp manual says Resolver_GetHostByName blocks, but Resolver_GetHost doesn’t, as the following bit of BASIC might show:

EINPROGRESS%=36
REPEAT
  SYS “Resolver_GetHost”,“mywebsite.com” TO status,hostent;flags
  error = (flags AND 1) = 1
  REM Perhaps inform user of the state of this lookup
UNTIL (error) OR (status != EINPROGRESS%)
REM hostent is a valid pointer, or NULL if no lookup

It’s used just like that by NetTime’s name lookup.

Thanks Sprow: That’s funny (in a sick twisted way). I was calling the C function gethostbyname – clearly I should have used the SWI – or gethost (if there is one)
I’ve never seen a double = in any statement before.
error = (flags AND 1) = 1
I can see what the flags AND 1 does, but the = 1 at the end has blown my mind.

Now I’ve gone down this road of pain, it seems a shame to turn around, as I do like tinkering with DNS stuff….

error = (flags AND 1) = 1

It means that the variable error is set TRUE or FALSE according to the truth or falsehood of whether (flags AND 1) is equal to 1.

Quite simple when you get your head around it. It’s just a shorthand way of doing:

IF (flags AND 1) = 1 THEN error = TRUE ELSE error = FALSE

neat!

The download speed for d4r is currently between 3 and 3.5MB/sec, pretty bad TBH. (on a pandaboard es)
I’m just seeing why it’s so bad – I did have downloads up to 5-6MB/sec at one point, but not with this current code base.
I’ve added a secondary rolling buffer to stop so many 17KB writes then do them once that fills, but it doesn’t seem to have helped.
It seems to run at the same speed in a task window or directly from the CLI – which is a good and bad thing.
Unless somethings changed in the ROM that affects it?
I was thinking maybe it’s the cache changes, I’ve noticed the “random” (but rare) corruptions in jpgs seem to have stopped in newsuk recently.
Better to be stable, than fast I suppose

I’ve updated it to v 0.03a with a secondary buffer and a “re-read”, which is a little strange – if I’ve just done a read, do a select immediately afterwards and see if we can read any more.
Surprisingly sometimes, though not often you can

I’ve found out my problem, for some reason the toolbox attach handler is being called three times per window.
The first and third time it’s assigning a random window id to it.
Only the second time is it actually being assigned a correct id, the others are invalid.
Most bizarre, the only thing I can think of is to have a global variable and only process the event_handlers when it’s called the second time. WTF?
This is just reinforcing the fact that toolbox is a complete waste of time, I’ve heard a lot of people say this.
I remember really weird behavior in rabber to do with the handlers, that I spent weeks working around because of weird behaviour.
Which was why I scrapped that implementation, there were so many work arounds in it became silly.
(i.e. please do this, nah, I’ll do it when I feel like it, not when you tell me)
What’s most bizarre I’ve compared the code to newsuk and it’s identical. Isn’t doing the same thing twice and expecting different results insanity?

I think it’s time to scrap it and manually create and manipulate windows using SWI’s.
For the moment, I’ll give up on wimp programming and focus on the CLI version, at least when you ask it to open a socket it does ;-)

I’ve found out my problem, for some reason the toolbox attach handler is being called three times per window.
The first and third time it’s assigning a random window id to it.

What do you mean by “attach handler”? Do you mean the event that fires when the window is created?

Do you perhaps need to check the window id matches the window id you’re expecting, and just drop out if it doesn’t?

The first and third time it’s assigning a random window id to it.
Only the second time is it actually being assigned a correct id, the others are invalid.

How do you define ‘correct id? There are a number of things to check. Firstly, what are the flag settings (auto create, shared) for the object in question? Secondly how are you creating and showing the object? You can run in to trouble if you both allow the toolbox to create objects (eg by entering their res template name into the definition of the ’calling’ object) and trying to manually create/show at the same time. It is possible to end up with more than one instance of the same object with different IDs without realising it.

This is just reinforcing the fact that toolbox is a complete waste of time, I’ve heard a lot of people say this.

I use the toolbox a lot, originally using Basic years ago, and now from C using oslib and oslibsupport. I cannot say I have found any real problems with it, although I did have a small headache recently with ColourDBox, due to something similar to that mentioned above.

I’ve read a few things about MPEG-DASH in the last few days, which prompts me to ask whether D4R would be useful towards a streaming player? Or do we already have any application that will play MPEG-DASH streaming audio?

Naive questions, both.

I’m also interested in the answers to your questions about mpeg-dash! Main reason being concerns about access to the BBC iplayer. Even if ‘real time’ rendering isn’t feasible, it would be very useful if we could save the stream to a file and process/use it later on!

Jim

Have you looked at the streams or any other information that tells us which codec Auntie is actually using?

From what I read on Wikipedia, MPEG-DASH is just a transport method for .mp4 stream, it is codec agnostic. In other words whatever codecs used by the file, so in the most usual cases: H264 for the video and mp3 or ACC for the audio.

@Chris, I attach a handler to the toolbox.
It detects the creation of windows, if the created window is called, say bob, I then store this in a windowid variable.
The window gets created three times, each time the window has a different ID.
I can update the icon on one of the three times when the window is created – the second time.
However if I use the same ID later in my code, it doesn’t work.
Neither do any of the other ID’s
However the ID of the window matches the second, as I checked that against an icon press in the window.
If I try and update an icon using that ID it only works directly after the window is created (for the second time).
It’s as mad as a box of frogs!
I wish I hadn’t debugged it to be honest, the results just freaked me out.

@Rick the “nightmare” I was talking about was if you leave sockets or file handles open, it screws things up badly.
IMHO this is a good thing, as otherwise you get 6-12 months down the line and get random effects, at which point you’re time to fixing it is going to be huge.

@Jim I’ll have a think about handling streams, it will definately handle using a memory block instead of a file further down the line, as that’s how I’m planning on expanding the code.

@Rick I’m having problems getting my head around claiming the event for the internet.
I’ve done all my code correctly, from what I can see.
However it seems that you have to release the event, nearly immediately.
Now that is kind of pointless, as I need to keep the event until my module quits.

I’m thinking that there is some voodoo going on here. (SVC or the like)
What I should be doing really is claiming & enabling the event only after I trigger something that will cause it.
i.e.
1) open socket
2) write headers
3) claim & enable internet event
4) read data when event is triggered
5) finished reading data and disable event and release it.

But for testing purposes I was hoping I could just enable it on the module initialise and release it on the final (when it quits).
It seems this is not the case.
What am I missing?

@Chris, I attach a handler to the toolbox. It detects the creation of windows, if the created window is called, say bob, I then store this in a windowid variable. The window gets created three times, each time the window has a different ID.

Do you mean you have a handler for Toolbox_ObjectAutoCreated? If not, what event ARE you handling which shows you the window getting created three times.

It does sound like something is amiss in your Res file, perhaps.

In a teenager tantrum I deleted the source.
There wasn’t much there, it would only take 5mins to recreate it.
When I get back to it I’ll delete the res file and start from scratch, as that makes the most sense. I did check the res file.
I would still like to drop using toolbox, but it is handy-if not a little frustrating at times. But at least it’s not tickle

As regards the module issue, I’m guessing the issue is that when the service memory call doesn’t clean up the memory – because I’m using events it causes an explosion.
I’m guessing this is due to the C module being userland, rather than kernel side (a linux metaphor sorry).
I seem to remember the C module compiler bit, decides whether the module is kernel or userland depending on what is used in the module.
So tonight I’m going to remove all “userland” code and see if it still explodes.
If it’s OK that way, I’ll have to write two modules: qfa and qfb.
QFA will deal with the socket events and read data, qfb will handle the queuing, data processing and memory allocation for larger buffers.

Thankfully I’ve narrowed this down, my event handler is OK. However I’ve got some global variables which are causing a violent fit.
So I’m assuming either global variables are not allowed at all – which would be a major bummer, or my char *buffer is blowing some memory limit in the module.
I’m hoping for the second!
Seems module coding could be good for my programming, you have to be far more careful – and the consequences are far more severe!

A mini update, though not much – but progress.
I’ve managed to get the internet event registered properly and not kill socketwatch!
Pasting in my entire user-land code into a module, perhaps was a wee bit optimistic. (Obviously I didn’t actually paste it in, but near enough)