URL_Fetcher and AcornHTTP

You know those times when you think you’re seeing the impossible?

I’ve got a working BASIC prog to request the attributes from a printer. It works, whether plain (response code 426 Upgrade) or https (200 OK). I’ve also got the same code built into my PDumperIPP module in C as a SWI. This works in https (200 OK), but in plain, it sends nothing. The hard work is done by the URL (and AcornHTTP) modules in both cases. Same calls.

It doesn’t seem possible that just changing the URL string (‘cos that’s the only change, in both cases) could cause such a difference in operation.

I’m not asking for help; just venting my frustration.

just venting my frustration.

I feel your pain. Hacking the HTTP version to 1.1, should not have “broken everything”, certainly not the GET fetches that appear to do that already!

Grrr! Arrrgh! 🧟🔥

Now here’s the real kicker. I took the relevant source file and put a minimal required main() function with it so I can feed it a URL from the command line. Note that this is not a Relocatable Module, it’s just an ordinary app.

It works. https gives 200 OK; http gives 426 Upgrade.

I haven’t changed my code to achieve this.

This really has to be a bug – and not in my code, I think. When attempting a POST to the http URL, there’s no crash, it simply doesn’t send anything when the code is in a module, but the same code works fine as an app.

It will be interesting to find this one.

Perhaps Steffen was right after all.

Perhaps Steffen was right after all.

Shh! ;)

I’ve realised another detail: if the fetcher system is going to transfer in sections (let’s not call them chunks, for fear of confusion with Chunked Transfer), the call to GetURL needs to specify two lengths: the overall length as required for the Content-Length header, and the length of each section of data.

I haven’t found any docs for libcurl on RISC OS yet – though I admit to not having tried very hard – but does libcurl allow doing a POST in sections?

For IPP, even allowing a POST of a big file is inconvenient. The PWG-Raster file shouldn’t contain the IPP data block that forms a preamble to the POST. The PWG-Raster stuff is printer-independent given a format (e.g. srgb8) and resolution, whereas the preamble is very definitely dependent on the printer and its URL. It’s possible to concatenate both, but that means another file copy operation.

I’m hoping that, whatever method is used, it won’t cause the computer to single-task while transferring the data, because that could take a while.

I haven’t found any docs for libcurl on RISC OS yet – though I admit to not having tried very hard – but does libcurl allow doing a POST in sections?

See here: https://curl.se/libcurl/c/post-callback.html

I’ve just had another cursory look at AcornHTTP. I can see a way to change the API to send data in sections: two flag bits, one to say sectionalised transfer, the other to say if more data follow. If we’re transferring in sections, we need two lengths: the total length (as required by the Content-Length: header), and the length of this section. So let’s assume that R4 points to a two word structure. On the second and subsequent sections, the headers must not be re-sent; this can be determined by whether any data have been sent already, which is tracked in the session.

So, how does the transfer progress? I thought one would send a section and wait for status bit 2 to be set, then the next section can be sent. But I can’t see anywhere that that bit is set, or even referred to, so I can’t see how to hook in like that.

Have I missed something?

Which module is making the calls to the socket or AcornSSL SWIs? Is there another layer?

So, how does the transfer progress? I thought one would send a section and wait for status bit 2 to be set, then the next section can be sent. But I can’t see anywhere that that bit is set, or even referred to, so I can’t see how to hook in like that.

Are you talking about the status returned by the URL_Status/HTTP_Status SWIs? HTTP_Status is implemented in AcornHTTP’s status.c, using the flags/enum values from protocol.h.

Which module is making the calls to the socket or AcornSSL SWIs? Is there another layer?

AcornHTTP. security.c contains the socket layer code. One set of functions for raw sockets, another for AcornSSL.