C library for easier HTTP(S)

Unfortunately I don’t have my RISC OS machine handy at the moment, but if you’re OK with posting what you’ve tried so far then I’m happy to take a look at the code and try to see what’s wrong with it.

Yes, no problem (ignore the ugly code but it is easier to clean up when it’s working) and thank you in advance! I have uploaded the main file here and the whole sources here .

OK, there’s only one call to textarea_set_text, which is right after the call to getResponse. The problem is that getResponse will return immediately, before it’s actually downloaded the data, and therefore you’re only ever setting the initial text.

You’ll need to put a textarea_set_text call in handleResponse, as that’s the function that runs once the download has completed.

Ah, ok, I understand – that makes sense! I will try it that way! Thanks again!

Unfortunately I am stuck again, when the handle function is called the value of status is NULL – so something else does not seem to work.

I think that might be a bug in HTTPLib. It’s blindly passing through the output from URL_Status without checking it. I’ll take a look tonight if I have some free time.

Edit: Forgive me if this is a bit too obvious, but I see that your example !Run file isn’t RMEnsuring URL or AcornHTTP… are you actually loading the modules? :)

Thanks Chris, once you do it right it works! I was too focused on the code and also assumed that these modules are always loaded. But of course you need to load them in the !run file. Sorry!

One of these days I’m going to have to figure out a way of returning error codes so that it’s not so much of a black box when it goes wrong. One of the things I miss from C# is the exception system, which makes error handling trivial.

It’s me again ;-) In the meantime I was able to play around with the library more and have two follow-up questions:

The first question is how to set a http header on GET requests? It seems that on RISC OS you put the header before the actual data body you send, separating headers and body by a blank line (am I right that the format is: header1 + “\n” + header2 + “\n\n” + body ?). AFAIK GET requests should not contain a body but I think that they may require headers to be set. But because RISC OS handles headers as part of the body (or “data”) it would probably be neccessary to enhance the library in order to do that? I had a look on the source of the library and I think I know what needs to be changed but I think it is a useful addition to the library so I would suggest to include it? (Sorry Chris, don’t have any e-mail address of you to contact you directly). Headers that need to be send are e.g. authorization headers in order to use a webservice. Also the requierement for headers is not limited to GET, but also other verbs like DELETE I think.

The second question I have is probably not directly about the httplib, but I am not sure. Currently I can only issue http requests but no https requests – I get a status code of 0. I am using AcornHTTP 1.04 (20.4.2020) and URL_Fetcher 0.58 (12.5.2018). Is there something else I need to setup or update?

The first question is how to set a http header on GET requests?

As you alluded to, that might not be possible with the current library. I haven’t looked at the code for a while :)

I might have a look on the weekend, if I get a chance.

I am using AcornHTTP 1.04 (20.4.2020) and URL_Fetcher 0.58 (12.5.2018). Is there something else I need to setup or update?

You’ll also need AcornSSL (I suggest 1.04 or later).

Great, but no need to hurry. Thanks for the hint, https works now!

Well, that didn’t happen. Where does the time go?!

I have nothing scheduled tonight so we’ll see what happens…

Try this one. I make no promises (I haven’t had a chance to actually test it but it’ll hopefully work…)

There’s a new function called httplib_get_with_headers which takes a new char * containing the headers (I suspect that you’ll need to use \r\n as a separator if you want to include more than one).

I suspect that you’ll need to use \r\n as a separator

If I remember correctly, it didn’t make a difference of it was network or RISC OS terminated.

Does anybody know if there’s a debug option available in the URL fetcher that can dump the contents of the transfer (headers and payload)? It would help debugging greatly if I could see what was actually being sent, rather than WireSharking data sent to a fake server on the PC…

Great, thank you very much! I was testing it now by adding a header to a request and it worked – but in this case the header was not necessary – so I can only say that it didn’t break requests. But I think the implementation is similar to POST etc. so should work. I just don’t have a good example server now to test communication with but will have in future.

May I suggest to also implement a httplib_delete_with_headers function? If you communicate with webservers you often need to send headers for all http verbs (most often used: GET, POST, PUT, PATCH, DELETE).

Yep, that was just a ‘quick fix’. Once I can do some proper testing I’ll release a new version that handles custom headers with all verbs.

Edit: It doesn’t work, at least not in a way that Postman Echo is happy with. I’ll have a bit more of a play…

Edit 2: Actually, it does work. You need to terminate, not just delimit, the headers with \r\n. \n by itself does not work; you need the \r.

I’ve now released 0.09 in the usual place. It supports custom headers and body for all verbs. The API has changed a little bit; you’ll need to pass in NULL if you don’t care about custom header/body, and httplib_get_with_headers is gone again (use httplib_get instead).

Note that you do NOT need to terminate your headers with \r\n in this version (and it’ll probably break things if you do!)

Would adding a jump table at the beginning of the library code and loading and calling from BASIC work?

Would adding a jump table at the beginning of the library code and loading and calling from BASIC work?

No. You not only have to set up the C environment before calling into it ¹, you also have to have the C library jump tables set up and links resolved, because the code calls library functions like sprintf and _swix.

<sarcasm> Cue Steve to say that Basalt can do it… ;-) </sarcasm>

¹ https://www.riscosopen.org/wiki/documentation/show/Using%20C%20in%20assembler%20components

Thanks for version 0.9 – already using it!

Version 0.10 is now available and fixes the zero page access reported here. Anyone using the test version posted in that thread doesn’t need to download the new one as it’s identical.

Thanks for uploading the new version. Unfortunately I just found a request that leads to an error. Can you please try to send the following request:

GET https://api.deepl.com/v2/languages?auth_key=dgdfg345-ggh6h54g434-g5t554
header: Content-Type:application/x-www-form-urlencoded

I get the following error:

I just tried – it used to work with version 0.9 of your library. Sorry for reporting after you release 0.10 but I found it just a couple of minutes ago. I can’t exclude that the error is in the HTTPTest code and your library works fine.

BTW: The authorization code is invalid so the expected response would be 401 – unauthorized or 403 – forbidden.

That’s always the way…

I’ll take a look when I can.

Give this one a go. Note that it’s only had a cursory test so I don’t guarantee that it’s bug-free! I’m also not happy with how I’ve worked around the problem, so I might release another test version at some point.

Thanks Chris, a short test seemed to be ok. I have updated httptest as well now and uploaded it.