Small AcornHTTP modification wanted

HTTP is used meanwhile with a couple of methods which are not defined inside the classical RFC. AcornHTTP is only supporting the classical methods. However I am thinking that it can be used for the other methods too without any big changes.

My intension is to modify the http_message_text function in the default section in such a way, that it is looking inside a file eg. Choices:WWW.AcornHTTP.UsrMthds for a user defined mapping.

The fetcher specification says that methods in the range 128-254 are free to be used by the user.

Of course I have to try whether this will work or whether other sections of AcornHTTP must be also changed.

Any objections ot my idea?

I’ve used AcornHTTP for a couple of very small projects. It doesn’t make me an expert – far from it – but it has made me realise that AcornHTTP is well out of date. For example, it only supports HTTP 1.0, while the rest of the world supports at least 1.1.

So I think it would be useful to see if there is any commonality between what you want to achieve, and what the rest of our community might like to use.

What methods are you proposing to add?

While you’re at it, don’t forget to add support for HTTP/2, HTTP/3 and QUIC.

Would any of this work need linking into the new TCP/IP stack work being done by ROD? Would it potentially duplicate any work already being done?

while the rest of the world supports at least 1.1.

I don’t think for connect-fetch-disconnect, it makes that much difference. There would possibly need to be an API change in order to handle keep-alive.

What I’d like to see, that shouldn’t be too hard to do, is for any non 200 OK status codes, to have the header and whatever the server returns presented as the payload.

Why? Mostly because 301 and 302 redirections. The current method of handling those is a joke.

• Fetch http://site.whatever/meh
• Server returns a 302 redirect. The URL module closes the connection with status 302.
• Fetch http://site.whatever/meh using the special incantation to return the headers.
• Server returns a 302 redirect. The URL module lcoses the connection with status 302, but at least this time you get something to look at.
• Parse the headers to discover the location is now https://site.whatever/meh
• Fetch https://site.whatever/meh
• Server returns 200 OK. The URL module does its thing.

There shouldn’t be any compatibility issues as a buffer was provided, but no data was expected upon a non-200 return, so existing software ought to just ignore the new payload.
Newer software, however, could pick up the new location from the payload and miss out the second fetch, which is ridiculous to have to do since the server just sent the necessary data.

Or, if there’s any doubt, use a flag. A flag that means “send me the headers and any data received upon an HTTP error condition”.

There is an argument that the URL fetcher ought to handle this for itself, but we run into the problem of how to report back to the user that the URL has changed, and what to do with circular loops.

Oh, and handling cookies on a per-site basis ought to be easier than “enumerate everything”.

Regarding new TCP/IP Stack:

I don’t have any information about it.
I hope that it provides backward compitibilty or we can throw away any existing RISC OS network software.
In such a case the new stack is a no use for myself.

In my opinion a stack has nothing to do with the applications based on it. HTTP/FTP etc. are parts of the application layer.

Regarding out of date:

Until now I was able to access any sources for MapView. I don’t now in detail whether there are important differences prohibiting access to sites.
I think improvements have been done in detail like keeping connections open etc.. So it may be not as efficient as it could be. But that is not the question here. I don’t have heard any arguments against my planned modifications.

Would any of this work need linking into the new TCP/IP stack work being done by ROD?

No. Different layers. HTTP is built on top of TCP, i.e. above the transport layer in the ISO layer model. QUIC is even based on UDP.

Would it potentially duplicate any work already being done?

All the fetchers duplicate other work in certain areas. Everyone could just use libcurl. BASIC developers can’t unfortunately.

I meenwhile modified AcornHTTP in the way described as follows

AcornHTTP is designed very flexibel but it is limited to the classical
HTTP Methods.

During the time a couple of other HTTP methods have been defined for various
purposes e.g. for WebDAV, CalDAV etc. which are more or minor popular.

It is a pitty that AcornHTTP can’t be used to handle them so I analyzed
the code and found that it can be extended quite simply.

I expanded AcornHTTP as follows to cope with additional HTTP methods.

Inside

Choices:WWW.AcornHTTP

a file called

UsrMthds

can be given which content if present will be loaded during the
initialization of AcornHTTP into an array of 256 entries.

It consists of lines of the following format

MethodNumber=[DecimalFlags,]Method

Flags are defined as follows

Bit
0-1 – Type
0 – Caller must setup everything
1 – Behave like PUT/POST
2 – Behave like GET
3 – Unused
2-31 reserved

MethodNumber is the number of the method passed to URL_GetURL.
Acorn has defined that number 128-255 can be used by the user for its own
purposes.

During start of an URL fetch. Acorn HTTP tries to look up the text used
at beginning of the first line of the URL. This was hardcoded. If no
translation is found launching of the URL will fail. I added a look up
into the array here. If there is an entry inside the array for the method
the text given inside UsrMthds as “Method” will be used to launch the URL.
Additionally flags can be defined. The only usage in the moment is to emulate
internal AcornHTTP header behaviour of PUT/POST or GET for the additional
methods.

I am checking it with an application in the moment.
It seems to work. Now I need to know how to contribute it to ROOL (I took the source from the nightly build two weeks ago)

Regarding response header.

I must check how module behaves in detail. In genral there seems to be to possibility to get header and body.

However after 2018 there seemed to be a change changing the behaviour in conjunction with headers which eg. caused HomeCntrl to stop working with newer version of HTTPServ. I cured this meanhwile by explicitely ORing &200 in methods in the fetcher program of HomeCntrl

However after 2018 there seemed to be a change changing the behaviour in conjunction with headers

Yes, it was not following the spec.

Sadly, the spec is a little overly complicated in places. I used to fetch headers by ORing &200 with the HTTP GET request, which is like you mention.

Well, when I copy-pasted that code into ESP32Cam, I removed the bits that set up custom headers because it’s a simple server so the complexity isn’t necessary. And the HTTP requests failed. Because when you don’t have custom headers, it wants the &200 value in… R5 or something. It will choke if it’s merged in with the HTTP GET.

Consistency.

:-/

Yes thats with the consistency. During my actual project I also got confused.

When no Header is given the information must not be ORed with the method in R2 but must be put inside R5 but and much worse shifted by 8 Bit to the right.
So you don’t have to pass &200 but &2 in R5 instead.

Confusing indeed :-(

However after 2018 there seemed to be a change

The original discussion is here. I’ve linked to the first post in the thread, but most of the relevant stuff is on page 2.