ROD's new TCP/IP stack.

Apparently 24 is from OpenBSD and 28 from FreeBSD. Are we talking about the ROOL stack versus the ROD stack here? If so, we’re heading for a nice, new type of mess…

The values are irrelevant except on the system. That’s why they’re given symbolic names. They can be whatever you want them to be. What matters is that they’re used consistently within a given system. Two RISC OS implementations uses 24, so that is the value. If the header supplied is at odds with that, then… well, I guess that’s wrong because it won’t work.

Did changing from 28 to 24 fix the problem? Does that mean that ROD’s stack is using 24? The ROOL stack has used 28 since July 1999.

This is not a good situation…

Apparently 24 is from OpenBSD and 28 from FreeBSD.

A quick Google also shows 10 (some sort of Linux), 30 (Apple), 22 (HP/UX)… why on earth is it such a mess?

They can be whatever you want them to be.

On RISC OS the value must be consistent because BASIC doesn’t support the use of C headers; nor should it change on the same system depending on what stack is in use…

The ROOL stack has used 28 since July 1999.

The ROOL stack supported IPv6 for the past 23 years? Or just defined a value that nobody could ever use?

why on earth is it such a mess?

In the Unix world it doesn’t matter. Programs are expected to be recompiled for the target system, so they’ll pick up the correct values automatically.

nor should it change on the same system depending on what stack is in use…

And that’s the exception. When the same binaries are used, then the values need to match.

Or just defined a value that nobody could ever use?

Yeah, that.

Taken what FreeBSD uses, most likely.

The ROOL stack supported IPv6 for the past 23 years? Or just defined a value that nobody could ever use?

Pace asked me to work on IPv6 support in 2001, shortly before firing their Internet Appliances team (after which my emails suddenly stopped getting replies…). So not surprised that a constant has been lurking in a header file for that time, since the rest of the world supported it a long time before Pace started thinking about it.

That particular line was added to RISC OS sys/socket.h (from FreeBSD) 28 July 1999. It was added to FreeBSD 30 August 1996 and OpenBSD 12 March 1997. It seems like the divergence is that BSD 4.4 Lite had AF_SIP (Simple Internet Protocol) at 24 which FreeBSD imported, and OpenBSD later moved AF_INET6 there to match BSD/OS (a proprietary BSD fork by BSDi). SIP (64-bit IP addresses) was one of the alternative proposals for what became IPv6 in December 1995.

I have to say this is the reason why having two IP stacks from different sources is a big headache, because now you have to start modifying them to make a common API – you can’t just do a simple port and call it done, you need to have a heap of local modifications. That’s what multiplies the work that has to be done each time there’s an upstream update, rather than simply turning the handle and making a fresh build.

On the face of it, it’s bad; but let’s dig in and find out how much of a problem we really have.

My first question is, do we actually have two stacks? ROD’s stack exists today, and is open to everyone to get and use. Although the bounty stack is being worked on, does it actually exist anywhere? If not, in view of the availability of ROD’s stack, is there any point in persevering with the other?

If there’s only one stack, and no possibility of a second and different one, then the only problem we have is that the header files (and maybe the libraries – I don’t know) are wrong in some places and must be corrected. That’s not too big a deal.

If there really are two stacks, it gets a bit more complicated, of course. The first requirement is to be able to detect programmatically which stack is in use. The second is to be able to make the adjustments. It seems to me that that should be done at run time, not at compile time, because I’m sure it will be possible to swap stacks (and probably need to reboot), so recompilation would be uncomfortable.

As always, I’m happy to be told if I’ve got things wrong.

If there really are two stacks, it gets a bit more complicated, of course. The first requirement is to be able to detect programmatically which stack is in use. The second is to be able to make the adjustments. It seems to me that that should be done at run time, not at compile time, because I’m sure it will be possible to swap stacks (and probably need to reboot), so recompilation would be uncomfortable.

It’s messy because not only do you have to account for the constants (which are no longer constant so you can’t just #define them) but also structure layouts (if there is divergence there, which I haven’t checked between FreeBSD and OpenBSD, but not implausible). You can’t just declare a struct in C and start using it, you need to decide which stack you’re going to use and then fill in the fields as appropriate. Effectively you have to write your networking code twice. In BASIC it’s less painful because both constants and struct layout is determined at runtime (and so you can do all your stuff with msghdr%!msg_flags% where msg_flags% is conditional on which stack), but in C you have to declare your struct layout at compile time – you can’t do dynamic typing (you can in C++).

Unfortunately a lot of apps are speaking to the socket interface directly, they aren’t going via a middleware layer like EasySockets where this can be implemented once.

My first question is, do we actually have two stacks? ROD’s stack exists today, and is open to everyone to get and use. Although the bounty stack is being worked on, does it actually exist anywhere? If not, in view of the availability of ROD’s stack, is there any point in persevering with the other?

I’m not familiar with Internet 6 that Charles refers to – I assume that’s the RISC OS 6 stack, and I’m not sure what the status of IPv6 support is there, and whether it’s of any relevance today.

That aside, this cuts to the heart of the issues of pursuing two stacks. At the end of the day, RISC OS has mostly got away with ‘the ABI is a localised version of that of 4.3 BSD’, but now we end up with ‘the ABI is a localised version of OpenBSD’ and potentially ‘the ABI is a localised version of FreeBSD’ – and to resolve that we can’t just lift code from one or the other, we have to settle on a common ABI. Which makes more work for everyone.

OTOH it is hard to comment without having seen the internals of the ROD stack.

If there really are two stacks

Clearly work has begun on the ROOL stack else no announcement would have been made. The author is probably not likely to want to give up after working this far, and given the announcement it seems as if ROOL want their own stack.

The first requirement is to be able to detect programmatically which stack is in use.

Oh god help us…

How are we going to tell one from the other, except perhaps to try it one way and if it fails try it the other way (and hope that it does fail and doesn’t just do something different)?

We can’t rely upon module versions because it’s making hardcoded assumptions about versions of modules with the same name which could cause problems somewhere down the line, not to mention the issue of the wilful f***wittery around version numbers.

No, I am vehemently opposed to dumping the whole problem onto developers.

it gets a bit more complicated, of course.

The ROOL stack should be modified to use the same constant as the RODev stack. Simple.
With any luck it’ll be a simple twiddle in a header file and there will be few such disparities. Fingers crossed.

I should add that I’m not a shareholder in RODev, there’s no financial interest here. It’s a simple reflection of the fact that their stack exists and is available here and now. First Past The Post…

My first question is, do we actually have two stacks? ROD’s stack exists today, and is open to everyone to get and use. Although the bounty stack is being worked on, does it actually exist anywhere? If not, in view of the availability of ROD’s stack, is there any point in persevering with the other?

Your question is wrong, although the conclusion might not be.

There are, to my counting, 9¹ different and distinct Internet stacks. Three of which are effectively dead, one of which is alledgedly coming, the others of which are in use in real products. I am only counting stacks which are sourced differently, or have been updated independant of the others.

First the legacy and unlikely to be usable versions:

1. Internet 4 (DCI2). Status: Dead.
2. Freenet 1 (DCI2). Status: Dead.
3. Freeeet 2 (DCI4). Status: Usable on older hardware, but effectively not used these days.

Next developments of Internet 5:

4. Internet 5 for RISC OS 3.7/RISC OS 4.
5. Internet 5 for RISC OS Select.
6. Internet 5 for RISC OS 5.

(5 and 6 are derivatives of 4, but they’re distinct here because they contain independant implementations of features)

Now the newer versions of the stack:

7. Internet 6 from ROD.
8. ROOL’s Internet developments.
9. Internet for RISC OS Pyromaniac.

¹ Yes, I have been doing the Monty Python Spanish Inquisition sketch whilst counting these.

You should not, under most circumstances, need to detect which stack is in use. It is relatively rare to actually need to know this information in any case, and the API should make it clear.

To be more explicit… if you wrote software for the original Internet module (at least, the Internet 4 – I don’t know the history futher back than that), your software will work on every single version of the Internet stack due to backwards compatibility within the interfaces and implementations. This applies to most of the interfaces but not all – for example, routing formats have changed between Internet 4 and 5 – I don’t know how the routing records changed for Internet 6, and they’re not currently supported under RISC OS Pyromaniac. But in the majority of cases, software written with the COMPAT_INET4 setting (ie using the regular SWI calls, not the _1 variants) will work just fine with a modern version of the stack.

It should not be necessary to make adjustments in most cases as the Internet stack should be compatible with others on RISC OS.

As for the headers in the TCP/IP releases … those are pretty much irrelevant where the features haven’t been implemented. There is a lot in the TCP/IP headers that don’t actually exist, or do not function. That has been the case for quite some years (like about 30), so if the feature isn’t implemented, you cannot rely on what a given header says that the interface is because… well, it’s not really relevant. Documentation without implementation is worthless.

Here’s my take on this…

The first Internet stack to provide IPv6 support was RISC OS Pyromaniac, in 2019, from the diffs:

5837e72c8 (Charles Ferguson 2019-12-24 20:05:22 +0000  13) AF_INET6 = 28

Which would have been deployed into the build service at the start of April 2020.

At that time I chose to use AF_INET6 as 28 because that’s what FreeBSD chose. In lieu of any other direction, that’s the way that I went, and my tests reflected this.

Then the Internet 6 was being beta tested, and I reported that it was using a different constant than I was using. That’s the version they had from their OpenBSD sources. From the discussion it was unlikely to cause the values to change, and in any case, this is a constant value that is not going to be used by anyone, and having incompatible stacks was unlikely to be useful. The Internet 6 module was most likely to be used by more people, so RISC OS Pyromaniac changed its constant to 24. It’s only a number and all that matters is that the value is consistent on the system on which it is used. Nobody would have been using the constant 28 anyhow, because only the RISC OS build system and shell provided the implementation (and the couple of people that I’d supplied RISC OS Pyromaniac to).

The RISC OS Pyromaniac tests were updated, and then published together with my findings for the compatibility – they’re mostly compatible and there isn’t much that is too different. It Just Works.

So… that’s the situation. The only two distributed and available Internet stacks that support IPv6 use the constant 24 for AF_INET6. Therefore, that’s what it is. It is in use, and has been for quite some time.

I’m not familiar with Internet 6 that Charles refers to – I assume that’s the RISC OS 6 stack, and I’m not sure what the status of IPv6 support is there, and whether it’s of any relevance today.

Apologies… Where I wrote Internet 6, read Internet 7 – I was meaning the ROD Internet stack.
My brain got confused.

I think we should turn it up to 11.

We do not need another stack. We DO need the ROD stack in 5.30.

We do not need another stack.

Yes, we do. Clearly these two stacks are going to present compatibility issues, so we need a third stack that will be the one that everybody can use.

Okay, sorry if I made you spit out your tea. So here…

We DO need the ROD stack in 5.30.

This. It’s here. It exists. It works. It’s been in use by several of us already…

Slight aside, but on the topic of the number of stacks…

Whilst at University, Chris Johns¹ and I created a proxy module which made every single Internet ‘Socket_*’ call get passed to a remote machine for execution. The remote machine in question ran SunOS, and so our internet stack was using all the values and constants present on SunOS – without any translation. We ran IRC clients to chat to #acorn, and a browser using that system. Is that a 10th Internet stack? Or just utterly insane? Both, I feel.

But the point is that the Internet stack has been, and can be, very different, and stuff will still work. But it should really be as close as possible and in this case… it’s not a difficult fix.

¹ Sorry Chris, I typo’d your name!

If you’re going to include the ancient stuff… Termite, Freenet, and ka9q?

;)

Termite did not offer a BSD Sockets interface; it did its own thing, and ka9q… well, I don’t know… I thought it had its own interface but I honestly don’t know. In any case I don’t believe either offered the Sockets_* SWIs so their compatibility is irrelevant.

Whilst at University, Christ Johns and I created a proxy module which made every single Internet ‘Socket_*’ call get passed to a remote machine for execution. The remote machine in question ran SunOS, and so our internet stack was using all the values and constants present on SunOS – without any translation.

I did the same thing to pass on Socket (and Resolver) SWIs to Windows for the Red Squirrel emulator, in that case both constant values and order of some structure fields had to be changed, or things wouldn’t work at all. But all UNIX derived stacks are far more closely related than what went into the aberration that is Windows.

Incidentally this was before Virtual Acorn suddenly gained internet capabilities.

I have created a bug report both on this site and on the ROD’s stack bug tracker.
Hopefully, since it is already the second incompatibility issue, they will start talking to each other.

How are we going to tell one from the other, except perhaps to try it one way and if it fails try it the other way (and hope that it does fail and doesn’t just do something different)?

All it needs is a SWI that returns a stack version code. Like 1 for the ROD stack, 2 for the next different competitor…

All it needs is a SWI

Shifting the problem elsewhere. Bzzt! Wrong answer.

All it needs is to have the two parties get together and bash out an API (that will, due to differences, require one or both stacks to be tweaked) in order that both adhere to a common API thus enabling the same application code to work with either.

Feature detection is always the way to do things when you have an difference in implementation. If you don’t have feature detection you’re forced to use version numbers, but feature detection is always the right way to do it. But, as Rick says, that is ONLY in the case where it’s not possible for people to work to a common API.

Bzzt! Wrong answer.

It’s the answer to your question. Wrong question?

Charles is, of course, right in that a common API – or one stack only, even – is the best way forward.

I’ve been playing a bit with the ROD stack, trying to see how it works and seeing if I can build it. It’s… big, and the kind of mashup of *BSD code and RISC OS customisation that reminds me a lot of the USB stack.

I mostly got it to build. Makefiles are roughly aligned with the ROOL build system – will need a bit of polishing but ok. A glitch I found was the following need to be created otherwise parts of the build (build.BuildAll) fail:

|Ensure target directories are there
x cdir  <MyAll$Dir>.^.deploy.!Install.Boot.Resources.Configure
x cdir  <MyAll$Dir>.^.deploy.!Install.Boot.Resources.!OBSD.bin
x cdir  <MyAll$Dir>.^.deploy.!Install.Boot.Resources.!OBSD.rm

but that’s trivial.

However, the one component that wouldn’t build is SysCtl. The reason for that is that:

INCLUDES=      TCPIPLibs:,C:

which is pulling the ROOL header files, but SysCtl refers to a constant which comes from OpenBSD’s sys/sysctl.h:

#define CTLTYPE_QUAD    4       /* name describes a 64-bit number */

But in ROOL’s tree it’s defined as:

#define	CTLTYPE_S64	4	/* name describes a signed 64-bit number */

which has been renamed in a commit from 9 months ago about a merge from FreeBSD 12.3. CTLTYPE_QUAD was removed from FreeBSD in January 2011.

This is, in itself, a trivial change. But when I started trying to fix it by using the OpenBSD headers as well as the TCPIPLibs ones, things got very messy. The problem is that the existing code expects various things like unixlib.h, which aren’t in OpenBSD. But if I include both I get masses of type conflicts. That’s not surprising because there has been 30 years of divergence between the two codebases.

Where to go from here? Well, the fixes for this specific issue are simple. Aside from just hacking the header file, arguably sysctl is a tool that is strongly keyed to a specific ‘kernel’ and so using the FreeBSD sysctl with OpenBSD or vice versa doesn’t make sense, so the ‘right’ solution would be to just fix it up to use the OpenBSD sources.

However, I’m concerned this is the tip of the iceberg. We’ve already found two header file points of divergence and thus far nobody is actually building anything against the ROD headers. If TCPIPLibs are based on the FreeBSD tree and the Internet module is using the OpenBSD tree, something is likely to go wrong.

It is all very well to say ‘we need a common API’ but it’s counter to the philosophy that both ROOL and ROD are taking, being to lift code from *BSD and make it easy to keep up to date with the latest BSD sources. If there’s an API layer in between that needs to be maintained, now that’s more work for everyone.

As for adopting the ROD stack, well that’s beyond my pay grade. It looks like it’ll need some cleanup both in sources and in build system, but I think it possible that it could be merged. At present ROD are just issuing zipfiles so there’s no history – it would ideally want to land as a series of patches showing its development, rather than a huge lump.

One other thing is that there are no tests, which is something I would like to see for a piece of software this large and complex, although I accept it’s hard to do that on RISC OS. There’s no cross-build support, which would be one way to run tests, although I think pieces could also be unit tested in application space (or perhaps via Pyromaniac).

I wouldn’t treat sysctl as a particularly vital part of the system. It hasn’t retained the same interface between different Internet versions in the past, and it opens up links directly to internal structures as I recall. So in terms of compatibility, you shouldn’t expect that interface to work the same way between versions.

(by which I mean it is vital, but it’s not a user-facing interface and really it’s not an interface that you should be using except as part of the implementation of the stack itself)

Something else to point out is that GCCSDK defines AF_INET6 as 26 instead of 28 or 24, and since I can’t find any obvious signs of unsupported values being filtered out or remapped before calling the socket SWIs, it seems likely that there are already RISC OS applications out there calling the SWIs with GCCSDK values, especially if they use cross-platform libraries like curl for internet access.