SecureSockets Bounty

I wasn’t saying Jeffrey’s code was bad, quite the reverse.
IPV6? Ah the thing that sounded really cool in 2000, then no-one bothered and everyone disabled?
Shudder!
My definition of old for RISC OS would be Iyonix and older, although I’m more thinking RISC PC etc. A lot of things will run on a RISC PC, but I tested NewsUK on the RISC PC and it was painful. I seem to remember over a minute to load and transform all the graphics, as against 10 seconds or so on a recent machine.
Most of the “interweb” stuff out there is graphics heavy, needs encryption, plus all the manipulation. If any apps are going to be developed for this, they need to perform very quickly – otherwise people will just turn on their PC – maybe not us “old school” people, but to attract new people.
I’m a big fan of Impression-X and Artworks, they are massive heavy apps that perform like running notepad on a fresh install of Windows.

That said running it on a RISC PC is a good way to optimise stuff – which is why I’ve got one.
I should drop it off at CJE for some maintanence – bit of a trek though.

That MINIX book is pure geek porn!

I think that was the basic principle behind how the x86 card worked, and there may be some ideas there regarding how to handle “other cores”.

In an “interesting” mix of C and assembler.
I think the assembler is mostly to avoid the use of OSLib IIRC.

IPV6? Ah the thing that sounded really cool in 2000, then no-one bothered and everyone disabled?

Sounds about right. My ISP actually supports it, my desktop supports it, my laptop supports it, my primary browser supports it… but I’m not using it because my router doesn’t like it. This is despite the marketing saying that it supports it!

I downloaded the SecureSockets source a couple of days ago, and I’ve had a brief look at some of it.

Wow, what a mess, and it’s not even clear to me which tools it needs. There are Perl scripts for some of the stuff. The notes that come with the source (from another David – not me!) suggest that the DDE is required, but the Makefiles fail immediately, using a syntax that I simply don’t recognise.

Anyway, looking at it from a high level…

Its API is based on the Socket API. There are a couple of nice things about sticking with the existing SecureSockets API: one is that existing RISC OS apps that use this API would still work with an updated module, and the other is that it’s a relatively easy step up for any code that uses the existing sockets API but wants to be updated to secure.

I have to admit very early on that secure sockets stuff is completely new and strange to me. That having been said, I can imagine a path forward, to get an updated SecureSockets module that uses mbedTLS. The question is whether enough time and expertise can be applied.

One key point is how to test any updated module.

The source contains server and client test code, but it appears to be not only single-tasking but blocking, which seems to restrict its usefulness.

Here’s my suggestions:

• document the SecureSockets API
• get server and client test beds going, multi-tasking and non-blocking
• link the existing object code and test it – if we can get this far, it means that each section of source code can be rebuilt and the result tested (at least briefly) to verify no obvious breakage
• write new makefiles so that everything builds
• remove deprecated crypto code
• assuming that work goes on in parallel to make a relocatable module out of mbedTLS, convert SecureSockets to call it.

That last point is only the vaguest of ideas. Feel free to shoot it down, preferably suggesting a better idea.

One other thing: the source I downloaded seems to be for SecureSockets 1.03 (09 Dec 2002) but the version I’m running is 1.04 (26 Nov 2005). I can link the object files and produce a 1.03 module but it’s much smaller than 1.04.

So we don’t even seem to have access to the latest source.

If you want the latest source, you could ask rcomp.
When I had a look at the source I found it confusing.
Not that that means alot, but “upgrading” it to a later version of openssl was something I discarded, that said if someone could…as I am a wanna-be programmer etc. (well actually a long lost programmer).
I’ve finished another brutal session of work, so all things going well I’ll get moving again in a couple of weeks (need to recover sleep first! Oh, and do a write up I’d promised that I would do that I haven’t had time to (tut tut!).
So just to let you know things will be moving again, hopefully.

Still working on the 1.03 source, I’ve compiled the single top level module code and the module header, and linked the code to produce the relocatable module file. Plenty of warnings but no errors. The original source linked against very out of date versions of inetlibzm, unixlibzm, etc. so I’ve updated the link script to pull in current versions from my Norcroft installation. Still links OK and produced far less warnings.

This is, of course, the easy stuff. Building libssl and libcrypto will be more of a challenge.

However, I really must see if I can get the test bed working. It’s no good building code if I’ve no idea whether it works or not.

On the topic of my vision of the path forward: the step of using mbedTLS to replace some of the SecureSockets code requires some thought, not to say more understanding of what’s involved than I have. As a minimum it would require that we understand at a high level how the present module works, what calls can be made to mbedTLS, and thus how we can combine the two. Any ideas are welcome!

I rate my chances of success, working alone, as slim to none; but I’d be happy to work with anyone.

That’s good work getting it to compile though.
Just to show my ignorance, how does securesockets work.
Does it just provide a “local” socket that you can connect to?

I contacted Rob Sprowson regarding the AcornSSL module, which is also long in the tooth. This is available in the ROOL source tree. It was written for !Browse, I think, and is used by the AcornHTTP module which is part of the suite of URL fetchers. I am interested in this module because we use the AcornHTTP fetcher in RiscOSM for operating some web APIs, but the secure equivalent (via AcornSSL) is not available as the cryptography is only 40-bit and out of date.

I think SecureSockets and AcornSSL perform similar functions, in that they are both about opening sockets, and don’t provide higher-level protocols like HTTPS as such. It would be good if we could end up with a single set of modules to update SecureSockets and provide an HTTPS fetcher for AcornHTTP: I quite like the AcornHTTP interface as it makes interacting with web services quite straightforward: you don’t have to talk HTTP yourself to operate it.

HTTP is a distinct protocol layer – I don’t think it belongs in the same module as SecureSockets or any later equivalent.

Tantalising. I’ve modified the test server and client so as to be non-blocking so that each will run in a task widow. The server accepts an incoming connection from the client, but nothing else happens. The client is stuck waiting for the SSL state to become 3, which it doesn’t. The server calls Secure_CTX_PrivateKey, ctx%, Cert$ TO success% and success% is returned as 0. This worries me – I’d expect something called success% to be non-0 for success; if 0 were a correct value I’d expect it to be called error%.

But this all points to the fact that there isn’t documentation, as far as I know, of the API of this module.

The client is stuck waiting for the SSL state to become 3, which it doesn’t.

Can the client connect to this site? Might help narrow things down.

This worries me – I’d expect something called success% to be non-0 for success; if 0 were a correct value I’d expect it to be called error%.

Certainly. The NULL-or-error is well known, but it’d be a first to see it called “success”. Would it be any help looking at the SWI handler?

My own recollection of the module code was looking at some of the files (and the makefile as noticed) and thinking “oh my god”.
It was just for the sake of interest anyway, I never seem to have time for doing much these days (got Big Eyes from the library, a week ago, I’m not sure when I’ll get to watch it).

Maybe we all ought to make a pledge here and now that if we make a system resource (such as a module), we’ll write an API spec to go with it. It doesn’t need to be distributed (if the module is not open) but it does need to exist.

Me. I’ll pledge that right now. It’s easy for me any way. My memory is crap so my modules usually have a file describing the API and a (more trustworthy) recap in the module code itself (either in the dispatch or each function, or both) of what is expected and what is returned. I tend to comment a lot as in the old days I didn’t comment at all (I wrote like a Beeb programmer) and came to regret it, especially those times it was quicker to rewrite than figure out WTF was going on.
Anyway, that’s not the point.
The point is we should pledge to document clearly.
I am. Who else?

Oh, and just as important – document data structures and magics.

Certainly. The NULL-or-error is well known, but it’d be a first to see it called “success”.

While looking for something else, I found this in the bmake source

/*


	
A boolean type is defined as an integer, not an enum. This allows a
	
boolean argument to be an expression that isn’t strictly 0 or 1 valued.

 /

typedef int Boolean;

#ifndef TRUE

#define TRUE	1

#endif / TRUE /

#ifndef FALSE

#define FALSE	0

#endif / FALSE /

/
	
Functions that must return a status can return a ReturnStatus to
	
indicate success or type of failure.

 /

typedef int  ReturnStatus;

/
	
The following statuses overlap with the first 2 generic statuses
	
defined in status.h:

 *
	
SUCCESS			There was no error.
	
FAILURE			There was a general error.

 */

#define	SUCCESS			0×00000000

#define	FAILURE			0×00000001

Edit: Mmm, notextile tags seem to do the same as code tags in this case. -re missing asterisks

This allows a boolean argument to be an expression that isn’t strictly 0 or 1 valued.

That sound you hear is my forehead hiring the desk.
Repeatedly.

isn’t strictly 0 or 1 valued.

TRUE, FALSE and maybe? That’s Wavy-Mave logic.

Quantum logic, AKA, ‘that depends’.

Back on topic…

Frank de Bruijn sent me some stuff that includes a brief description of the API – whereupon I realised that I already had it and had overlooked it. My apologies to everyone concerned.

Much more progress: the reason that success% was coming out as 0 (and it really is success: 1 for success, 0 for failure) was that the code was referring to a file’s leafname without a path. Much more trial and error, and I now have a testbed server that works in a task window, and a client that is nearly as far forward. So I expect to be able to make elementary transfers before long. (Presumably that’s put the kibosh on it.) So far, I get one page of a transfer, because I don’t know what conditions to use to end the non-blocking loop, so it’s ending after one page.

AFAIK 1.03 is the latest source. I believe back in the day I sent it to Tom Hughes, who compiled it against StubsG to produce 1.04. IIRC 1.01 was the 26bit version 1.03 the 32bit version and 1.04 the StubsG (26/32bit neutral) edition.

The StubsG version doesn’t work on RISC OS 3.7, so we tended to include older versions alongside for legacy machines.

IIRC the test stuff ignores certificates and rather assumes all is well with them.

All original compile stuff was done in Norcroft etc, except for one file in LibCrypto (I think) which wouldn’t compile under Norcroft. That was done under GCC and munged back into working with the rest of the stuff. At the time, GCC couldn’t do modules, so Norcroft was the only choice from a module perspective, so it had to link and produce final code.

AFAIK 1.03 is the latest source. I believe back in the day I sent it to Tom Hughes, who compiled it against StubsG to produce 1.04. IIRC 1.01 was the 26bit version 1.03 the 32bit version and 1.04 the StubsG (26/32bit neutral) edition.

Thanks for the info, Andrew.

IIRC the test stuff ignores certificates and rather assumes all is well with them.

Probably. I removed the top set of certificates and keys from the server/pem file because Firefox complained about it only being a 512 bit key. On trying again, Firefox told me that the certificate (that was now in use) expired in 2001. It would be kind of useful if the module checked that.

Since the server/pem file is dated 1999, which would be about right for the age of the source code, I presume all the certs must have long ago expired. How do I go about getting at least one set of certificates that is good enough to do development and testing with?

One other thing: after that last test, I tried to collect my email. The handshakes timed out. I presume that my testing had killed the module. I rebooted and email worked fine. This is still all with the current 1.04 distribution – I still haven’t tried any binary that I’ve built. I’ve been concentrating on the test bed.

To be honest, I think the test stuff was intended only to be used with itself (ie. not firefox). Since the certificates may not actually be checked, the test stuff shouldn’t really care, but I’m not 100% sure. Sadly this is a case that David wrote it nearly 20 years ago, so I’m not sure of the ins and outs – my checks tended to be high level “can I fetch https via this…?”.

The original remit was a module to make it easy to do secure transactions so that https could be fetched for WXL. We wanted something generic and easy for coders to use, so we settled on the idea of secure sockets, much in the way that normal socket code worked. This would allow SSL to be added with minimal code changes for existing http or other non-secure protocols.

If memory serves, David spent most of the time doing the SSLeay/CryptLib stuff, and that was the big challenge in terms of compiling etc. Then, once he’d done that, he wrote the RISC OS socket veneer to it in quite rapid time. I guess it is much more pleasant crafting your own code/API than trying to make large blocks of other peoples’ code work (not to mention getting to grips with it)!

For high level testing of the recompiled code, don’t use WebFetch (WXL) since for some reason the StubsG version just sits waiting forever (1.01/1.03 were fine). I’d try it with Hermes (POP/SMTP) or Messenger Pro (IMAP eg. Gmail).

I should add that I’m more than happy to offer my services for testing etc.

HTTP is a distinct protocol layer – I don’t think it belongs in the same module as SecureSockets or any later equivalent.

I agree. I wasn’t clear enough in saying that the AcornSSL module performs a very similar function to SecureSockets, in that it only handles communication at the socket level. To quote Rob:

It looks like AcornSSL took the approach of mirroring each of the functions
1:1 from socketlib, which would make sense since ultimately socketlib maps
1:1 to SWIs in the Internet module. The SecureSockets API looks like one level of abstraction higher up, it’s trying to bundle socket operations into Open/Read/Write/Close.

The AcornHTTP fetcher module handles the HTTP protocol and does so via the Internet module or via the AcornSSL module if that is available.

Quoting Rob again:

Ultimately they’re both built from SSLeay (later OpenSSL). Probably the most flexible is to keep SSLeay as a library, and AcornSSL as a
module, and SecureSockets either links with SSLeay or becomes a shim which
calls AcornSSL. I don’t know how deep the tentacles from SecureSockets into
SSLeay is, or at least not without spending a few hours digging.

I appreciate it would be easier to ignore AcornSSL though, as there is hardly anything using it.

Here’s a command within a Makefile, followed by the warnings it generates:

Link -rmf -o SSLMod @.API.o.code @.API.o.Header @.out.libssl @.out.libcrypto  TCPIPLibs:o.unixlibzm TCPIPLibs:o.socklibzm  TCPIPLibs:o.inetlibzm  C:o.Stubs
ARM Linker: (Warning) Attribute conflict between AREA ssl(C$$code) and image code.
ARM Linker: (attribute difference = {32bit}).
ARM Linker: (Warning) Attribute conflict within AREA C$$code
    (conflict first found with ssl(C$$code)).
ARM Linker: (attribute difference = {32bit}).
ARM Linker: finished,  2 informational, 2 warning and 0 error messages.

I’d appreciate a bit of help here. I assume it means that some of the code being linked is not 32-bit, but how do I find which bit?

Since there aren’t any actual errors, the module is generated, but when I try to run it on a BBxM, the test code (which works with the 1.04 current distribution module) crashes at one of two places, neither of which looks plausible with respect to the error messages.

The old library code (libcrypto and libssl) that I’m linking with has file dates back in 2000, so I assume it’s entirely possible that it isn’t 32-bit compatible, and, even if it is, it may not be BBxM compatible. I was just hoping to be able to change (i.e. recompile) only a little bit at a time and see when it breaks, but unfortunately it breaks immediately :-(