e-Mail lost connections

However, different source equals different version.

Unless I’ve missed something, the problem here is that it isn’t a different source. All that has changed is the toolchain.

A released artifact should never get rebuilt and should definitely never have potentially different contents day-to-day.

To be fair here, the nightly odd-numbered builds are clearly marked as that. The problem isn’t helped by some companies cherry-picking some odd-numbered builds, doing some mystic arm-wavery and then declaring them “stable” releases for their own hardware.

And you couldn’t bump the RISC OS version numbers every night just in case something had changed somewhere. They best that you could probably do is change dates (still messy) or keep full build snapshots.

All internal version numbers should probably get a bump when transitioning 5.26 → 5.27 and then again from 5.27 → 5.28. [ETA, thinking about it, should they? If the source for the component is the same, why bump the version?]

If I let that kind of nonsense happen in my day job, I’d be searching for new employment pretty quickly.

I’m sure that, as a bunch of people doing this largely in their spare time (AIUI), ROOL would very much appreciate your expert input into improving the build process. :-)

Unless I’ve missed something, the problem here is that it isn’t a different source. All that has changed is the toolchain.

I’m not referring to this specific issue, I’m referring to this madness:

There were three AcornSSL v1.04 variants – for mbedTLS 2.16.0, 2.16.1 and 2.16.2
So far there have been three AcornSSL v1.05 variants – for mbedTLS 2.16.2 and 2.16.3 (dated 9 Sep 2019),

And you couldn’t bump the RISC OS version numbers every night just in case something had changed somewhere.

Certainly. And I have no issue with that as the behaviour is understood. However AcornSSL is released as a standalone component. In at least five incarnations bearing two different version numbers.

I’m not referring to this specific issue, I’m referring to this madness: […] There were three AcornSSL v1.04 variants – for mbedTLS 2.16.0, 2.16.1 and 2.16.2 […] So far there have been three AcornSSL v1.05 variants

I don’t disagree, but that wasn’t what everyone that you replied to was unhappy about in this thread. Tony and Stuart seemed to be arguing for every nightly build to be treated as a new version, which quickly becomes tricky without a lot of infrastructure that isn’t in the ROOL build system.

However AcornSSL is released as a standalone component.

Is it? I have to admit to not following too closely, but I can’t help thinking that AcornSSL should be handled as a standalone component completely outside of the odd- and even-numbered OS scheme. Ideally as a clearly labelled separate download from the ROOL downloads page, which can be installed on any OS version.

We should still get nightly builds to pick up things like the compiler breakage, but new – and identifiable – stable releases need to come out (and be obvious to a casual visitor to the site) as and when mbedTLS changes, and not just when the next even-numbered OS release rolls around.

I’ve not seen this, however. How does one get the latest AcornSSL at present? Is it a download of the disc image, or the ROM image?

Historically, RISC OS hasn’t really had the concept of “security updates”, but maybe it needs them now. :-)

+1 to the idea of “security updates”.

I understand ROOL were already looking at separating out the individual !Boot components (so that the Packages server could deliver them) and it would – I hope – be a relatively simple modification to have the Packages server offer a selection of downloads for each component. For example:

• Latest nightly build
• First build of current version
• Previous two numbered versions

I don’t disagree, but that wasn’t what everyone that you replied to was unhappy about in this thread.

My contribution was on the subject of version numbering of various items.
Taking up what I believe would be the Nemo stance¹ (and with 20/20 hindsight as to the effects of a toolchain change) I think everyone can agree that if the output of the build process is a different binary then the version number should change to reflect this.
In the case of the toolchain change it was perhaps optimistic of ROOL to expect that making changes there would not have any effect on the items built with it and it might have been better to run parallel builds with old and new toolchain comparing the two and running performance tests where the two differ.

In some respects the SSL component version numbering reflects this in that supporting elements have changed with differing performance as a result while the headline (user observable) version number has remained the same.

Secondary to that is the specific Nemo hobby horse that modules in RO5 that are far newer than ones in RO4.3x have version numbers that are lower thus you could have a ROOL sourced module with all the functionality of the ROL equivalent but with bug fixes and/or new features and yet it won’t easily replace the ROL module because the version number is lower.

¹ There was a bit of an insult fest that may have made him wander off for an indeterminate period.

I think we have forgotten the difference between stable releases and nightly builds.

Nightly builds are NOT to be considered stable. There are frequent code changes (though not necessarily frequent in each piece of code). If we bumped the version number of each piece of code with each new code change orbuild, I think we’d pretty soon find ourselves with silly version numbers. So I don’t think we should apply such a requirement.

Stable releases are a different metter. Where I worked before I retired, each project had a build machine that was ONLY used for release candidate builds. These machines were never buggered about with, and all updates to OS and build tools were made only after careful consideration. Incrementing the version number of generated code was automatic. Code from the build machines was the only code given over to the test department for them to spend their time on.

The only item under discussion here that doesn’t fall neatly into one or the other description is the AcornSSL module, simply because it has been the subject of several beta releases. The other tantalising aspect is the inclusion of code from elsewhere that contains its own version number. If we decide not to update our version number until a release, why should the inclusion of modified code from elsewhere be treated any differently from our own modified code?

One other thing. R-Comp, or anyone else, are perfectly entitled to take code at any stage between stable releases, do their own testing on it, and declare it as stable enough for the purposes of their own customers – because these companies take on the burden of support themselves. They take the code, and they take the consequences.

Tony and Stuart seemed to be arguing for every nightly build to be treated as a new version

No, I don’t agree with that. It would be… messy. What I do feel ought to happen is a rolling archive of the last seven builds (so will be at least a week’s worth), along with rolling checkpoint backups, say, every Saturday. Doesn’t need to be full source (that’s available from Git), just ROM images. So if something is spotted, people can o back and say “ah, it seems to have started around here”.
This is because the stable releases are infrequent – a year for the Pi, a year and a half for the SD image.

Is it?

Yes. It is available within !System, modules for RO350, in the harddisc4 collection. It isn’t available in the separate !System download (is this an oversight?).

I think everyone can agree that if the output of the build process is a different binary then the version number should change to reflect this.

If a different source created it… It’s possible that an improved compiler might offer better code generation (or, in this case, wonky code generation ;-) ), should that be a version bump too?

run parallel builds with old and new toolchain comparing the two and running performance tests where the two differ.

I’m surprised this wasn’t done as part of the real-world testing of the compiler changes.

In some respects the SSL component version numbering reflects this in that supporting elements have changed with differing performance as a result while the headline (user observable) version number has remained the same.

What are the actual differences in the various mbedTLS “incarnations” (there’s that milky word again!)? I sort of see this as “behind the scenes improvements to the security aspects of SSL”. The module must carry a different version number as what’s inside it is not the same, even though to the end-user it probably looks and feels like the same thing.

If we decide not to update our version number until a release, why should the inclusion of modified code from elsewhere be treated any differently from our own modified code?

That would be a viable argument if AcornSSL was built into the OS and was part of the collection of modules that comprise RISC OS.
It isn’t.
It’s a standalone module, released into the wild for people to use, and is indeed being used. Therefore, every new release must have a different version number so that they are unique and can be identified as such by RMEnsure, should it be necessary.

They take the code, and they take the consequences.

But what do they do about the version numbers if said things escape into the wild? Normally this wouldn’t be a big problem, but when you have “this one weird” module that doesn’t bump it’s number much, and multiple different versions, and then you throw a third party build into the mix. You can see it can get… difficult.

Added:

thinking about it, should they? If the source for the component is the same, why bump the version?

No, if it’s the same thing… I mean, Paint and Chars have had (or will have) version bumps as the code is currently being worked on and improved.
When’s the last time anybody did anything with !Draw or DisplayManager? ;-)

It’s possible that an improved compiler might offer better code generation (or, in this case, wonky code generation ;-) ), should that be a version bump too?

Lets see, was it a source change that changed the binary to something broken?
Ah, no. So my point stands – unless there are parallel builds:

I’m surprised this wasn’t done as part of the real-world testing of the compiler changes.

Me too. Fortunately my hindsight is way better than my regular version.
However, if in a similar situation at work I would be recommending the parallel build option.
We’ve got an authentication system replacement in the planning, guess who pushed the “side by side” install with a selective location by location migration of the wired auth and finishing on the wireless?

The last involves a site by site change affecting more people. Leave it until last.

If a different source created it… It’s possible that an improved compiler might offer better code generation (or, in this case, wonky code generation ;-) ), should that be a version bump too?

Yes. The same for “but only the help string changed” or “only a small bug was fixed” or other seemingly small change. It is different, so it needs a different version number. This is essential when tracking down differences.

Remember the SharedUnixLib fiasco? Or was it DigitalRenderer?

So on the current setup, binaries would need to be checked, and if different, the version number needs to be bumped up.

should that be a version bump too?

Yes, of course. Different binary, possibly different behaviour (as we’ve seen here). How could anyone even consider keeping the same version number? I find that totally baffling.

Meanwhile, at work, we wouldn’t bump the version number for a compiler change. That’s not due to policy, but rather because we wouldn’t release a new version unless the source had changed (technically we do rebuild everything daily, but the resulting builds are internal-only).

I can definitely see the benefit in rebuilding things (putting aside compiler issues, it also shows when changes to dependencies break other components, especially when you have automated testing in the mix), but this is where the concept of a “nightly ROM” (or, in that case, nightly HardDisc4) complicates things. It is, by definition, a nightly build, but then we have the problem of unchanged code looking like an unchanged module (in terms of version numbers), even though the module is actually different.

Now that I’ve typed all that, I’m not sure how useful any of it is, but I’m posting it anyway! :)

Well this raises an interesting question doesn’t it?

A change of compiler (or build options) should result in an executable that expresses the same behaviour (as described by the source code) even if the output is not binary identical.

But if the binary is not identical, it could produce different results, and so there should be a way of identifying there has been a change and what – so there should be a different version. Otherwise we will see similar amounts of time totally wasted as with this particular case.

I can’t really put it better than what Dave wrote above:

I think we have forgotten the difference between stable releases and nightly builds.

Nightly builds are NOT to be considered stable. There are frequent code changes (though not necessarily frequent in each piece of code). If we bumped the version number of each piece of code with each new code change or build, I think we’d pretty soon find ourselves with silly version numbers. So I don’t think we should apply such a requirement.

The problem that we have is that RISC OS internal version numbers date back to the 1980s, when the idea of releasing nightly ROMs was not even considered.

But if the binary is not identical, it could produce different results, and so there should be a way of identifying there has been a change and what – so there should be a different version. Otherwise we will see similar amounts of time totally wasted as with this particular case.

I agree, to a point, but it seems unworkable given what we have to work with. RISC OS isn’t built using a modern system, but using Acorn’s old tools running in RCPEmu which simply spit out a finished zip file of the ROM image (and a separate one of the disc image?) at the end. Any fixes for this either need to be implemented from scratch in the RISC OS build system, or we need to get the cross-compiler build system up and running so that modern solutions can be applied.

Neither seem to be particularly easy options.

I was going to ask what other OSs do, but how many other OSs are actually completely built every night? Linux and Windows are treated as a set of component parts, which are pushed into package managers when updates take place, I think¹. Therefore you can track their changes separately, and only build when things do change. It’s the “all or nothing” approach of our system which is causing the problem here.

¹ A change to MSPaint wouldn’t force a rebuild of their cryptography tools (you’d hope), but a change to !Paint would trigger a rebuild of the whole of RISC OS including AcornSSL.

Presumably a change to !Paint will not change the AcornSSL binary?
So it is not an AcornSSL version change – just perhaps a waste of computer time to rebuild it!

Tony and Stuart seemed to be arguing for every nightly build to be treated as a new version

Yes and no. In an ideal world, here’s how it would work:

• Nightly builds wouldn’t exist. They’re a waste of processor time.
• Component builds happen when the codebase changes. Dependencies are definied as fixed versions within the codebase, so a dependency update is a de-facto code change, which triggers a build.
• Component builds can also be manually triggered for things like toolchain and build environment changes. Or you can store your build config in a respository and monitor it – then when your compiler version changes, builds are triggered where required.
• ‘Bleeding edge’ unstable builds are specifically labelled as such – usually the version number has a ‘-SNAPSHOT’ and/or timestamp suffix, and are stored in a repository where only the latest build is retained (to a degree. It’s not that simple, but then again, it never is).
• Stable release builds are given a standard layout version number and uploaded to a repository where they are accessible as such and can never change.
• Binary-level dependencies are only ever pulled into a build from the repository – they are not re-built each time.
• SNAPSHOT builds may have SNAPSHOT dependencies. Release builds must use released dependency versions

Implement the above in a sensible manner and you get consistency and traceability with your built artifacts.

Unless I’ve missed something, the problem here is that it isn’t a different source. All that has changed is the toolchain.

Sorry, yes – my initial post was rather simplistic. Follow the guidelines above and this becomes a different version. Either timestamped SNAPSHOT, or new release.

I’m sure that, as a bunch of people doing this largely in their spare time (AIUI), ROOL would very much appreciate your expert input into improving the build process. :-)

If I remember rightly, RiscOS uses Jenkins for its builds? I’d actually happily take a look in that instance, given that’s my bread-and-butter :)

Meanwhile, at work, we wouldn’t bump the version number for a compiler change.

With us, it depends. Minor compiler version updates? We’d be the same as you. On the other hand, we had a sizeable project to move one application (and its sub-components) from a mix of Java 1.5, 1.6 and 1.7 to 1.8 – everything got a version bump for that, even the items that didn’t require a code change for it to work. Sometimes a version change for the sake of it provides useful metainfo in itself (and release notes).

Implement the above in a sensible manner

Please, feel free to. :-)

RiscOS uses Jenkins for its builds?

It uses Builder, some custom code that calls AMU (or whatever) in order to get stuff done. It’s all hosted on RISC OS itself and probably hasn’t changed much in several decades.

they are not re-built each time.

I think the reason for this is so that all the dependencies match up correctly. To give an example, there are two MessageTrans compressed dictionaries. A documented one, and the one the kernel uses.
The latter is not documented, because it is different from one OS release to the next. Indeed, it may even be different between Beagle, Pi, and Ti – depending on what heuristics are used to select the things to compress (I’m guessing word/phrase frequency). As such, it’s not quite as simple as it might seem to shake up the build process. But please, feel free to try. The only requirement… It must work on RISC OS (so those of us rolling our own can continue to do so).

Using the one test available here, BASIC HTTP_Client, then AcornSSL 1.05 (09 Sep 2019) mbedTLS 2.16.3, size 188348, in today’s, 2019-12-29, HD4 beta download looks to be fixed.

In case anyone wants to re-test secure TLS connections, there is now a 5th variant of AcornSSL v1.05 (09 Sep 2019) available in the latest HD4 beta download – this time with mbedTLS 2.16.4

Full marks to the developer & ROOL for enabling such rapid updates.
No marks for the static module version.

In case anyone wants to re-test secure TLS connections, there is now a 6th variant of AcornSSL v1.05 (09 Sep 2019) available in the latest HD4 beta download – this time with mbedTLS 2.16.5

Full marks to the developer & ROOL for enabling such rapid updates.
No marks for the static module version.

In case anyone wants to re-test secure TLS connections, there is now a 5th variant of AcornSSL v1.05 (09 Sep 2019) available in the latest HD4 beta download – this time with mbedTLS 2.16.4

On the remote chance that Nemo is quietly reading this¹ I can make his day/week/year by saying that I totally agree with him on the unbelievable insanity of people who persist in putting out new versions of the same module with exactly the same version number.
I mean, if they really want to avoid rapidly ascending version numbers why not just make the version number 0.00 and then everyone knows it’s total bollox and can ignore it.

¹ Which I doubt very much.

a 5th variant of AcornSSL v1.05 (09 Sep 2019) available in the latest HD4 beta download

At least the Verma info says which mbed version it is – but I can’t help but agree with you!

No marks for the static module version.

If there were 25 releases a year, then even after 32 years of incrementing the module version number we would only have reached 9.05, so it doesn’t look like there would be much of an issue changing the number.

If there were 25 releases a year, then even after 32 years of incrementing the module version number we would only have reached 9.05, so it doesn’t look like there would be much of an issue changing the number

You’re assuming that the system only allows for versions of the form n.nn and in another thread some while back Nemo demonstrated that this was not true..

Thus the range to play with is more extensive and your comment has even more validity.

I don’t know the complexities behind the code, but is there a way to break it into the base module and a separate support module that contains the ever changing part? The base module number only changes when that code changes and the support module gets its own version numbering. Just theorising, I don’t know if this is a practical approach.