The problem with StubsG

From: https://www.riscosopen.org/forum/forums/1/topics/15080?page=1

might benefit from linking to StubsG rather than Stubs

[…]

Does StubsG support

Andrew is very pro-StubsG. He doesn’t realise that it is a flawed and broken system to a non-issue nearly two decades ago.

Non-issue? StubsG was designed for apparently oh-so-difficult task of keeping SharedCLibrary versions up to date, back in the days when there were numerous iterations of said library.

A good thing would have been for CLib to refuse to die once it had been softloaded, but alas it still allows itself to be replaced even when clients have jump tables pointing to it.

A sensible method of dealing with the CLib issue would have been for somebody to write a little “installer” program that would check the current version of CLib, and if the one that was desired is later, to install it into !System and then invite the user to reboot.

A much less sensible method of dealing with the issue is to hack together a kludge that will “just work with any old CLib” without hassling the user about it, by degrading everything to a lowest common denominator. This is asinine. This is StubsG.

So what’s wrong with StubsG?
Well, for a start it was “too hard” to sort out the assert() issue (it’s different in later CLibs), so it gets downgraded to the older simple assert. Not so useful if you wanted to make use of the newer features.
Speaking of newer features, there’s no way to get the extended C99 features working on older CLibs, so StubsG’s solution is to simply ignore them. Which means that StubsG is great if you don’t mind using the feature set of the Norcroft compiler circa Y2K. Everything degrades.

It’s also closed source, so it isn’t as if anybody can actually attempt to fix this nonsense.

The supplied “package” is also NOT 32 bit compatible. StubsG itself is, but RISC OS Ltd “helpfully” supplied their own versions of many libraries in 26 bit and 32 bit incarnations … why? At least the nasty horrid Castle way was to transition to 32 bit and build 26/32 neutral code so it’ll work on either platform, instead of having the programmer have to chose one or t’other.
Anyway, these libraries come with 32 bit versions right?
Wrong.
Tollbox’s renderlib and wimplib contain a non-amusing amount of 26 bit code in the 32 bit version. They’re still distributing this brokenness seventeen years later.

A non-issue?

Yup. Back “in the day”, a person using just one single program that required an updated (26/32) CLib would have preety much made StubsG’s entire raison d’être pointless as that user would have the better CLib running and could just use it.

Yes, there were some problems with programs blindly trying to load a new CLib if the one present wasn’t good enough. This could often be put down to lazy users not bothering to correctly read the instructions – the programs I’m aware of that required a newer CLib either specified it OR supplied it for the user to install.

This isn’t really an issue these days as CLib is incremented infrequently, is built into ROM for the newer machines anyway, and for older systems it is provided as a part of the !System update.
The latest is 6.03 (or something).

The most recent that any of my apps look for is Fireworkz which wants 5.56. One of my unreleased programs specifies 5.90 (so timezones work correctly).
It seems pretty much everybody else does the weirdness of RMEnsuring 5.17 and loading if not found, and then immediately RMEnsuring 5.34 and erroring if not found. I can only guess this was a bodge for one of the early Castle releases to check if the user is running a 26/32 CLib, but not to try loading anything if they are, but instead to simply error out of the version they have is too old. At any rate, it isn’t necessary to be exactly up to date, unless one needs some feature that was fixed or added in the recent release. In which case there’s a chance StubsG might be an impediment. You know, like with 64 bit stuff.

So, remind me again, what are the actual benefits of StubsG that outweigh the fundamental downsides?

I really had to check the date of this post to make sure I’m not replying to some ancient thread, bit no it is 2020!

StubsG is abhorrent and fundamentally broken, please let it die.

no it is 2020!

Yeah, this nonsense is still kicking around.

Turns out, doing some further research, there was an early version of Select that wouldn’t even let you softload a new CLib. Which is plenty ironic given how many new versions of CLib used to be supplied with applications (many came with their own mini !System directory) and on magazine cover mount discs. It’s one of the bits of the OS that has always been “evolving” (or being bugfixed, if you’re less generous) so as much as it is problematic to update to a newer CLib with one in-situ (because the stupid thing doesn’t refuse to die), it is surely far worse to rely upon horrible bodges to coax the user into a sense of blatantly false security with “it’ll just work, you don’t need to upgrade anything”.

please let it die.

+1^E99

Turns out, doing some further research, there was an early version of Select that wouldn’t even let you softload a new CLib. Which is plenty ironic given how many new versions of CLib used to be supplied with applications …

The softload blocking was really annoying, but the situation at the time was a war with competing CLib versions coming from both ROL and ROOL each with their own bug fixes and quirks. ROOL had it easy as ROL’s 26-bit versions would not be accepted by the system. On the other hand ROL was certainly receiving issues related to programs using the 32-bit versions, so I am not really surprised that they’d want to block softloads and incite developers to use StubG instead.

The question of whether to use StubsG has come up again.

I’m updating NewsHound, which AFAICS is using C just like from K&R, so no new language features. I’ve built it with the most recent DDE and libraries.

The RISC OS user base is much wider than we would like to admit, and involves some very old platforms, many of which are 26-bit (even Virtual RiscPC, which is a lot of users, is 26 bit), and some platforms are even older and not updated. So, having set the scene:

Does StubsG give any advantage over Stubs?

Please, factual replies only, including justifications. If all you want to say is something like “it’s broken, don’t use it” and aren’t prepared to show why, then please don’t take part.

I hope we can all extract a once-and-for-all answer to this question.

Does StubsG give any advantage over Stubs?

Isn’t that question mainly a case of needing to define your use case?

I hope we can all extract a once-and-for-all answer to this question.

As with many such situations there is no single answer, unless you tighten the parameters in the question. So, to remove a bit of ambiguity – do you envisage the application in use on a system that cannot load the current sharedC?

Does StubsG give any advantage over Stubs?

Why StubsG is good – you can link with StubsG and it’ll work on pretty much any version of CLib out there.

Why StubsG is bad – it does so by denigrating the library linkage to the lowest common denominator. You might not think that you need any of the “new” features in more recent libraries. Your code might disagree – do you ever use assert, for instance?

The problem – keeping CLib up to date, coupled with the fact that a softloaded CLib stupidly doesn’t refuse to die, the machine blows up shortly after something tries to RMLoad a later version.

The wrong solution – to use a bodge to allow people to keep using an outdated library? Surely that would make it more likely for something to detect an old CLib and try softloading a newer one?

The correct solution – we have tools such as SysMerge. Ask the user to update and reboot. It’s not a hardship.

The raw painful truth – if a user is so tied to a twenty or thirty year old C library that they refuse to update to something from this millennium, they don’t deserve to run your software. Honestly, StubsG is an Elastoplast to make people feel good about not updating. They’ve probably already ruled out the use of any 26/32 bit neutral programs (anything that calls the newer APCS32 API), so you shouldn’t bastardise your program to cater for this.
And if they want to use something that relies upon a newer (as in “within the last decade”) version of the library? Well, then they’ll have to have installed a more recent library and, well, there goes StubsG’s attempt at a use case right out the window.

Another hard truth: my software, built with a recent or latest DDE is quite happy running under emulation on the likes of 3.70. How do you think I tested my Virus game? It was built on the Pi, punted over to the EeePC, and run under emulation on an Acorn era version of the OS. It works fine provided a suitable CLib is present.

What to do: just RMEnsure a recentish (5.23?) version of CLib but don’t RMLoad it, we know that blows up. Supply the latest you have (6.xx?) and SysMerge. Isn’t this in the EndUser directory? I forget. Then it’s the user’s problem, not yours.

StubsG solves what really ought to have been a non issue. But then, for a while a certain release of the other branch refused to softload the 32 bit CLib, so I’m suspecting this non issue is more political than technical.

Please don’t make your software broken by design.

Oh, and “mind the gap” – that being the 32 bit libraries helpfully provided with the StubsG bundle.
Don’t use them.
Seriously, do not.

Chuck them into ARMalyser and marvel at the amount of 26 bit only code sequences in a 32 bit library. Proof to the adage that doing something in a mediocre way can be worse than not doing anything at all.

Does StubsG give any advantage over Stubs?

I can’t find a reason in 2021 to use StubG (even on older computers). Probably best to ask users to update to latest SharedCLib and load it at boot time.

The only argument for keeping an old Archimedes with all 26 bit software only is trying to squeeze performance out of it, but on this regard an ARM3 and the 12Mhz MEMC1a are probably a better investment.

just my opinion, hope it helps.

Several points that people might not realise/consider…

1) There are (or at least, were) material differences in implementation in CLib 26bit and 32bit. The reason I first started using StubsG 20 years ago was because I found that a key part of HTMLEdit (at the time, a significant part of my livelihood) behaved tangibly differently on 26bit vs 32bit Clib. This was the syntax checking binary. It worked fine on a 26bit Clib, but something in the 32bit Clib caused line endings to be treated differently, resulting in dozens of errors being reported in the HTML. Ultimately the checker was updated by its author to support both behaviours, but nevertheless, this was quite a scary moment (I didn’t have source for the checker component).

I subsequently discussed this oddity with others and it was admitted that yes, there were implementational changes in 32bit Clib.

How important is this? Who knows. Probably not important at all. But my take-away from this was/is that if one program is affected like this, then others could be too. I felt it was rather unlikley that my main income program just happened to use the “only” binary in existance affected by this. As such, was it ethically right to force the choice of Clib onto users knowing this, if my code didn’t need it?

2) AFAIK products such as Virtual RiscPC (RISC OS 4 versions) still ship with 26bit Clib out of the box. Certainly I recall running into issues in the last couple of years. There are support issues here which might not be immediately obvious.

3) I know Rick was being tongue in cheek, but can you imagine if someone rang up to run one of our commercial apps on RISC OS 4 VRPC (or whatever) and I told them “you don’t deserve to run our software” (that you just paid for)! What I’m gently saying is that what you can get away with in the free software space differs somewhat from what you can get away with in the commercial space.

Just to be clear, as Rick and othesr point out, there are absolutely times when StubsG is not the right tool for the job (notably assert or C99+ features). I suspect even its author would agree. From my own perspective, I don’t use those language features so it has never been a problem for me, but equally I expect I’m in a minority there. Also, I wouldn’t go StubsG for a RISC OS 5-only application because there’d be no reason to.

There seem to be a couple of factual points:

1) The reason to use StubsG is that it, and only it, are compatible with the earliest versions of the Shared C Library that are still out there. Is that correct? (And you can get away with using StubsG if you don’t use assert.)

2) The problem that affected HTMLEdit was with the Shared C Library itself, not Stubs or StubsG. Is that right?

And if the Shared C Library itself caused such problems, was that a temporary thing, or a permanent change to fix something that was always wrong; or what?

A change in the Shared C Library more recently caused another problem – I forget what it was, but I’m sure Rick knows. But what apps were affected? Would they run on machines that have the very early versions of Shared C Library that only StubsG support?

The reason to use StubsG is that it, and only it, are compatible with the earliest versions of the Shared C Library that are still out there. Is that correct?

Probably.

Though, given that it only takes one application to be linked with “32-bit” stubs in order to force the user to install a “32-bit” SCL, didn’t the StubsG ship sail before it was even released? Anyone who needs NewsHound to be linked to StubsG will find that a lot of other software won’t work on their system either.

And you can get away with using StubsG if you don’t use assert.

Or C99 stuff. And on that, is anyone still writing code that uses the “non-n” versions of sprintf(), strcpy() and so on? And if so, given that it’s 2021, why?

The problem that affected HTMLEdit was with the Shared C Library itself, not Stubs or StubsG. Is that right?

The functionality is in the library, so that seems far more likely.

A change in the Shared C Library more recently caused another problem – I forget what it was, but I’m sure Rick knows.

There have been two “headine” breakages which have caused wailings and gnashings of teeth that I can think of:

• A fix to fscanf() which broke Writer. I think this was fixed in the ROL library first, then in the ROOL one, so perhaps the change at the ROL end was what Andrew saw when reading in HTML files (the two libraries would have been different from whenever ROL did their fix until 2015 or so).

• An issue with (IIRC) timezones and territories, which I’m sure that Rick will be happy to explain…

And you can get away with using StubsG if you don’t use assert.

Or C99 stuff.

Or C18 stuff.

It worked fine on a 26bit Clib, but something in the 32bit Clib caused line endings to be treated differently, resulting in dozens of errors being reported in the HTML.

I won’t comment without knowledge of the source to know what the problem actually was, but I rather suspect that this was a casualty of the infighting in that bugs were fixed in one version but not the other. So behaviour may differ. This shouldn’t have happened, but it’s how things are when you have two of the same thing being developed side by side without any correlation between the two.

I subsequently discussed this oddity with others and it was admitted that yes, there were implementational changes in 32bit Clib.

As there were in the 26 bit library – any bug fix can count as an implementational change if there is a difference in behaviour…and that’s generally why stuff gets fixed, to resolve cases when things don’t work as they’re supposed to.
So, admit it: yes, there were implementational changes in both versions.

As such, was it ethically right to force the choice of Clib onto users knowing this, if my code didn’t need it?

Yes. Because your code does need it.

I’ve yet to come across a program that didn’t use sprintf() to write a formatted string to a string buffer, or vsprintf() to write a formatted string with variable parameters (akin to printf()) to a string buffer.

These days, your software can be considered a risk if you aren’t using snprintf() and vsnprintf() to specify buffer size to avoid overruns. You are using these functions aren’t you?
Guess what. I know you aren’t. Because StubsG doesn’t provide them.

AFAIK products such as Virtual RiscPC (RISC OS 4 versions) still ship with 26bit Clib out of the box.

Colour me surprised…

I know Rick was being tongue in cheek,

Actually I was being dead serious. The ONLY reason that StubsG would be required is if somebody is intentionally running a 26 bit system and refusing to update, and expecting you to go along with that mentality.

As has been pointed out numerous times, your users (regardless of platform) will NEED to update their CLib in order to use any one of the many applications currently being developed that don’t use StubsG – which renders StubsG pointless – if they have a reasonably recent CLib (like, last five years?) then it’ll run all of these 26/32 neutral programs just fine.

I told them “you don’t deserve to run our software”

Sorry, appeals to emotion don’t tend to work with a person known for being “the emotionless”.

Let’s look it another way. You manage to find the one luddite that doesn’t want to touch anything from the 32 bit world or that expects to run on the 32 bit world, yet want to run your software and expect you to bend over backwards to support their stupidity ?

Just specify that they need XXX version of CLib like everything else. OvationPro? Fireworkz? NetSurf? OpenVector? PhotoDesk (JPEG handling)? BeebIt? They all expect a suitable version of CLib, not StubsG.

What I’m gently saying is that I’m trying hard to be the last StubsG cheerleader standing.

There, fixed that for you.

By enabling this, you are only ultimately causing yourself a bigger support nightmare (how many permutations are you making yourself support now?). It might seem like a simple “works with anything” solution, until something goes wrong.

Just to be clear, as Rick and othesr point out, there are absolutely times when StubsG is not the right tool for the job

There are no times in this day and age when StubsG is the right tool for the job. It is not HARD to have a user update a module and reboot. Hell, even the AmCog games come with a tool to merge in the RDSP module if you don’t have it.

Like, literally, it’s pretty much a one off job. There’s only one version of CLib being developed, and it doesn’t get developed that much. Hell, there are plenty of programs happy to ask for older versions (5.43 or somesuch?) because they’re more interested in it providing the 26/32 bit neutrality and basic extensions, they don’t need 64 bit support and so on.

my own perspective, I don’t use those language features so it has never been a problem for me,

Shame on you. :-)

You really should be using (v)snprintf.

Please, stop coming up with reasons to defend this and just ask your users to update (helpfully provide the module and sysmerge for them) like literally everybody else.

So much so, that I’d find it hard to believe that there are actually people who haven’t touched any of the software released in, like, the last decade and who therefore don’t already have a suitable version of CLib.

A fix to fscanf() which broke Writer.

My understanding was that fscanf() was broken and fixed at different times in the different versions of CLib, leading to a situation where the function may or may not exhibit broken behaviour.
It didn’t need to have been like that, but, well, we all know how that story went.

An issue with (IIRC) timezones and territories, which I’m sure that Rick will be happy to explain…

I’m probably the only person to have actually run into this, given that I’ve hacked together my own UK Territory that supports both UK and European timezones.
The problem was that the library calls for time handling were very obviously using the wrong SWI, which was giving the offset of the first timezone, rather than using the SWI to give the offset of the timezone actually in use.

My gnashing of teeth was over the complexities of a ridiculous set of regression changes necessary to correct an obvious problem. I understand the need for testing behaviour on a resource that is softloadable on older machines, but… jeez… timezone and territory handling is a broken mess on the older machines (and a marginally less broken mess in this day and age).

Why was it important? I don’t imagine Americans would be terribly happy if programs written in C kept telling them the time in Baltimore when they live in Portland.
[ditto any other country with multiple timezones]

Still, it got fixed. That’s one cockroach nailed to the wall, forty thousand to go. :-)

Several points that people might not realise/consider…

This had me pondering for a while.

Lessons learned:

1. Don’t try and fix what ain’t broke (GStubs “fix”)
2. If you do find something broken, report it, or it takes years for it to be discovered and fixed.

So, for the specific question in this thread – the current CLib works all the way back to RO3.7 and an OS earlier than that isn’t likely to have a working network stack that meets the current server requirements.

Some of the differences between ROL version and the Pace/ROOL one are mentionned on Gerph’s rambles

Yes, I read that a while back. Doesn’t mention that the helpfully provided 32 bit libraries are not 32 bit safe, or that it essentially works by downgrading the library behaviour to something compatible with the original library (really, only looking at this from one point of view).

He is right the you could use the Castle era compiler to create 32 bit software….that promptly failed on the Beagle because rotated loads and… I dimly recall something use to do with mucking around with R14 instead of CPSR? At any rate, rebuilding with a newer compiler sorted that out. ;-)

The main crux of the problem comes down to two things -

1. Everybody and their uncle’s cousin’s kitten does a CLib test by RMEnsure followed by RMLoad (because that’s how it was done in the ARM2 days…and there ain’t nobody that’s gonna say otherwise)
2. A softloaded version of CLib will happily allow itself to be replaced.

Point two is the main fault here. Given that the library works by patching in jumps to specific addresses, a softloaded CLib should have refused to exit. Just point blank refused.
But it doesn’t, machines where a softload is loaded on top of a softload go kabluey, and things like StubsG get written to attempt to make the best of a bad situation.

But time has moved on. The ROLtd version of the library is forever doing the time warp again, while the ROOL version (and C itself) is advancing to the point where retaining total compatibility with StubsG is an actual hindrance.
Plus, as has been said repeatedly, the use of any software that expects the newer CLib will mean that the user has to upgrade to a more capable version. In other words, these days I can only see mediocre excuses to justify the continued use of it as a solution.

And Andrew is wrong to associate this argument with somehow telling the user that they don’t “deserve” to run the software. What you are, and should, be telling them is that in order to run software from this decade ¹ you need a C library also from this decade.
Also, they have to understand that they are going to be more and more left behind, just like people such as myself have to understand that nobody writes stuff for XP any more.

By continuing with this solution for the hold-outs, you’re not doing the rest of us any favours.

¹ By “this decade”, I mostly mean the period of 2010 to 2019, since we’re only 13 months into this decade and it has already proven itself to be defective and not fit for purpose.

I will certainly not recommend StubsG for anything “modern”, but there are still use cases around where StubsG is “good enough” or has even advantages.

• depending on the compiler you use, newer CLib features might not be needed, because you cannot use them anyway – in my case, that is GNAT 3.03, which happily produces ARMv8 compatible code while linked to StubsG
• there are still applications around that real people use on real machines or RISC OS 4 emulators that are not happy when combined with any modern 32bit SharedCLibrary
• softloading a whole bunch of new CLib-and-friends modules might make it difficult to run any other software on memory-constraint machines (RISC OS 3.1, unexpanded Risc PCs or A7000s)

So, as always, it depends. Over the years, StubsG certainly got a lot less useful than it once was. If you use modern DDE and modern C features, you won’t have a choice, anyway.

Reading Gerph’s blog linked by Andre, it would seem he actually went to some effort to support C99 and other enhancements in StubsG, so it may be that even I am judging it too harshly. The assert function is also present, just not in its extended (and hence probably more useful) 32bit Clib form.

Has anyone hand-on-heart tested the things they’re saying don’t work? (C18 obviously won’t, as Sprow says).

I was just surprised to read that Gerph had clearly spent some time on implementing things, so if he says he added C99 functionality and re-implemented printf etc, do we actually have evidence that he didn’t?

Edit – I have just spent 15 mins doing some test code to check things. As Rick and Steve have said, the “n” C99 functions printf commands do not seem to be present in StubsG 0.04. assert() is also not particularly useful in its base form, and any conditional examples I tried just seemed to call a non-existent “assert2” function. Apologies for any time-wasting this post causes.

Has anyone hand-on-heart tested the things they’re saying don’t work?

If you can tell me how to use StubsG with GCC 4.7.4 then I’ll give it a whirl, although I won’t be using it for production code because – as I’ve already noted – that ship had already sailed back in 2003.

It’s worth noting that back in the day, I repeatedly asked ROL when support was going to be provided to developers for using StubsG with then then-current, native versions of GCC, and was repeatedly told “soon, we’re working on it”. I’m sure that you’ll be as surprised as I was to know that nothing ever materialised.

that ship had already sailed back in 2003.

StubsG = Norwegian Blue?

Has anyone hand-on-heart tested the things they’re saying don’t work? […] As Rick and Steve have said, the “n” C99 functions printf commands do not seem to be present in StubsG 0.04.

I can’t hand-on-heart say I have ever used StubsG (nor will I), but before I wrote what I wrote (and noting what Gerph said in his rambles), I did read the header files and noted the absence of the ‘n’-type string functions.

OK, well I figured that I would ask Gerph, since, well, who better?

Mystery solved. (Information provided for historical record)

Gerph did implement both the C99 routines and the assert() function (at least enough to work with examples I found online). He also appears to have it working with GCC (according to the changelog in October/November 2003). RISC OS Ltd appears to never have released his work after May 2003. I cannot explain why, but I’d left ROL by that point, so wasn’t privvy to any of that discussion.

The new version is somewhat larger than the old version, as it adds C99-compliant printf and derivatives, so that coupled with C18 are pracical reasons to still prefer Stubs32 (old StubsG was fractionally smaller than Stubs, newer one is larger). In-use RAM has gone up by 16KB. A smaller version without C99 (StubsGS) is also included.

The “current” version appears to be 0.18 by the changelog. The only version I found on RISC OS Ltd’s site was 0.04/0.05 vintage.

Again, for clarity, I am not attempting to convert the likes of Rick and Steve, merely to marry up the mismatch between the Rambles and the version for download on the RISC OS Ltd site. I think we can all agree that 0.18 sounds like a significant improvement, even if (as Steve F) says, “that ship has sailed”. The exercise has been fruitful, I think, however. Previously I had my own ways and functions for doing things similar to snprintf(), but Rick has shown me some useful C99 extensions that I would not have known otherwise. I don’t know if I will need to use them, but if I do, I need no longer fear compatibility with Clibs :)

I’ll speak to Gerph about the release situation, since even if no-one really uses them, I think having newer versions available is helpful.

The “current” version appears to be 0.18 by the changelog.

Which is a heck of a lot further developed than the “official” release!

Previously I had my own ways and functions for doing things similar to snprintf(),

Yeah… I think I’m going to write a “safe” version of strncpy. Such a monumental pain in the ass (and rather incomprehensible/bloody lazy) that the specification doesn’t require it to return a valid string. It just copies up to n characters and if there are more, it’ll return as many as fits with no null terminator. Stupid. Especially given that snprintf does provide proper terminated strings.
I mean, is it too much to expect the string handling library to handle valid strings?

but Rick has shown me some useful C99 extensions

You’re welcome. :-)

I need no longer fear compatibility with Clibs :)

For the moment…

I think having newer versions available is helpful.

Certainly. Even though I disagree with the use of StubsG in 202x, I disagree more with the use of outdated StubsG, period!

LOL Rick – exactly – when I read that description of strncpy back in the 90s, my immediate response was to write [for HTMLEdit] html_strncpy(). It’s crude, but does exactly what you say – ensures the resultant strings are null terminated.