Pragmatic suggestions for the zero page restriction incompatibilities

Okay, now step back. You’re still thinking with the mindset of a geek.

Me?

I don’t think I’ve been referred to in that way for,

ooh let me think,

must be nigh on…

…four hours.

Down to last couple of hundred network ports to migrate onto the new kit now. About 50-60 servers and the firewalls and it barely seems a blink since January. Is there a real world out there?

hat’s to do with Aemulor because of specifics of what’s there that Aemulor uses.
I thing ADDFS won’t run on low vector builds… for pretty much the same sort of reason.

IIRC Jon has developed ADFFS in recent versions to specifically not use features which would be likely to change in a multicore OS or steps on the way. Some changes in OS have been done after conversations about the inter-relationship.

Aemulor was always, as Adrian has said a clever hack intended only to cover the situation until the authors of the supported software fixed the errant item to work in the new world. I’d call it an extremely clever hack but long overdue for finding itself unnecessary.

did strike me as a plea to mark time, at the very least.

No it was a plea to give people the best performing riscos for machines available now. Most current riscos machines would be more compatible if there was a LV version of the latest sources available for these machines. For that loss of compatibility you gain what for your current computer well your guess is as good as mine. Yes run HV on a machine that needs it but I see no reason for people who don’t want to be testers to put up with it. I don’t say this for my own sake it’s no problem for me I use my own version of the roms anyway.

I don’t have the expertise to roll my own, unfortunately.

Do you have the DDE?

If so – https://www.heyrick.co.uk/blog/index.php?diary=20131105.

The only thing to note is that the ftpc.iconbar.com link for the UnTarBZ2 tool has gone, but I have that version in $.Utilities, so it might be a part of the official harddisc image now?
Load yours and see if it is version 1.06a2 (or later?).

did strike me as a plea to mark time, at the very least.

https://en.wikipedia.org/wiki/Hyperbole

This should have been a clue: works perfectly and multitasks brilliantly on 39 cores :-)

Is there a real world out there?

Got a news app on your phone? Read it. Then you’ll understand:

1. There is a world out there.

and:

2. You don’t want to be a part of it.

Got a news app on your phone?

My phone sits on the table at the side of the sofa (and a few other locations.

The work mobile is close by me and links to the work email system. It does have a browser which gets very infrequent use.

1. There is a world out there.

and:

2. You don’t want to be a part of it.

I had much the same feeling when observing things out of the pub window some years ago. Glad to see things haven’t changed radically. Then we worried about some Russian/American tits blowing us all to hell and wondering how many beers we could drink between being warned and the actual event. No change.

This should have been a clue: works perfectly and multitasks brilliantly on 39 cores :-)

Indeed, after all who would make a SoC with an odd multiple number of cores?

Oh, it has 48 cores. It’s just that it multitasks really badly on 9 of them.

In the original Ovation, there was a test that was something like if (array==NULL) || (array[element] == NULL). This test would pass if array was unset, as it would be NULL, but the conclusion of the line implies a read from the null pointer.

Are you sure? That code is a classic use of C’s ‘short circuit evaluation’: if array is NULL, C knows that the overall || must be true and so the second bit – the dereference – is skipped. This is defined behaviour and very common. The clearer form is the ubiquitous

if (ptr != NULL && *ptr != 0)

My recollection was that Ovation had quite a few tests of the form

if (*ptr != 0 && ptr != NULL)

This has the desired effect overall, but will always dereference ptr to get there.

sorry to be pedantic but the ovation code isn’t a bug

What was in Ovation was very definitely a bug.

R-Comp’s Messenger and other parts of their email software is a good example.

I find R-Comp’s attitude on Messenger quite depressing, to be honest.

Here’s what we need. We need ZeroPain to be baked directly into the RISC OS image. Not an add-on that may or may not vanish some day, but an official part of the ROM. We need it to have logging enabled or disabled. I really don’t care what the default is, so long as those who do not want it can turn it off.

This. Very much this.

However we also need everyone – including the two main commercial outfits – to start being honest about what Alignment Exceptions and Zero Pain bugs actually are. These aren’t two optional nasty things that have been introduced by the OS developers to break software, such that turning them off makes things OK again. They’re both tools to show that something is going wrong – turning them off simply hides the fact that something is continuing to go wrong quietly in the background.

Applications haven’t been “stomping all over Zero Page” for many years: all that HV builds are stopping is stuff reading from there. It’s also a fallacy to believe that Zero Pain events are “harmless”. Some are: the classic Ovation bug

if (*ptr == 0 || ptr == NULL)

is indeed harmless. However, the bugs that Zero Pain found in my own software have all been a bit more exciting. Something more akin to

free(ptr);
ptr = NULL;

/* A bit more stuff happens after the deallocation. */

data[ptr->index] = 0; /* Oops. */

where data is usually a flex pointer. I’m genuinely puzzled as to how the pre-fix versions of CashBook worked at all, but they could well account for the numerous “works for me” bug tickets that remain open on my system and run along the lines of “…and then my machine froze completely, losing all of my unsaved work.”

A Zero Pain log entry is RISC OS’s way of telling you that one of your applications has just behaved in a way that its developer probably wasn’t expecting, and that it might have now done several more undefined actions to undefined bits of memory as a result. Similarly an Alignment Exception is the system’s way to tell you that a program has just accessed memory using an ARM instruction that very likely no longer works in the way that it did when that software was written.

However much people want to pretend otherwise, neither are good things to have happening on your system.

Yes run HV on a machine that needs it but I see no reason for people who don’t want to be testers to put up with it.

The reality is that most of the OS developers are doing the work for fun in their spare time, as are many application developers. In that situation, while users do hold some sway (developers do need users, after all), there needs to be some give from their side. If development ceases to be fun, interesting and rewarding, developers move on to other things which are.

Also, the RISC OS market isn’t big enough to have dedicated alpha and beta testers. End users have no choice, if they want relatively stable software.

this may be a silly question but is it possible to build a rom that give you a choice on startup to be hi or low vector?? it is after all only a change to FPEmulator can we not have two a low vector and a high vector with a choice on startup?? or would you need two versions of the kernel?

However we also need everyone – including the two main commercial outfits – to start being honest about what Alignment Exceptions and Zero Pain bugs actually are. These aren’t two optional nasty things that have been introduced by the OS developers to break software, such that turning them off makes things OK again. They’re both tools to show that something is going wrong – turning them off simply hides the fact that something is continuing to go wrong quietly in the background.

Thanks Steve, nice phrasing of what I was trying to convey. Most especially the bold bit.

The concern is that without the ZPP and Zeropain module there was no means of identifying the fault.
As everyone seems to agree the module needs to be a fixture for continuing bug trapping.

What was in Ovation was very definitely a bug. 

but not in Rick’s example. In your other example of dereferencing a freed pointer had you not set it to NULL after freeing it you would not have had the exception and you could have caused equal havoc – how many of those have you got in your program.

Who on earth is saying that fixing bugs is a bad thing no-one in there right mind would, but banning programs that can work / may be needed is just an indulgence and quite frankly arrogance on the part of developers.

It would be nice if Zero Pain’s log – could be limited to the last few errors.

I changed the logs name – so it would appear at the head of the directory – so if an error was given it could be quickly spotted.

to start being honest about what Alignment Exceptions and Zero Pain bugs actually are

And what exactly are they. Intentional nonaligned reads and zero page access are not bugs so what is happening is that RISC OS has changed the API. If RO changed fileswitch to 64 bit without backward compatibility to 32bit it would break everything is that exceptable? If not why? Is it the scale of the incompatability?

 continuing to go wrong quietly in the background.

Show me a program of any size that isn’t.

Maybe a solution would be for ROOL to put a link in Downloads to, for example, http://www.svrsig.org/software/update4.zip, a LV rom produced by Chris Hall which runs on all the Pi versions

It will work on the Raspberry Pi Zero, Compute Module, model 1, 2 and 3 A, A+, B and B+. It will support AX8872B-based USB to Ethernet adapters

and is Aemulor-friendly. So those who wanted/needed maximum backward compatibility could use ‘update4’, on the understanding that it would not be part of any official OS development.

@Colin

Most current riscos machines would be more compatible if there was a LV version of the latest sources available for these machines. For that loss of compatibility you gain what for your current computer well your guess is as good as mine.

Whilst I agree a more recent update would be good that is why ROOL are making available development builds as they have had limited testing and exposure and therefore even if a LV version was made available it could still cause issues. They also have over the course of the last months enabled fixes to be put in place for a lot of software and that makes the next step easier. Though I wonder how many people have thanked those developers for making the effort?

What I agree is need is a good 5.24 supported release in perhaps LV and HV modes but then the development releases after that should be HV as to get to multi core, enhanced stability, functionality /future proofing needs the step. Somewhere we have to get past it worked and 3.1 is really good as it meets my needs and I don’t like Green/Blue folders type arguements as otherwise the few developers we have left will stop being interested and go and do something more fun.

I think Rick and David have suggested a good way forward with either a enhanced Zeropain module in the ROM or ability to patch but then we have also have as I said Emulators and if fixed Aemulor.

Forcing people to abandon software they use, however buggy, for the sake of some utopian future where the os works perfectly and multitasks brilliantly on 39 cores but has no software is… – I have to agree with an earlier post Suicidal.

No one is forcing anyone, There is a stable build of 5.22 or company supported versions by RComp/CJE/Ident or development builds. You don’t have to upgrade at this stage unless you wish to experiment, risk breakages using development versions or help the developers test their changes. Most people just want a stable build hence why we still have users on old OS and software even when official updates are available.

@George

Maybe a solution would be for ROOL to put a link in Downloads to, for example, http://www.svrsig.org/software/update4.zip, a LV rom produced by Chris Hall which runs on all the Pi versions

Whilst I agree that what Chris is doing is excellent it is not without its own issues as I am sure he would acknowledge. It would add one more support headache for developers and users a like who would assume it is an official sanctioned build and then potentially be driven away when it causes issues.

As I see RComp/CJE/Ident etc is the route for people to get supported versions that work with a known feature and program set and any other means including ROOL development versions are for those who want to experiment/assist ROOL in the development of the OS and not as a means to get added features/bleeding edge and supported versions until they become a supported stable build.

No one is forcing anyone, There is a stable build of 5.22 or company supported versions by RComp/CJE/Ident 

Of course they are. The only thing stable about 5.22 is calling it stable. In a world where RO lags far behind anything is an improvement sticking with an old version isn’t an option. Before I started playing with RO networking I found it virtually unusable which was the reason I got involved. my changes were added to RO but networking still isn’t perfect it’s an ongoing process. There’s a lot more going on with RO development than just the drive to a multicore OS. The fact that these companies make there own builds tells you that the ROOL distribution isn’t good enough – they are not willing to support it. It wouldn’t take any effort on ROOLs part to distribute a ‘Maximum compatibility’ version of their product.

No one is forcing anyone, There is a stable build of 5.22 or company supported versions by RComp/CJE/Ident

Of course they are. The only thing stable about 5.22 is calling it stable. In a world where RO lags far behind anything is an improvement sticking with an old version isn’t an option. Before I started playing with RO networking I found it virtually unusable which was the reason I got involved. my changes were added to RO but networking still isn’t perfect it’s an ongoing process.

I agree with you a 5.24 stable release is a good idea as I have already stated and I’d also like to thank you for the work that you have done on the networking side. On that subject of course as a commercial entity RComp/CJE/Ident etc have the right to update and have some exclusivity and if it means they get more sales of their version then all the better. Somewhere along the line there has to be a balance between “free” versions and paid for ones and as long as the two don’t go in diverged directions and we end up with multiple incompatible versions then it can only help the market.

ROOL have tried to drive some of this with bounties with varying degrees of success and there is a Networking one on the go. I have made donations in the past for bounties just as I have supported the commercial companies with various schemes and one thing I think we should all ensure is that there is no repeat of the ROL situation where developments are lost.

It almost sounds as if some ROOL/Developer/Commercial entity meet up is required to ensure things are worked on and in a way that doesn’t mean that things go down a dead end.

The fact that these companies make there own builds tells you that the ROOL distribution isn’t good enough – they are not willing to support it.

I do not agree. It just shows that these companies have other priorities than ROOL have. They think that their users are better served when following their own priorities.

This might well be true. But it is also true that they provide RO updates in a very delayed fashion, and nobody knows from which source they build their distribution. Which is close to a support nightmare for other developers, because it is unknown which bugfixes from the main RISC OS source have been integrated and which not. And which new issues have been introduced by picking and mixing stuff.

Maybe this will get better in the future now that the first parts of the iMX6 port have been merged into the open part of the RO code base.

Don’t get me wrong – I can understand that staying with a special OS build to keep Aemulor running because you know that a large amount of your user base depends on it might look like a good idea. But it has ts downsides. There is no “one size fits all” solution until we get the ultimate emulation-and-backwards-compatibility-layer baked into RISC OS, or alternatively all applications are instantaneously updated by their developers whenever they are broken by an OS change (free of charge of course). Both of which will never happen.

one thing I think we should all ensure is that there is no repeat of the ROL situation where developments are lost.

Why are we even discussing this aspect? Source is available, download a snapshot. Even if mythically the licence changes, one can’t retrospectively change your licence.

As for companies making their own ROMs – I do the same. I can apply some custom patches that aren’t part of the official ROM. Did you ever think maybe the company selling equipment does the same sort of thing? Take the Pi-Top as an example. I know nothing about it so could be speaking of of my backside, but… If the display is not FullHD, there might be a desire to change RISC OS to start up in a mode that is the same size as the display panel. That’s one of the things I changed in my ROM, the machine boots into a 1280×1024 mode. If something were to go horribly wrong to abort startup, I can at least read the text on the screen…
Now let’s assume the device has a non standard something. Keyboard, trackball, whatever. Would it make sense for the driver module to be loaded during boot, or for it to be baked into the ROM?
It’s stuff like this that gives companies the opportunity to create their own custom ROMs…

That doesn’t mean it is necessarily a good thing – a very interesting point has been raised above regarding the level of compatibility when testing issues ¹. What is the heritage of each custom build?
Ultimately it will come down to whether it creates more problems than it fixes, or fixes more problems than it creates. As custom company specific ROMs exist, I guess right now it’s on the winning side…

¹ This is a problem I run into with ROLtd releases. They aren’t free, they aren’t necessary enough to buy, hell they aren’t developed any more….. So my official stance is “If it works, great. If not, too bad.” I’m afraid I’d be inclined to lump custom ROM builds into the same category.

One thing that occurred to me when reading this thread, is to wonder whether the required hooks for Aemulor could be baked into the ROM in some way, since Aemulor is one of the stated reasons for sticking with a low vector build – in fact, an old low vector build. The OS is a moving target. At least identifying Aemulor’s targets might be a help.

Just a naive thought.

one thing I think we should all ensure is that there is no repeat of the ROL situation where developments are lost.

Why are we even discussing this aspect? Source is available, download a snapshot. Even if mythically the licence changes, one can’t retrospectively change your licence.

The reason being if a commercial entity gets frustrated with development or goes off on their own track then things could diverge as depending on the licience agreement there is no need to feed things back. Hopefully unlikely but shows that there needs to be a level of co-operation and mutual shared development roadmap so that what limited resources we have are best utilised.

Anyway as I said interesting idea of yours re – baked in ZeroPain and of Dave with the Aemulor hooks piece above.

is to wonder whether the required hooks for Aemulor could be baked into the ROM in some way

I think cooperation between ROOL and Adrian Lees on making Aemulor function is the way forward. Either that or cooperation between Richard Keefe and Adrian Lees to make Impression have Aemulor baked in so that it works on current ROMS. The second way would provide a fee-based method. Or just pool resources to get the 32-bitting of Impression finished?

All this talk about developers needing to fix the zero page bugs misses an important point – most of the affected software doesn’t have a developer or even source code to fix! RISC OS is not overflowing with applications so we can’t afford to have the standard/official build being one that breaks existing software.

Also, unless I’m mistaken, there’s no reason why we can’t more the vectors high for safety, but leave a zero page there so that reads and writes don’t cause the program to instantly bomb out. A win all round.