Important software compatibility notice

Starting from tomorrow, July 5th, the Iyonix, OMAP3, OMAP4 and Raspberry Pi development ROM images will be making use of “zero page relocation”, a change to the RISC OS memory map which moves the kernel’s “zero page” workspace away from address zero and up to the high end of the memory map. This greatly increases the system’s resilience to a common type of software bug known as a “null pointer dereference”. However, there is a catch: because the OS has always had memory mapped to address zero, large amounts of RISC OS software contains cases of “harmless” null pointer dereferences where the code reads from page zero and then does something insignificant with the result (usually ignoring it completely). With the new memory map, these programs will most likely cease to operate, exiting with a data abort as soon as they try to access page zero.

Since relocating zero page is very beneficial to the stability and security of the OS, the goal is to have the feature enabled for all future stable releases, starting with RISC OS 5.24. But due to the large amount of buggy software out there we can’t simply turn it on and be done with it – we need to have a transition period in which developers can fix their code without worrying about the fact their compilers, text editors, etc. are buggy too. We also need a way for regular users to get involved with the testing process.

With that in mind, we are putting into effect a two-phase testing process:

Phase 1 of the testing process begins tomorrow. Zero page relocation is enabled, but a compatibility/logging module (“ZeroPain”) is provided to you in order to allow most buggy software to continue to run unmodified. ZeroPain traps most attempts to read page zero and emulates the operation, providing a safe level of compatibility with the old memory map. And for any page zero access which it emulates, it adds an entry to a log file so that the user/developer is aware of the issue.

Phase 2 of the testing process is due to begin on 1st Jan, 2016. ZeroPain will refuse to run on any ROM built on or after that date. This will help to ensure that any previously unnoticed bugs are found and fixed prior to the release of RISC OS 5.24.

ZeroPain can be found within each ROM download archive. Make sure to install it before installing the new ROM, otherwise your system may not boot correctly. Please help ROOL make RISC OS a better OS by testing all your software and reporting any issues you find to the developers.

High processor vectors and IOMD support

Modern ARM CPUs support a feature known as ‘high processor vectors’, whereby the processor vectors are moved from their old location at &0 to a new location at &FFFF0000. Zero page relocation within RISC OS makes use of this feature – without it we would still need some memory mapped to address zero in order to contain the processor vectors. Although high processor vectors is currently a requirement for zero page relocation, this is likely to change in the future, in order to support zero page relocation on the IOMD (RiscPC/A7000/RPCEmu) build of RISC OS.

OS_PlatformFeatures 0 can be used to determine whether high processor vectors are in use (flag returned in bit 20). Software which needs to interact with the processor vectors directly should use this to determine their location. The address of the processor vectors should not be used to infer the address of the zero page workspace, and vice-versa.

Apart from the change in location, the processor vectors continue to operate as normal – so for FIQ handlers, you can still rely on having 228 bytes of space available for your handler code (from &FFFF001C to &FFFF00FF inclusive). Also, for future compatibility, code should not assume that the processor vectors are readable from user mode – regardless of whether high processor vectors are in use or not.

Unaligned loads

Another change that is planned for RISC OS 5.24 is to re-enable support for unaligned loads/stores on ARMv6+. For the past few years this feature has been disabled by default, to protect you against potential compatibility issues with software which assumes the older ARMv5 “rotated load” behaviour is in effect. But we believe the time to re-enable the feature is drawing near – expect to see it happen sometime during the zero page relocation testing process. By re-enabling support for unaligned loads/stores the performance of some OS operations will be improved, and third-party software aimed at ARMv6+ can more easily make use of the feature.

So you have your log file describing the action of a dodgy application. Then what? Post it here? Notify the author? Something else?

Mr. A. User Esq.

So you have your log file describing the action of a dodgy application. Then what? Post it here? Notify the author? Something else?

Would you not think both?
Then the author has a chance to fix the issue and everyone else gets to know there is a problem under specific circumstances.

Well yes, but I was wondering if it was wanted somewhere specific. As in, in order to collate the info and avoid some repetition.

For info.

The pain is not exactly zero, Aemulor feels pain other than the pain that ZeroPain catches. On a Mk1 Raspberry Pi AemulorPro 2.34 will not start on the high processor vectors ROM OS5.23 (05-Jul-15), it was OK with yesterdays ROM. ZeroPain catches a large number of events but the show stopper is :-

*where
Address 2028EE54 is at offset 000039E0 in module Aemulor
*

There is no entry for that address in the ZeroPain log.

a change to the RISC OS memory map which moves the kernel’s “zero page” workspace away from address zero and up to the high end of the memory map

Another great achievement in the milestone for RISC OS Open. Really making RISC OS more stable and modern. Great to see that decisions are being made to make positive long term changes to RISC OS.

Thank you to everyone involved.

Is anyone else getting a ZeroPain log when they try to compile anything (even something completely trivial like PtrTest)? I’m using DDE 25 on a Pi 1.

Is anyone else getting a ZeroPain log when they try to compile anything

Oh yes. Also using DDE25 on a Raspberry Pi Mk1. Amu is in pain.

Please remind me, as I am nervous about touching !Boot.Loader. I have RO 5.21 on an RPi2. To install the RO 5.23 ROM do I have to go through the WinDiskImager rigmarole, or do I just copy riscos out of BCM2835Dev/5/23/zip, rename it to RISCOS/IMG and then copy it into !Boot.Loader? For backup purposes, in case things go wrong, is it OK to copy the current (5.21) riscos out of !Boot.Loader? In other words, does reading things in !Boot.Loader leave it unchanged?
Probably a silly question, and a nursery one on this thread, but I like to be sure ;).

It’s not too scary (although this post may make it so)! Use Configure’s Boot merge dialogue to install ZeroPain, then open up Loader and back up RISCOS/IMG (I renamed it to RISCOS/BAK). Copy in the new riscos file and rename it to RISCOS/IMG, then press Ctrl-Break¹. The system should come back up without any issues (although in my case I had to reconfigure the screen resolution).

If it fails to come back up, you can restore the old version from the CLI (go into $.!Boot.Loader, delete the non-working RISCOS/IMG and rename RISCOS/BAK back to RISCOS/IMG).

¹ I found that if you do a normal shut down then it’ll write your 5.21 CMOS into the 5.23 image and may fail to boot.

Using two identical cards, I use CloneDisc to make a full backup card before doing anything particularly adventurous. The newly made backup then goes into the Pi to verify its goodness.

To upgrade the ROM I just delete or preserve elsewhere the old ROM and then copy in the new one, rename it and then restart.

!Boot will need to be updated in this case, though I have forgotten what the specific issue is.

It’s not too scary

Thanks Chris and David. I have followed your advice and all appears to be well. A quick glance at ZeroPain showed that PhotoFiler (2.08) was a baddy. So I commented it out in Choices.Boot.Tasks.Boot and restarted.

ZeroPain appears to be 1,052,813 bytes in size. What should I be doing with it? Its initial records show a time of Fri Jan 2 00:00:13 1970 and I presume these refer to my first switch on with RO5.23. The first record has

Last app to start: <Obey$Dir>.SetHome Obey$Dir

The next complains about Organizer (version 1.59). From there on it is PhotoFiler all the way till the date changes to today. I presume that the subsequent records show what happened after my second restart.

Should I be editing ZeroPain – it must contain a heck of a lot of redundant information – or is that taboo? And to whom should I be sending it and when?

ZeroPain does come with a readme file which covers most of the above. In particular:

Remember to review the contents of the log file at regular intervals, to see if there are any issues which haven’t already been reported to the relevant application developer. The information below (“Understanding log entries”) can help you to try and work out which application caused a particular access.

ZeroPain will pause logging once the log file exceeds 1MB in size. Deleting the log will allow it to resume.

Last app to start: <Obey$Dir>.SetHome Obey$Dir

That is !UnixHome, the SetHome absolute just needs a recompile, source is included in the UnixHome application.

readme file.

OK. Thanks. That is clear. I jumped the gun. I guess authors of offending software are going to be bombarded with queries from users about when they will be producing updates – unless they make announcements soon. And we will be discovering that certain applications are no longer maintained.

Is anyone else getting a ZeroPain log when they try to compile anything

We’ve spotted a number of naughty offenders amongst the tools in the DDE. It’s fair to say we’ll need to release an update ahead of making the zero page relocation a permanent thing.

Well some hopefully are still maintained like:

Organizer 2.23 which fails to load but gives some good log file information which I have emailed on to Martin Avison.

So far the following RComp applications:
!Netfetch 3.66
!SafeStore 2.09
!Hermes
SecureG
Plus !Newshound give log file data and Ian Hamilton’s !PrintSpool 2.18d as well.

Doug

We have a ZPP-safe version of NetFetch, but it requires compilation against ROOLs version of RISC_OSlib rather than our modded/32bitted Acorn version. The ROOL one isn’t drop-in compatible, and has a few odd faults (eg. incorrect iconbar menu positioning – probably not taking into account rule-offs on menus) and so on.

I don’t know who at ROOL maintains RISC_OSlib, but if someone would like to email me, I’d be happy to see if we can sort out a blend of our fixes into the master one. Unfortunately, IIRC some of the function parameters have changed over time too, making this a bit less straightfoward than it should be.

If anyone wants the ZPP update to NetFetch, just email me and I can send a new !RunImage.

Secure/Secure32/SecureG need at serious attention – the source is available to anyone who wants to help, but so far no-one has made any headway.

SafeStore/Hermes – I’ll need to ask Alan about.

The PhotoFiler module version 2.08 (05 Feb 2008) by David Thomas appears to peek address &FF8 every few microseconds.
A good thing that ZeroPain has a 1Mb cutoff! Fortunately the assembler source of the PhotoFiler module is distributed with it. I have so far been unable to trace which part of the source is responsible for peeking &FF8. Does this address ring any bells with anybody?

Isn’t that where (part of) the task handle of the currently paged in task is stored?

&FF8 = ‘DomainId’. It’s used to store the (internal) task handle of the current task. Preferably change the code to use Wimp_ReadSysInfo instead, as I can’t think of many situations where directly peeking at the workspace would be beneficial (admittedly, ZeroPain peeks at the value, but that’s because I wanted to avoid assuming that the OS is in a safe state for SWI calls from within the abort handler).

If using Wimp_ReadSysInfo doesn’t seem like it’s going to be possible, you can read the address of DomainId using OS_ReadSysInfo 6

(p.s. this is also something covered in the ZeroPain readme, DomainId is one of the locations where ZeroPain performs special handling)

Thanks everybody. I have updated Photofiler and now it runs without zeropains. The changes are:

File s.FilerActive
<< line 24, 25
;        MOV     r0, #&F00
;        LDR     r0, [r0, #&F8]
>> line 24, 32
        STMFD   r13!, {r1-r2}
        MOV     r0, #6
        MOV     r1, #0
        MOV     r2, #70
        SWI     &20058;    XOS_ReadSysInfo
        MOV     r0, r2
        LDMFD   r13!, {r1-r2}
File s.Divide
<< line 67
;        DivMod  r2, r0, r1, r14
>> 67, 69
        DCD     &E732F110;  UDIV  -- Armv7
        MUL     r14, r2, r1
        SUB     r0, r0, r14

OS_ReadSysInfo will return the address of DomainId, not its contents (I’ll update the wiki to make that clearer). So you need to replace the MOV r0, r2 with LDR r0, [r2]

Although if you’re calling a SWI every time you want to read DomainId then you might as well replace it with a call to Wimp_ReadSysInfo. The code will presumably need some extra tweaking to cope with the different value being returned, but it’s one less peek of kernel workspace in the world.

Just out of interest, why is PhotoFiler looking at the current task handle?

Would it be possible to modify the ZeroPain module so that it would run on an older Rom (ie using un-relocated zero page) so that it would log and allow zero page accesses? As a small-time developer, it would make it much easier to track down any small problems if I could run it on my (stable) RO5.22 machine (which I do not want to update to a non-stable development ROM).