Recent Low Vector ROM for Pi

Is there a recent low vector ROM for the Pi available anywhere? Last one I can see is Chris Hall’s October 2016 version. Any newer ones, preferably with the EDID stuff in them? I have no idea or the necessary software to roll my own, unfortunately.

Why not use the latest high vector ROM? The kernel now sticks a dummy page at page zero, so most software which accidentally tries reading from page zero should work OK.

Oh okay! Not quite sure what that means, but I’ll give it a go! ;-)

Will ZeroPain still be required?

Will ZeroPain still be required?

Depends on the situation. ZeroPain can emulate some old zero page locations, so might help compatibility with software which is deliberately accessing zero page. Plus it produces a log file entry for any zero page access, so that’s useful for reporting/fixing bugs.

Where is the documentation for the ‘high vector’ build – how much zero page has moved, where has it moved to, how is that detected etc?

Where is the documentation for the ‘high vector’ build

Hmm, it’s kind of scattered all over the place.

how much zero page has moved

&0 to &4000, i.e. the processor vectors, low kernel workspace and VDU driver workspace. ScratchSpace has been left as-is (for now).

Moving the processor vectors isn’t possible on IOMD, so that hasn’t had any alterations (yet – moving everything else but keeping the vectors at &0 is a future possibility)

where has it moved to, how is that detected etc?

Processor vectors have moved to &FFFF0000, i.e. the standard ARM high processor vector location. There’s an OS_PlatformFeatures 0 flag to indicate this, and also OS_PlatformFeatures 32, which should probably be used in preference just in case we decide to move them again in future.

OS_Memory 16 has also been extended to report the size & location of things which have been traditionally at &0 to &8000. But just because you can read the address of kernel workspace, you shouldn’t assume anything about its layout or content – firstly because it’s meant to be private to the kernel, and secondly because it is likely to change quite a bit as the SMP work progresses. But if you do need to stick your fingers where they don’t belong, OS_ReadSysInfo 6 has been taught quite a few new locations in recent years in order to allow the rest of the OS/ROM to gracefully deal with the workspace moving.

There’s also OS_Memory 20 which can be used to control the compatibility page.

A lot of what I do is very low level. Please do try to keep things in the same place if they haven’t changed. In other words, don’t do what ROL did and move almost everything because the serial input file handle byte became a word. Grrrrr!

also OS_PlatformFeatures 32, which should probably be used in preference

Which talks exclusively of the processor vectors and not of all the other stuff that has moved. Presumably OS_Byte &A6 returns the correct high address?

I understand the great advantage of moving the first page (the actual “zero page”) but I’m not sure that &1000-&3FFF was any more vulnerable than any other pages. One good reason for moving it is to indirect it to enable multiple graphics contexts.

Is the “compatability page” a set of read-only mappings of the real pages, or a few select values poked into some RAM? Is there any documentation of what this “compatibility” implements and guarantees?

I don’t see a way of reading the start of the VDU workspace (&1000)… must one infer it from the list of WrchV handlers? Will that always be readable at &1000?

ScratchSpace has been left as-is (for now).

Don’t you dare! That’s a defined part of the API, even if the provisos for using it are basically a secret handshake and magic incantation.

By all means add a wrapper to “claim” and “release” it (enabling people without magic knowledge to use it).

A lot of what I do is very low level. Please do try to keep things in the same place if they haven’t changed. In other words, don’t do what ROL did and move almost everything because the serial input file handle byte became a word. Grrrrr!

I’m not making any promises. If you need access to something, ask for it to be exposed via an API. Or add the API yourself!

We can’t get stuck in the situation where we can’t change anything out of fear of breaking someone’s dodgy add-on that peeks and pokes kernel workspace without permission.

Presumably OS_Byte &A6 returns the correct high address?

No it doesn’t. Have Acorn ever documented that call as working under RISC OS? &A8, &AA, &AC and &AE don’t do anything useful, so why should &A6 be maintained?

I understand the great advantage of moving the first page (the actual “zero page”) but I’m not sure that &1000-&3FFF was any more vulnerable than any other pages.

One advantage of moving them is that it frees up the low area of the memory map, so that emulators/compatibility software can map in their own pages to those locations and do with them as they wish.

Plus there’s the whole “not being shackled to a kernel workspace layout from 1987” thing.

Is the “compatability page” a set of read-only mappings of the real pages, or a few select values poked into some RAM? Is there any documentation of what this “compatibility” implements and guarantees?

https://www.riscosopen.org/wiki/documentation/show/OS_Memory%2020

It’s a single page at &0 that’s intended to act as a crutch for buggy software that accidentally reads from null pointers. It doesn’t provide an emulation of kernel workspace locations. So it’s mostly empty, apart from the first 64 bytes or so which contain magic values that are intended to make the buggy software behave in a “safe” manner.

I don’t see a way of reading the start of the VDU workspace (&1000)

Correct. It’s almost like it’s meant to be private or something!

I’m not making any promises.

And such things are now your personal decision (or the whim of whoever can be bothered to check in a change)? Acorn is long dead, but I worry that there is no longer a policy to achieve backwards compatibility unless impossible.

someone’s dodgy add-on that peeks and pokes kernel workspace without permission.

Well, this comes down to a largely philosophical question: Do we want to say “if you want this API then you have to use this version of the OS from this supplier, and if you want that API, then get that OS from them ”, or is it reasonable that people might want APIs and functionality to be available across OS versions?

If it’s the latter, and I am of that camp, then somebody has to provide such compatibility layers. This puts those somebodies in a position somewhere between users and the appointed high priests.

Concrete examples:

VDULarge

There is a known limitation with VDU24, VDU28, VDU29 and VDU31 that cause a problem with very large displays (theoretical) and large sprites (long standing). My little module solves that (to some extent). It can only do that by manipulating VDU workspace. There is no API to do that that is the very problem it solves.

You will argue that such a solution (if not this one) belongs in your OS… but that does not solve the API problem in general. I boycotted RO4-only APIs for precisely the same reason. Acorn used to make updated modules distributable and soft-loadable on older OSes. That went out of fashion for (unsound) commercial reasons. I cannot support that approach.

VectorExtend

A module that greatly extends and enhances vector behaviour, including the automatic vectorisation of all SWIs, can only be implemented at the very lowest level. Knowing the location of the OS vector heads is insufficient, it needs to free the “chocolate block” vector nodes and reallocate, and completely replaces all kernel vector code. There isn’t a polite way of doing that.

Debugging utilities

*Vectors lists all vector claimants, in order, including offsets into modules.
*Afters lists all scheduled callbacks.
*Commands lists all “star” commands or finds the location of a command – the name of the module or the full library path.
*Services lists all service handlers, in order.
*SWIs lists all SWIs and the modules that provide them.

I’m unimpressed by the argument that such things should not exist, or that such things must be “dodgy”. But you’re entitled to your opinion.

No [OS_Byte &A6] doesn’t. Have Acorn ever documented that call as working under RISC OS?

It was part of Arthur and has not been deprecated. An important clue is that it returns the correct value on all versions of RO except the one where zero page moved. Perhaps whoever moved it forgot about &A6? (It requires two extra instructions in the high vector case)

And such things are now your personal decision (or the whim of whoever can be bothered to check in a change)?

No more or less than they are yours, I’d imagine.

And such things are now your personal decision (or the whim of whoever can be bothered to check in a change)?

We have forums where changes can be discussed. We have public CVS where changes can be viewed. You have ample opportunity to object to any changes that are made before they appear in stable OS releases.

Now compare that to the situation where ROL shuffled everything along because they changed a byte to a word.

Acorn is long dead, but I worry that there is no longer a policy to achieve backwards compatibility unless impossible.

How can we be backwards compatible if we don’t know what we need to be backwards compatible with? Unless we know what kernel workspace is being used (and how/why), we don’t know whether any change we might make might break compatibility. Therefore we can’t make any changes.

VDULarge … You will argue that such a solution (if not this one) belongs in your OS

And I would be correct to do so, if it’s something that involves peeking and poking private kernel workspace.

Acorn used to make updated modules distributable and soft-loadable on older OSes. That went out of fashion for (unsound) commercial reasons. I cannot support that approach.

Either you’re mad or I’m misunderstanding you, because it sounds like you don’t support the practice of having APIs that are supported across all OS versions. And yet, APIs which are supported across all OS versions seems to be the exact thing which your modules aim to provide.

VectorExtend … Knowing the location of the OS vector heads is insufficient, it needs to free the “chocolate block” vector nodes and reallocate, and completely replaces all kernel vector code. There isn’t a polite way of doing that.

Polite or not, it could still ask the OS for the locations of the things it needs, instead of resorting to guesswork or undocumented APIs.

I’m unimpressed by the argument that such things should not exist

It’s a good job I’m not making that argument, then.

or that such things must be “dodgy”.

I can use the word “impolite”, if you’d prefer.

No [OS_Byte &A6] doesn’t. Have Acorn ever documented that call as working under RISC OS?

It was part of Arthur and has not been deprecated. An important clue is that it returns the correct value on all versions of RO except the one where zero page moved. Perhaps whoever moved it forgot about &A6? (It requires two extra instructions in the high vector case)

Arthur was declared obsolete with the release of RISC OS 3, and I can’t see any obvious mention of OS_Byte &A6 in the RISC OS 3 PRMs. To me this is a pretty strong indicator that the call has been deprecated (regardless of whether it was meant to be part of Arthur or not).

Perhaps whoever moved it forgot about &A6? (It requires two extra instructions in the high vector case)

How can we know what those two extra instructions should look like, if we have no documentation for the call? There are many different ways in which the returned R1 and R2 values can be combined to make an address, unless we know what the “correct” way is there is no way for the call to be extended to return a 32bit value.

Arthur was declared obsolete with the release of RISC OS 3

Although there is documentation of calls such as OS_Control that have been deprecated, I can’t find any mention of OS_Byte &A6 being withdrawn. Pardon my candour, but I suspect it has been forgotten. It’s easy to miss – it’s in the initialisation data for the MOS vars. That approach will no longer work for addresses with more than 16bits, hence a couple of extra instructions required.

There are many different ways in which the returned R1 and R2 values can be combined to make an address

No, there are two ways, and one of those is too highly contrived to consider. So: ORR R1, R1, R2, LSL#8

And I would be correct to do so, if it’s something that involves peeking and poking private kernel workspace.

And you dodge the question: How is that API provided to users of other versions of the OS, if not by a module manipulating workspace? Everyone accepts that end users, application authors and hobbyists should not be accessing such things. But some things have to, by definition.

Steve added

No more or less than they are yours, I’d imagine.

And the point I am making, and I think it is a reasonable one, is that one should not move things for the sake of moving them, fix things that are not broken, etc. Moving zero page has a benefit to a particular theoretical usage case that cannot be achieved currently. Ok. But moving hundreds of locations as ROL did merely because they changed one byte to a word should be avoided. Move the thing that is changing, don’t move the things that aren’t.

There is a huge amount of precedence for that, just look at the sources. That is all I asked for, and got the response “no promises”… which I’m disappointed by.

I have just recalled that UKVDU23V includes a reference to the VDU queue, so that is a publicly accessible part of the “private” VDU workspace… at least for the duration of the vector call.

Don’t you dare! That’s a defined part of the API, even if the provisos for using it are basically a secret handshake and magic incantation.

Oooh, handbags at dawn! :-)

You do know you are a little late to this party, right?

BTW, if it’s a secret handshake and magic incantation, then it is not a defined part of any API.

You’ll be referring, no doubt, to the glaring cock-up that is in PRM 5a-41 where it says that scratch space is a “public area” that may be used by any module that is neither in IRQ mode nor going to call something that might also want to use it (it seems FileCore likes to use this). FWIW, the only mentions in PRMs 1-4 are OS_HeapSort (PRM1) and where filing system temporary buffers are claimed (PRM2).

However one should probably consider that table to be invalid since it is describing a 26 bit operating system and references to hardware (VIDC20 etc) that no longer exists on most machines. Example? Look at where it says the SVC stack is. Now call OS_ReadSysInfo 6, 0, 16…
Likewise, the Soft CAM map is given in that table as 30M+8K which ought to put it around address &01E02000. OS_Memory says in RO5 it’s at &FB000000.
Etc etc etc. If these are all changing to accomodate a newer memory map, then two questions:
1, Why not the “scratch space”?
and:
2, Why aren’t you claiming a lump of RMA or DA like everybody else? The use of “scratch space” was a huge dose of laziness on the part of Acorn when they originally wrote RISC OS (maybe even Arthur) so they reserved 16K of memory to use as dump space instead of doing things correctly like we’re all supposed to.

Now I suppose Jeffrey could add an extra value to OS_Memory 16 if you really need access to this memory, but really shouldn’t you be using a legitimate private claim like all the rest of us? Or should we all consider that 16K to be open season and available for free use as required? I’m sure you can see where that would end up…

or is it reasonable that people might want APIs and functionality to be available across OS versions?

Yes/no. Yes, the API and functionality should be available as far as is logical (later builds will always have more features – consider things like VFP or the ability to have display modes greater than FullHD, or the RGB/BGR issues); but the collarary to that is to ask should development of features be deferred, mangled, or just abandoned because it might affect backwards compatibility? The answer to that lies mostly in what it is in question – I doubt anybody will ever add pen pressure readings to OS_Mouse because anything that expects the API to use only R0-R3 will break the moment that suddenly R4 is used. However APIs that were largely undocumented are liable to change.

Acorn themselves provided a pretty good example back in the day. It was well known that using OS_UpdateMEMC, 64, 64 would speed up BASIC. I’m not entirely convinced I ever saw much benefit on my RISC OS 2 A3000, but it turned up all over the place.
Along came RISC OS 3, and the instant that SWI was executed, instant machine freeze. A lot of stuff needed to be fixed in order that it work correctly on RISC OS 3.

that cause a problem with very large displays (theoretical)

You mean the part where there is no way to set the text cursor to a position >255?

I have long said that messing around with OS_Byte to get information from the keyboard, and spitting VDU calls to adjust the behaviour of the text cursor is a horrendous API that has barely changed since the BBC Micro. Ideally we need to consider VDU-anything to be legacy, and have an API suitable for the modern era.

Debugging utilities

Debugging utilities, by their very nature, will always play fast and loose with the internals of the operating system. I’d bet DDT does a lot of nasty stuff too…

It was part of Arthur and has not been deprecated.

Hmmm, &A6 isn’t in my StrongHelp file.

[…]

After half an hour of rummaging through the source – it looks like you probably ought to pick up the pointer exposed with OS_ReadSysInfo 6,71 which provides a pointer to the OS workspace (akin to that which you would get from OS_Byte &A6 onwards).
Kernel.s.Middle, around line 1555.

However, it looks like it’s a two-byte value, so by definition it cannot hold a 32 bit value (Kernel.hdr.KernelWS, around line 303).

If you ask really nicely, Jeffrey might alter those initial five entries to be words instead of halfwards. Of course, everything else will be offset by ten bytes and all the OS_Byte calls >&A6 will be wrong too, but hey, it worked for ROLtd right?

Acorn used to make updated modules distributable and soft-loadable on older OSes. That went out of fashion for (unsound) commercial reasons. I cannot support that approach.

Hmmm? You had noted that there was a download called PlingSystem hiding in plain sight on the Miscellaneous downloads page.
It’s sole reason for existing is for use on old systems.

You will argue that such a solution (if not this one) belongs in your OS

If the utilities were included in the main OS (and in the aforementioned PlingSystem) then everyone has them, either in the default OS or as a drop in for the older versions like v4.39 but it does require the author to put them forward for inclusion.

And I would be correct to do so, if it’s something that involves peeking and poking private kernel workspace.

And you dodge the question: How is that API provided to users of other versions of the OS, if not by a module manipulating workspace? Everyone accepts that end users, application authors and hobbyists should not be accessing such things. But some things have to, by definition.

You’re worried about future versions of RISC OS changing the layout of kernel workspace and breaking your code.

My argument is that:

1. ROL’s fork of the OS is dead, so there’s virtually no chance of a new version being released which changes kernel workspace. So when your modules run on that OS version they can continue to peek and poke the hardcoded addresses without any worries.
2. ROOL’s fork of the OS is alive, so there’s a high chance of changes being made to kernel workspace. So to avoid us having to worry about changes breaking things, we can either (a) extend the OS to expose the addresses and other information that your modules need, or (b) implement the functionality provided by your modules directly into the OS.

Move the thing that is changing, don’t move the things that aren’t.

There is a huge amount of precedence for that, just look at the sources. That is all I asked for, and got the response “no promises”… which I’m disappointed by.

If my answer disappoints you, try contacting ROOL directly. They’re the ones that make the rules around here.

However, do realise that “no promises” doesn’t mean I’m going to completely randomise the kernel workspace on every check in. Generally I try and keep things in the same place, but guaranteeing that is becoming increasingly difficult (many different kernel build configurations to consider, and not much free space left to insert things). Hence, until ROOL decide otherwise, I reserve my right to move stuff around as I deem necessary.

A lot of what I do is very low level. Please do try to keep things in the same place if they haven’t changed.

Where the OS places it’s internal variable should come out of the build process and nothing outside of the OS should be touching it. Any “low level” tinkering should be done via legal OS calls. If a specific value isn’t currently available via an OS call, I’m sure it will be considered if its requested.

[OS_Byte 166] was part of Arthur and has not been deprecated

Although it was possibly never officially deprecated, its not a call that was for public use under RISC OS, its always been “Acorn use only”. The only program I’ve ever come across that used it was !65Host, but that does far dodgier things than just poke around in Page Zero.

randomise the kernel workspace

Why don’t we do this? Linux and Window both randomly allocate their private workspace specifically to stop idiots from poking around in OS private space.

Arthur was declared obsolete with the release of RISC OS 3, and I can’t see any obvious mention of OS_Byte &A6 in the RISC OS 3 PRMs.

It’s not listed in the Arthur PRMs either; the index of OS_Bytes skips &A6 (and a few others). It looks like it was never intended to be used by third-party developers.

Edit: Actually, page 14 says this:

For [OS_Byte] calls with R0 between &A6 (166) and &FF (255) on entry, the action is the same for each value. R0 acts as an index into the RAM which contains certain system variables. These variables are held in consecutive memory locations, with R0=&A6 accessing the first one, R0=&A7 affecting the second one, and so on.

So it seems that &A6 is (or rather, was) documented as providing access to system variables, although the PRM doesn’t appear to document what that particular variable at &A6 is.

Is there a recent low vector ROM for the Pi available anywhere? Last one I can see is Chris Hall’s October 2016 version.

Now that ROOL have provided a page of memory at &0xx for those apps that harmlessly read a null pointer and ignore the result and now that a high vector version of Aemulor is available to support such apps as Impression-X, my current recommendation is that users should move to a high vector rom. The transition is now much simpler.

My only gripe with current roms is that if you set monitortype to EDID but then run with no monitor connected, the boot process hangs. Previously with no monitor connected on a Raspberry Pi the HDMI system simply doesn’t power up (saving battery drain). It would be nice if the boot process didn’t hang with no monitor connected and EDID configured. (I have an OLED display and only connect an HDMI display to debug things.)

Have Acorn ever documented that call as working under RISC OS? &A8, &AA, &AC and &AE don’t do anything useful, so why should &A6 be maintained?

These calls are detailed in the BBC Micro Advanced User Guide by Bray, Dickens and Holmes.

&A6 and &A7 Read start address of OS variables
&A8 and &A9 Read address of ROM pointer table
&AA and &AB Read address of ROM information table
&AC and &AD Read address of keyboard translation table
&AE and &AF Read VDU variables origin

&A6 returned the start of the OS_Byte values, which for RISC OS 3.1 started at &8FA. The complete list of undocumented OS_Byte’s previously referred to the following OS variables:

8FA Start of OS_Byte values (VarStart)
...
9A0 VarStart # 2 ; &A6,&A7
9A2 ROMPtr # 2 ; &A8,&A9
9A4 ROMInfo # 2 ; &AA,&AB
9A6 KBTran # 2 ; &AC,&AD
9A8 VDUvars # 2 ; &AE,&AF
...
9AC KeyBdSema # 1 ; &B2
9AD ROMPollSema # 1 ; &B3
9AE OSHWM # 1 ; &B4
...
9B1 CFSRFS # 1 ; &B7
9B2 VULAcopy # 2 ; &B8,&B9
9B4 ROMatBRK # 1 ; &BA
9B5 BASICROM # 1 ; &BB
...
9C7 RS423CFSFlag # 1 ; &CD
9C8 EconetOScall # 1 ; &CE
9C9 EconetOSrdch # 1 ; &CF
9CA EconetOSwrch # 1 ; &D0
9CB SpeechSupr # 1 ; &D1
9CC SoundSupr # 1 ; &D2
...
9D1 StartMessSupr # 1 ; &D7
...
9E1 u6522IRQ # 1 ; &E7
9E2 s6850IRQ # 1 ; &E8
9E3 s6522IRQ # 1 ; &E9
9E4 TubeFlag # 1 ; &EA
9E5 SpeechFlag # 1 ; &EB
...
9E9 Shadow # 1 ; &EF
...
9EB UserFlag # 1 ; &F1
...
9EE SoftKeyConsist # 1 ; &F4
...
9F1 BREAKvector # 3 ; &F7,&F8,&F9
...
9F6 LangROM # 1 ; &FC

There’s no reason to maintain any of these as they’ve never been officially documented for RISC OS and should not be used outside of the OS.

There’s no reason to maintain any of these as they’ve never been officially documented for RISC OS

Not to mention, the addresses are sixteen bit so are surely functionally useless for RISC OS. So this behaviour is (yet another) BBC MOS throwback…

https://www.riscosopen.org/wiki/documentation/show/OS_ReadSysInfo%206

N.B. This call is for internal use by the operating system and certain utilities (e.g. SoftLoad, ROMPatch). It should not be used by user programs. The purpose of the call is to return information about the Kernel’s internal variables.

Item 71:-
71 5.17+ Address of OS byte variables

https://www.riscosopen.org/wiki/documentation/show/OS_ReadSysInfo%206%20Items

Is this a modern OS_Byte &A6.

Yes, OS_ReadSysInfo 6 item 71 is a modern replacement for OS_Byte &A6.

I can’t remember what it was, but something in the OS was trying to use OS_Byte &A6 (perhaps SpriteExtend?). Extending OS_ReadSysInfo 6 seemed to be more sensible than extending OS_Byte, since:

1. Putting everything internal in OS_ReadSysInfo 6 makes it easier to search the OS sources to see where internal things are being accessed (just grep for ‘OSRSI6_’)
2. OS_Byte provides read/write access to the value, which is nonsense, because the kernel doesn’t support the OS_Byte variables being moved
3. OS_Byte stores/accesses the address as two 8-bit values, which is nonsense, since we want a single 32-bit value
4. Nobody’s provided any evidence that OS_Byte &A6 has been fully & officially documented under RISC OS, so apart from dodgy bits of the OS and the occasional program ported from the BBC micro nothing should be using it.

Perhaps it would make sense for OS_Byte &A6-&AF to return an error, at least for high vector ROM builds.

Nobody’s provided any evidence that OS_Byte &A6 has been fully & officially documented under RISC OS

Did Acorn ever do so for the BBC Micro. The BBC Micro Advanced User Guide referenced above is a third party publication. These calls are not in Acorn’s BBC Micro User Guide.