OS_ReadLine, OS_ReadLine32 and ReadLineV, a history

I have to draw a distinction between the intention of the RO6 API and its actual implementation which is buggy, naturally enough.

OS_ReadLine calls ReadLineV with no changes (as Arthur intended)

OS_ReadLine32 sets R1b31 if R0:b26-31 or R4:b8-25 are set. Otherwise, it moves the flags from R4 to R0 but forgets to clear R1b31 if it was set on entry.

ReadLineV uses the ‘new API’ if R1b31 is set, or if it is running in 32bit mode which is nonsense¹, otherwise it uses the ‘old API’.

The intention is robust, the implementation is wobbly.

(The user of) OS_ReadLine is susceptible to unexpectedly high addresses, but these are unlikely really. There’s not much that could be done to protect against that, as a valid high address truncated to 26bits stands a good chance of being seen as a valid address in application space.

¹ This is likely to be an attempt at compatibility with RO5’s “ReadLine32V” decision, but 32bit<>RO5, and running old code in 32bit mode doesn’t magically change where it put its flags.

Incorrect. R1b31 (on ReadLineV) indicates the OS_ReadLine32 API is in use, otherwise the OS_ReadLine API is in use.

From the above document:

R1 = maximum length of line
     26bit - bit 31 should be set to indicate the full address range is used.
     32bit - bit 31 will be ignored

Further backed up by the text: On 32bit systems: The ‘Full address range interface’ will always be used for ReadLineV. On 26bit systems. Bit 31 of R1 indicates whether the ‘Full address range interface’ or the ‘Old-style interface’ is present.

This implies that bit 31 will not be set on a 32 bit system as the API is always 32 bit.

RO5 in similar. On a 32 bit system it is always 32 bit. On a 26 bit system, I have no idea if CallASWI forces the 32 bit API or not? If this is not the case, then assumptions can be made by looking at the high bits of R0. It can’t be a valid address on 26 bit, and on 32 bit it is always presented to ReadLineV in it’s 32 bit incarnation.

Except it might not be. Are you asserting that b30+valid_RAM_address, b31+valid_RAM_address and b31+b30+valid_RAM_address are always invalid addresses? I don’t think that’s true.

On a 26 bit system, the upper six bits of PC are flags, so these bits can never be used for valid addresses. Indeed, any value over &3FFFFFF cannot be a valid address on a 26 bit system.
So yeah, I am asserting that. ;-)

ON A 26 BIT SYSTEM.
On 32 bit, it looks as if both versions call the 32 bit API. Which is logical.

Now, I understand that what the docs say and what the OS does may differ, however, going by the documentation:

OS_ReadLine on RO5 – Broken
OS_ReadLine32 on RO5 – calls ReadLineV
ReadLineV on RO5 – same as OS_ReadLine32

Why is OS_ReadLine broken on RO5? ReadLine should be bodged to look like ReadLine32, with ReadLineV using the 32 bit API.

OS_ReadLine on RO6 – calls ReadLineV
OS_ReadLine32 on RO6 – tries to downgrade to OS_ReadLine, or sets R1b31
ReadLineV on RO6 – R1b31 selects the API

My understanding of the documentation is that a 26 bit version uses R1b31 to select the type of ReadLine in use, while a 32 bit version always uses the 32 bit form just like RO5.

This sort-of applies to RISC OS 4 as well, for the top of the page says “Clients should use this interface if they know they are to run on Select 4 or later. Specifically, this interface is available where the ReadLine module version 0.05 is present.”.

OS_ReadLine on RO3 – calls ReadLineV
ReadLineV on RO3 – original API only

Yes. RO3 predates all the fancy 32 bit stuff.

OS_ReadLine through CallASWI – calls ReadLineV
OS_ReadLine32 through CallASWI – calls ReadLineV
ReadLineV with CallASWI – no way to detect API

Ah, now the important thing to look at here, given that addresses over 64MB are invalid on a 26 bit system is what version of ReadLineV is used by CallASWI. It would, in this case, be perfectly feasible for it to bodge the ReadLine32 into something that looks like ReadLine (flags in R0) and call that. Because b30 and b31 are safe to use in a 26 bit address.

I reserve the right to be utterly wrong (and maybe the documentation is too), but it looks like you’re making this way harder than it is.
A 32 bit OS should be using the 32 bit API. Always.
An older 26 bit OS should probably be using the 26 bit API. You’ll need to check the R1b31 flag in SIX (or later Select) and you’ll need to look to see what 26 bit CallASWI is actually doing with ReadLine32. Other than that…….

For example, if CallASWI is in use on RO4 and OS_ReadLine32 is called, then the ‘new’ API is in use, but there’s no R1 flag to indicate it.

You seem to be claiming that CallASWI changes the behaviour of OS_ReadLine and ReadLineV on 26bit systems. I’ve just downloaded the latest version of CallASWI (0.18) and tested it on RO4.39, with the following results:

OS_ReadLine calls ReadLineV
OS_ReadLine32 moves flags from R4 into R0 and calls ReadLineV

i.e. CallASWI has not changed the API of OS_ReadLine or ReadLineV. This matches the ROL and the Castle documentation, which both state the 32bit ReadLineV API is only used on 32bit systems (or RO6 with bit 31 of R1 set), and is therefore detectable. What OS and CallASWI versions are you testing with?

The other thing I discovered is that the RISC OS 4.39 taskwindow module is calling OS_ReadLine with bit 31 of R1 set. Despite being documented as a Select 4 feature it’s actually available in 4.39, unless you’re using the LineEditor module which doesn’t recognise the flag.

Rick surmised

This implies that bit 31 will not be set on a 32 bit system as the API is always 32 bit

It is not cleared on entry to either SWI, so if the caller sets it, it will be set for ReadLineV.

The general point, that the processor mode defines the API of ReadLineV, is something that should be clearly documented here, rather than inferred from the other place. That has an impact on existing code.

RO5 in particular nobbles OS_ReadLine so it doesn’t work properly (can’t do passwords) and then passes its parameters down ReadLineV which, you assure me, us now using OS_ReadLine32’s API… which means OS_ReadLine has changed into a broken version of OS_ReadLine32. It can only work by coincidence.

On a 26 bit system, I have no idea if CallASWI forces the 32 bit API or not?

The reverse, it forces the 26bit API, as has always been the case for ReadLineV, and raises an error if an incompatible use of OS_ReadLine32 is attempted.

On 32 bit, it looks as if both versions call the 32 bit API. Which is logical.

Not for existing code. You can do what you like with new code, but for existing code OS_ReadLine has been damaged.

clients should use this interface if they know they are to run on Select 4 or later

I presume they mean “claimants”. Users of OS_ReadLine32 can be completely unaware of R1b31 but it will still get set under certain circumstances. R1b31 is a good thing.

Why is OS_ReadLine broken on RO5?

Despite the fact it is known to be the 26bit interface, and hence is likely to have flags in R0, that is treated as an address under all circumstances. Also, R4 is always set to zero, even if R0 contained the password flag and R4 the password chr.

So OS_ReadLine under RO5 no longer does password input. That’s broken.

My understanding of the documentation is that a 26 bit version uses R1b31 to select the type of ReadLine in use, while a 32 bit version always uses the 32 bit form

Nearly! That is the intention, but unfortunately though the default ReadLineV implementation does check the processor mode and hence jumps to 32bit mode automatically, the OS_ReadLine implementation doesn’t check the processor mode, and so doesn’t switch to 32bit mode.

Just like RO5, the 26bit API gets misinterpreted as the 32bit API. In effect, a vital piece of information is discarded.

it looks like you’re making this way harder than it is

Try to do a password input using OS_ReadLine in RO5. (Can’t) Is that because of the processor mode, or the OS version, or the version of CallASWI? Is that defined behaviour or a bug? Is OS_ReadLine deprecated?

Does processor mode define the API on ReadLineV? (Looks like it) Isn’t that a really important thing that should be front and centre on ReadLineV’s documentation here?

edwardx confirmed

CallASWI has not changed the API of OS_ReadLine or ReadLineV

Correct. The 26bit interface is enforced. Under RO6 the R1b31 flag indicates the 32bit API. Under all circumstances (it appears) 32bit processor mode requires the 32bit API regardless of R1.

Meanwhile, the documentation here says ReadLineV does what OS_ReadLine does. :-)

The other thing I discovered is that the RISC OS 4.39 taskwindow module is calling OS_ReadLine with bit 31 of R1 set

8-O Checks… Oh heavens. Right.

4.39 includes the usual ReadLineV implementation in the Kernel and a separate ReadLine module which replaces it. And yes, the only difference is the detection of the R1b31 bit. However, OS_ReadLine32 is not implementated anywhere, so one must fall back on the CallASWI version that converts back to the 26bit interface.

So the only way to make use of the ‘new API’ in 4.39 is to call ReadLineV manually with R1b31 set… which TaskWindow is doing.

What OS and CallASWI versions are you testing with?

I’m looking at the code of every version of the OS. :-(

I have now created documentation for ReadLineV

So OS_ReadLine under RO5 no longer does password input. That’s broken.

Holy crap. File a bug report, for this is technically not broken (it is documented) but it is utterly utterly awful.

Quoting: https://web.archive.org/web/20160316155735/http://www.iyonix.com/32bit/APIchanges.shtml

“In a 32-bit OS, OS_ReadLine now interprets R0 as a 32-bit address, and acts as though the flags are both clear. This reflects the most common usage, and allows applications not wanting to use the flags to remain unaltered.
A new SWI, OS_ReadLine32, takes its flags in R4. Bits 31 and 30 correspond to the original flags. Bits 29-8 are reserved and must be zero. Bits 7-0 are used as the echo byte (if bit 30 is set). As before, R4 is preserved by the call.”

Here’s the code (https://www.riscosopen.org/viewer/view/castle/RiscOS/Sources/Kernel/s/Middle?rev=4.24#l22):

ReadLineSWI
         Push    "r4,lr"
         MOV     r4, #0
         SWI     XOS_ReadLine32
         Pull    "r4,lr"

What is this crap?

It is never acceptable to arbitrarily break an existing API. I hope everybody reading this is aware that R0 to ReadLine can never be more than 64MB, so bits 30/31 will always be flags. As such, it isn’t exactly tedious or difficult to look to see if these bits are set, and if so, to merge them into R4. Instead, for some half-assed “common usage” scenario, which from my understanding of what is going on, is equally broken (so it’s blanking R4 and tossing the result to the ReadLine32 SWI – exactly where is it clearing the flags in R0?) we are left with an original API apparently broken by design.

This is awful. Please fix!

I hope you understand now why I’ve been obsessing over what is such a simple interface that it predates even Arthur, and yet is now so complicated. The fact that there has never been accurate documentation cannot have helped.

we are left with an original API apparently broken by design

Yes. It ought to do better than that. It is difficult to imagine how an existing user of OS_ReadLine could acquire an address higher than 3FFFFFFF and hence confuse this API. But passing off what is known to contain flags in OS_ReadLine as a pure address in ReadLineV is simply nonsense.

It ought to move the flags to R4 first. And, I would argue, OS_ReadLine32 ought to set R1b31 regardless of processor mode, but it may be Too Late Now™ (having to check two things before you know you’re in the 32bit version of ReadLineV is irritating).

ReadLineSWI
         Push    "r4,lr"
         AND     lr, r0, #3<<30    ; extract flags
         BIC     r0, r0, #3<<30    ; and remove them from r0
         BIC     r4, r4, #3<<30    ; clear bits in r4
         ORR     r4, r4, lr        ; and add flags to chr
         SWI     XOS_ReadLine32
         Pull    "r4,lr"

One could check the resulting R0 is a valid address, but I don’t think it’s necessary.

I have now created documentation for ReadLineV

You wrote: “In 26bit processor modes, the API is dependent on R1 bit31. If it is set, the API is as above (and bit31 must be masked out). Otherwise it is as OS_ReadLine:”
This is not accurate. Select/Six will (mostly) honour this. All versions of CallASWI on older builds won’t.

It ought to move the flags to R4 first.

Anything else is serious stupid. Your fix, to correctly support the existing API, runs to four instructions. Hardly earth shattering.

And, I would argue, OS_ReadLine32 ought to set R1b31 regardless of processor mode,

That’s another non-problem. I would argue that R1b31 is only necessary if wanting to use the 32 bit API with the 26 bit call while ignoring the 32 bit call. That’s really all it is there for.

Far simpler for the 32 bit API to degrade to the 26 bit on a 26 bit system (it’s not as if it can work otherwise) and the 26 bit API to be promoted to the 32 bit API (properly, not that crap above) on a 32 bit system. This is what is supposed to happen and would make sense. R1b31 is therefore completely unnecessary, and yet more API junk to have to deal with.

This is more controversial:

ReadLine32SWI
         ORR     r1, r1, #1<<31       ; 32bit API
         MOV     r11, #OS_ReadLine
         B       VecSwiDespatch
...
VecRdLine ROUT
 
         Push    "R4-R7"
         BIC     R1, R1, #1<<31          ; we are always 32bit API
         MOV     R7, R4                  ; R7 = flags + echo byte

That is what I would have done back then, not for the benefit of claimants running on RO5, as they would have to be rewritten anyway, but for the benefit of the API.

Rick misunderstood

This is not accurate

Yes it is. It is the documentation of ReadLineV for the benefit of people writing claimants. People wishing to input lines of text use OS_ReadLine or OS_ReadLine32. This is an accurate description of the API of the vector, not the API of the SWIs. Since RO6 OS_ReadLine32 does set that bit, it ought to be described for the benefit of claimant authors.

R1b31 is therefore completely unnecessary

It is possible to have high memory dynamic areas in some 26bit machines, is it not?

I hope everybody reading this is aware that R0 to ReadLine can never be more than 64MB

Not exactly true. It can be more than 64MB, but (until the RISC OS 5 / Select tweaks arrived) it wouldn’t have worked because the kernel would have treated the top 6 bits as flags. The result of this is that code written for RISC OS 3.5 had to be aware that some areas were safe for use (application space, RMA, SVC stack) while others weren’t (e.g. most other DAs).

Fast-forward to RISC OS 5, where the RMA and SVC stack are now located above the 64MB limit (definite breakage), and application space can now be up to 512MB (chance of breakage). So code which worked fine under RISC OS 3.5 now stands a reasonable chance of failing because memory locations which were defined as being safe for use are no longer safe for use.

Faced with a choice between “break the flags by assuming they’re zero”, or “break the addresses by assuming the flags are valid”, Pace/Castle went down the “break the flags” approach. Both have their own pros and cons. They could have certainly made things easier by adjusting the RISC OS 5 memory map so that the RMA & SVC stack are located under the 1GB limit (allowing just the top two bits of R0 to be used for flags, like with RISC OS 6), but maybe that was a slippery slope considering that OS_ReadLine isn’t the only API that stored flags in address values. And if we’re playing the blame game then it’s really all Acorn’s fault for sticking flags in address values in the first place.

Of course the safest option when faced with this problem would have been to remove OS_ReadLine and force authors to switch to OS_ReadLine32 instead. Likewise the other SWIs that had 32bit replacements introduced (OS_HeapSort, OS_SubstituteArgs). Maybe I’ll do that once RISC OS 5.24 is released.

Instead, for some half-assed “common usage” scenario, which from my understanding of what is going on, is equally broken

The “common usage” is for the flags to be zero, so under the assumption that OS_ReadLine will only ever be used for the common usage scenario there are no flags to transfer over to R4.

as they would have to be rewritten anyway, but for the benefit of the API.

There is nothing wrong with the API. 26 bit behaviour on a 26 bit system, and 32 bit behaviour on a 32 bit system, with transparent fudging between the two.

What is wrong is API spam and broken implementations.

Since RO6 OS_ReadLine32 does set that bit, it ought to be described for the benefit of claimants.

…and very carefully identified as such, because – and read this carefully, it’s from ROLtd’s own documentation: Old API: bit31 – must be clear, would cause buffer overflow if set on old systems

You cannot arbitrarily change an existing API and not expect there to be problems. Not you, not Castle. On older machines, a buffer size is a buffer size and NO bits were reserved for other purposes.

It is possible to have high memory dynamic areas in some 26bit machines, is it not?

While Jeffrey (who knows) was elaborating on the technicalities of DAs (etc) on RiscPCs, I mused that I wasn’t actually sure how they would have been handled in a 26 bit world… But would OS_ReadLine be using a DA as a buffer, given what it is?

And if we’re playing the blame game then it’s really all Acorn’s fault for sticking flags in address values in the first place.

Yup. :-)

The “common usage” is for the flags to be zero, so under the assumption that OS_ReadLine will only ever be used for the common usage scenario there are no flags to transfer over to R4.

Two points:

1. Assuming anything breaks the API (while there is a chance that you might be able to reach a high memory address on a RiscPC, let’s face it, what OS_ReadLine does is block the entire machine waiting for keyboard input, so it’s likely to either be a module (RMA) or a user mode application (appspace). You know, since we’re assuming stuff. :-)
2. They are “assuming” the flags will be zero, and not clearing them otherwise. So it’s like a broken API broken twice.

And if we’re playing the blame game then it’s really all Acorn’s fault for sticking flags in address values in the first place.

8~) Yes – but can you really imagine ever needing 64MB of address space, never mind more than that?

But would OS_ReadLine be using a DA as a buffer, given what it is?

OS_ReadLine itself probably not but a caller might. One of the changes in LineEditor 2.76c was to clear all top 6 bits of the buffer address.

At the same time RISC OS Select moved the workspace for TaskWindow to a DA. The DA address was such that the bottom of those 6 bits was set so LineEditor 2.76c promptly aborted.

When I reported the bug Darren Salt reverted the code back to clearing just the top 2 bits for 2.76d (which I believe is the most recent version).

reverted the code back to clearing just the top 2 bits

Yes, as described at the start of this thread even in ‘26bit’ OSes the actual address in R0 has variously been 26bit and 30bit depending on version.

Jeffrey said

the safest option when faced with this problem would have been to remove OS_ReadLine

…or emit a more diagnostic error. I agree that would have been best. And then, if the individual user wanted, they could load a compatibility module that provided it and whatever fudging worked best for their case.

There’s a very strong case indeed for deprecating those three right now.

Yes – but can you really imagine ever needing 64MB of address space, never mind more than that?

That, I’m sure, was a bait item for the old DOS 640k chestnut.

Speaking of LineEditor (2.76d), it does have a bug in its ReadLineV handler, which I mentioned here along with a fix. Essentially it corrupts R4 and there’s another bug that breaks the BASIC OFF command.

That, I’m sure, was a bait item for the old DOS 640k chestnut.

8~) Really? 8~)

Okay. I’ve had a play with this, and here is what I propose.

• All 26 bit versions of RISC OS will use the 26 bit API.
• All 32 bit versions of RISC OS will use the 32 bit API.

This much is nominally the case.

• If a 26 bit version of RISC OS supports the 32 bit API, it should perform whatever adjustment is necessary to translate the 32 bit API into its 26 bit form.

I cannot find any sources to the CallASWI module to check that this is being done correctly. It looks as if ROLtd’s versions do sort-of perform this action, albeit with the nonsensical R1b31 bit to flag which API may be in use. Personally, I rather suspect R1b31 was being used to switch API before ReadLine32 was known about, and it become a legacy issue once that SWI was implemented.

• All 32 bit versions of RISC OS should correctly support the 26 bit API.

RISC OS 5 utterly fails here. There are dubious assumptions and a failure to clear flags from something that would appear to be wanting to be treated as an address.
nemo is in a better position to comment on Select/SIX, but from the documentation API it does seem as if this is the case.

After testing on RISC OS 3.70 on RedSquirrel, it appears that all addresses handed out by OS_DynamicArea for creating an area are beyond the range of 26 bit addressing (so I was wrong about that – when speaking about all addresses being within the 64MB boundary, I forgot the RiscPC did weird things!).
However all of those ‘higher’ addresses (on my ‘32MB machine’) are within the range of 29 bit addressing. So for the legacy API, it should suffice to treat addresses passed to OS_ReadLine as having bits 30 and 31 being flags, and all of the rest being a potential address (this allows us to address &3FFFFFFF (note, that’s seven F’s, not six!)).
There may be edge cases where OS_ReadLine does indeed fail on RISC OS 5 as this solution is clearly going to be imperfect, so for these:

• OS_ReadLine is deprecated on all 32 bit systems, and is not to be used (supported only for legacy reasons).

(but, supported correctly – see nemo’s suggested code above, there’s no excuse whatsoever for the current embarrassingly terrible code)

Does this sound acceptable?

For comparison, by the way, here is the code to OS_HeapSort:

HeapSortRoutine
        Push    "r1,r7,lr"
        AND     r7, r1, #2_011 :SHL: 29
        BIC     r1, r1, #2_111 :SHL: 29
        TST     r7, #1 :SHL: 30
        ORRNE   r7, r7, #1 :SHL: 31
        SWI     XOS_HeapSort32
        Pull    "r1,r7,lr"
        B       SLVK_TestV

(weirdness because of emulating a long-time bug where setting bit 30 also set bit 31)

Oh jeez…

OS_SubstituteArgs (which can take a flag in the top bit of R0):

XOS_SubstituteArgs_code
        Push    "r5,lr"
        MOV     r5, #0
        SWI     XOS_SubstituteArgs32
        Pull    "r5,lr"
        B       SLVK_TestV

NO COMMENT!!!