Porting ARM32 BBC BASIC to ARM64

I was going to put this in Aldershot … but it does raise a point : Acorn BBC BASIC is intrinsically tied to ‘assembly language only’, but also to ‘ARM32 only’. Yet, a port is needed (to ARM64, and one day to RiscV 32/64). ARM BBC BASIC was created for the ARM Evaluation Kit, so existed before even Arthur, so is in many ways a fundamental part of RISC OS.

So … is such a port possible (given the esoteric nature of the ARM32 code)?

At some point, there will be a need to do this conversion. The current BBC BASIC code is tied to ‘assembly language’, in that converting it to C is pretty much impossible (at least in an efficient manner), because it relies very heavily on passing information in the status flags across many subroutines. A C conversion of the current code would need to use an emulated status flags global variable, which would need to be manually updated at each stage (with a lot of use of carry, so lots of complex code at each step to emulate ARM32 carry).

One challenge to converting it to ARM64 assembler would be the extensive use of ‘TEQ’ to do comparisons without altering carry…
– TEQ is EORS (discard result),R1,R2
– ARM64 does have EOR
– but not EORS
– ARM64 EOR (immediate) uses a bitmask immediate, so it cannot encode EOR X1,X2,#0 (as cannot encode all zeroes or all ones)
– but EOR X1,X2,XZR can be used for that case
– many numbers ‘TEQ’d’ also cannot be encoded as a bitmask immediate, so there is a need to MOV the number into a register first before the EOR
– once the above is done, one could just do ANDS X1,X1,X1 to set flags as if a TEQ has just been done
– but no! On ARM64, ANDS just clears the C flag (so the flag-preservation of ARM32 is no longer a thing here).
– so, use of MRS and MSR may be needed!

Essentially, each TEQ needs to be fully understood in terms of impact to flags etc, and a specific replacement needs to be decided upon – that alone is a lot of work!

The above is one example where it will be a challenge rewriting BBC BASIC to ARM64
– others include the stack, because it will need 64 bit pointers, so the entire size / alignment of the stack needs to change because BASIC / BASIC code may be placed anywhere in the 64-bit address space.
– (another ARM64 stack issue is that the stack needs to be aligned to 16-byte alignment at all times, which removes much of the clever usage of the stack that BBC BASIC ARM32 uses)
– another issue is that ARGP is set up so that the BASIC internal variables etc are accessed by [ARGP,#-number], but ARM64 doesn’t have good enough ‘#-‘ addressing modes without writeback, but does have a very good #+ addressing mode without writeback, so ARGP will need to be shifted to before the variables list, so ARGP=#&8000 …
– but the VCACHE starts at ARGP=0, so many changes needed there too!

Has anyone tackled this issue thus far? Any thoughts?

Acorn BBC BASIC is intrinsically tied to ‘assembly language only’, but also to ‘ARM32 only’.

You are mistaken on both these points.

BBC basic is already happily running on arm64, with two implementations written in C.

http://brandy.matrixnetwork.co.uk/

https://www.bbcbasic.co.uk/bbcsdl/index.html

It would make more sense to base any 64 bit version on an existing C implementation, particularly as if there is ever a 64 bit RISC OS, it’s OS API is likely to be almost as different to 32 bit RISC OS as the cross platform code is.

“You are mistaken on both these points”

This is why I said Acorn BBC BASIC is tied to assembly language only, and ARM32 only (emphasis on Acorn) – as I was referring to the BBC BASIC that comes with RISC OS, as opposed to the reimplementations of BBC BASIC. I guess I was thinking about not having to discard the original BBC BASIC source code entirely (which it seems we logically* have to at some point

I of course agree that a 64 bit RISC OS API would be very different from 32 bit RISC OS, although there would be some similarities – and the Acorn RISC OS BBC BASIC is quite well abstracted (BBC BASIC for ARM evaluation system loads and runs without modification on RPi4, with error handling working, printing the intro string and >, OS_ReadLine etc, and only later throws errors due to 26-bit code). ARM BBC BASIC existed before Arthur even – It is surprisingly well abstracted (and probably influenced how early RISC OS was developed). The MOS emulation itself was probably very useful before ARM26 MOS itself existed!

Edit : RPi 5 changed to RPi 4 re : BASIC 1.0

ARM BASIC was written by Sophie Wilson who both wrote the original BBC BASIC and designed the ARM instruction set, so there will not be a single clever trick you can do with it that will have been missed, including all the problematic 32 bit integer and 5 byte float equivalence stuff we’ve discussed elsewhere. None of those tricks are suitable for the rather generic and boring ARM 64 bit instruction set, which is designed for compilers to churn out rather than assembler wizards to hand craft.

Taking a C implementation where the functionality has already been extracted from the clever coding, will save a lot of time, and make it hell of a lot easier to work with whatever OS API is chosen, and most importantly; to enhance further with modern programming constructs.

…the rather generic and boring ARM 64 bit instruction set, which is designed for compilers to churn out rather than assembler wizards to hand craft.

Very succinctly and accurately put – and the basic reason why I decided to retire when I did…the future at ARM was beginning to look boring!!!

I of course agree that a 64 bit RISC OS API would be very different from 32 bit RISC OS

Would it though? The survey of the current API says much would look and feel the same. If you stick to _swix in C, SYS in BASIC, or swi in python you’d see identical source in most cases because the SWI maps to processor registers any with pointers in get correspondingly wider in a 64b world. For example, OS_Write0 takes a pointer in r0, which naturally maps to x0 in AArch64.

The main gotcha in API terms is when we put pointers in memory, because now you want an 8 byte quantity in a space reserved for 4. The above linked document enumerates those problem ones, and it’s only about 50 – work to do, but tractable.

Would it though?

Umm, yes?

Because if you’re rewriting RISC OS in ARM64, you’re basically kicking the can a little further down the road and remaining heavily tied to specific processors.

And if you’re looking to rewrite it in C, I can’t help but think that it is going to be a nightmare to try to get C interacting with a highly assembler-specific OS that has utterly no concept of a calling convention other than “whatever we decided this SWI requires”, not to mention weirdness with the flags for some of them. Plus doing it like that will still remain tied to specific processors.

To be honest, I can’t help but feel that any implementation that doesn’t try to fix the many architectural issues of the past would be a fool’s errand. This is, for most software, a breaking change (only pure BASIC will escape unscathed), so it’s a good opportunity to fix all of the bad bits and jiggle the Wimp to support multiple cores (not to mention pulling the task switching out of the Wimp).
Yes, it’s going to be a huge task, but then so would be converting the OS as it is (in assembler) to ARM64.

This is irrespective of whether or not some LLM is used. If a ‘bot is not used, you’ll spend much time going through each line one by one to translate it. And if you use a ‘bot you’ll spend just as much time going through to ensure the output is coherent and it isn’t blithely screwing things up.

Oh, and don’t forget that there’s no reason why a shiny new OS based upon RISC OS principles but written in C can’t be backported to existing 32 bit systems.
Look at Linux, Minix, etc as inspiration.

I have a nagging suspicion that this is all cloud cuckoo land, whichever option one is considering.

Meh. Having spent years on trial and error, mostly error, I’ve come to the conclusion that, like the Black Ship in the Hitchhiker’s Guide to the Galaxy, the OS is best left alone, as much as possible.

Legacy code that expects to have to manage context changes, application entry and exit, callbacks, etc. has to essentially run under a single thread (actually a single legacy SVC stack).

New code, though, can run on multiple cores, have transparent context changes, and communicate via simple mechanisms (pipes, basically buffers, but the tasks sleep while waiting for new data or space).

They could even be 64 bit, but don’t hold your breath. (I did work on a 64-bit OS with a wierd mechanism and 32-bit VM, but much of it can be repurposed to be more RO-like while more secure.)

Right now, I can’t even get RO to build.
https://www.riscosopen.org/forum/forums/11/topics/19461

Asking without any relevant knowledge, but would it make sense to create an emulator to run existing BBC Basic code rather than create BBCBasic64? I’m thinking that new applications would be written in something more modern and portable like Python, C++, Rust(?), etc., so we would only need this for supporting older code to stop it being lost.

32-bit emulators are ten a penny.

Run one within another. There. 64 bits. Easy. 😂

“ARM BASIC was written by Sophie Wilson who both wrote the original BBC BASIC and designed the ARM instruction set, so there will not be a single clever trick you can do with it that will have been missed, including all the problematic 32 bit integer and 5 byte float equivalence stuff we’ve discussed elsewhere. None of those tricks are suitable for the rather generic and boring ARM 64 bit instruction set, which is designed for compilers to churn out rather than assembler wizards to hand craft”

Agreed (this is one of the reasons I wouldn’t want to ‘lose’ the original source code) – it is a work of art, and tied to the creation of RISC OS, and even the ARM processor itself (quite a lot of the functionality lost during the switch to ARM64 were things that were very useful in BBC BASIC, such as RRX, shifter carryout in general, TEQ to compare without messing with carry): these are amazing features for assembly coders, but C barely or even doesn’t use these, so compilers didn’t need them, do they were dropped for ARM64 😬

Regarding emulation, I started work on an ARM32 emulator a few months ago; the platform was ‘a RISC PC with an ARMv8 processor’; I started by getting BBC BASIC to work (because that is a good way to ensure you have shifter carryout sorted, reducing the number of annoying bugs when trying to boot RISC OS itself). I managed to get it to boot to desktop, and learnt a lot doing so!

Back to Acorn BBC BASIC – I want to learn how it really works, and want to learn ARM64 as well, so I have started work on an ARM64 port as of 2 weeks ago (attempt 2; attempt 1 was very experimental / helped me understand the code). I am about 50% done so far. I am writing it on Raspberry Pi OS on a Pi 5, and currently have 1.5x the speed of ARM32 BBC BASIC (the aim is 2-3x the speed on the Pi5, as the Pi5 is 2-3x faster than the 4). I am not using CBZ, CBNZ, TBNZ, TBZ, CCMP or CSEL yet, but am leaving comments in the code where optimisations will be possible. I have kept the code structured the same as the RISC OS ARM32 source. I also have an unoptimised TEQ macro, so that is another place for speeding things up. I am writing it ‘as if’ it would be on an ARM64 version of RISCOS64 in the future; it is well abstracted, so I have made macros for SWI, so I can just put RISC OS SWIs in the source code, and it works as expected. I made a simple VDU engine (enough for text, using the Zap larger characters for now).

My aim with this project is to learn ARM64 (so I may be able to help with the low level code of RO64 in the future), to learn how the ARM32 BBC BASIC code really works, so, among other things, I can overcome the challenges of making a dual 32/64-bit BBC BASIC in ARM32 code … without needing to worry (yet) about the register shortages on ARM32 BBC BASIC. Once I am able to use 64-bit ints, and can also run various RISC OS ARM32 BBC BASIC programs (with 32-bit integer support), I can then add 64-bit int support back to RISC OS ARM32 BBC BASIC!

Once this ARM64 BBC BASIC is fully / mostly complete, I will also write an EL2 hypervisor’ that (on the Pi 4) keeps the upper 4GB of ram free, hands control to RISC OS32 running at EL1, but then hands a core to BBC BASIC 64 so that I can experiment with 64-bit stuff while also having RISC OS as a development environment (it will involve a single semaphore being added to the RISC OS workspace 🤪)

Will do a bit more work now (started Array code yesterday, mostly done too!)

Edit : I should add that the ‘abstraction’ I have done uses assembly to a degree, but the I/O is actually abstracted to C functions, making it easy to compile on various OSes etc, as long as they have ARM64)

“Because if you’re rewriting RISC OS in ARM64, you’re basically kicking the can a little further down the road and remaining heavily tied to specific processors.”

“And if you’re looking to rewrite it in C, I can’t help but think that it is going to be a nightmare to try to get C interacting with a highly assembler-specific OS that has utterly no concept of a calling convention other than “whatever we decided this SWI requires”, not to mention weirdness with the flags for some of them. Plus doing it like that will still remain tied to specific processors”

I agree – although if an ARM64 port of RISCOS32 appeared, it would kick the can 30 years forward (assuming ARM64 lasts as long as ARM32 did in (ARM CPUs with MMUs etc);

My thoughts on the transition of RISC OS to 64-bit include:
– recoding the ‘core’ in ARM64
– minimising the ‘core’ code to the HAL, and the core part of the Kernel … mostly just enough code to get text on the screen; the rest could then be coded in C.
– areas of slowdown could be identified, and optional CPU-specific alternate code could be added. This would complement the C code, and allow RISC OS to run in a CPU-agnostic way, but also to have hand-optimised assembler in areas that can benefit from a major speedup form a small assembly loop.

Of course, the API, memory allocation, workspace layout etc needs to be decided upon, before any work can be done there!

assuming ARM64 lasts as long as ARM32 did

I think that’s a very shaky assumption. “generic and boring” was David’s very apt description of ARM64. ARM26 was, in its own way, ground breaking; and ARM32 was a largely non-deal-breaking upgrade of it. That it lasted so long is only a bit surprising; for a generic and boring instruction set to last that long without deal-breaking “upgrades” would be very surprising indeed.

ARM64 is – as David wrote – designed for compilers, not for human wizards. Don’t try writing in assembly language for it (which is why I really wasn’t interested in writing assembly language manuals for it). And if you’re writing in C (or, preferably, some better higher language – has anyone designed one yet? It’s bloody needed) then you’re not tied to the particular architecture.

“assuming ARM64 lasts as long as ARM32 did

I think that’s a very shaky assumption. “generic and boring” was David’s very apt description of ARM64”

I do agree mostly; in a way it will be good if ARM64s dominance is challenged, because it does take a fair amount of the ‘enjoyment’ of ARM32/26 away. I do want to find out if there are ways to use the characteristics of the instruction set for clever optimisations, though (as an example, losing shifter carryout and TEQ is annoying – but some things, such as MOVK, CBZ / CBNZ are good additions to ARM64, and things like ANDS wiping the carry flag may be useful in certain ways (a trivial example of CBZ is reading through a zero-terminated string;

Is quite succinct, and the fact it doesn’t touch any flags at all can probably be used in clever ways. Other things like the bitmask immediates are interesting, as are things like TBZ/TBNZ, and the use of a zero register.

It is sad that Apple essentially dominated ARM, and changed the direction of ARM during the shift to 64 bits. And also sad that ARM32 is being slowly deprecated.

I have most of ARM BBC BASIC running on ARM64 now, and in reality I am doing it in my spare time, as a ‘hobby’, not as work, and it is already fast on ARM64. The worst case scenario is a fast ARM64 BBC BASIC V/VI interpreter for ARM64, which follows the design of ARM32 BBC BASIC, so may be useful at some stage for RISC OS. Plus I get to learn ARM64, which may be helpful for low level 64 bit kernel coding in some areas. Understanding the BBC BASIC code means I could also port it to RISC V 32/64 in the future (I prefer assembly over ‘higher level languages’, but have finally started doing some C as part of the project)

or, preferably, some better higher language – has anyone designed one yet?

I can’t find it now, but someone managed to make a very basic OS in C#. I use the language regularly but I can’t imagine e.g. rewriting RISC OS in it!

Edit: Found it!

I don’t know C# – how much of an advance on C is it? I’m imagining something that makes C look very, very primitive… sadly, only imagining it, not even visualising what it might look like…

Quick and dirty example:

Console.Write("What is your favourite OS? ");
var fav = Console.ReadLine();
if (fav.Contains("RISC"))
{
  Console.WriteLine("Good choice!");
}

Even the simple stuff like .Contains can be much easier than the “C way”. In fact the second and third lines could be condensed to if (Console.ReadLine().Contains("RISC")). You could also specify an IgnoreCase argument to the comparison.

Current status (ignore the missing parts of some letter ‘g’s – I just put some 1-bit characters into my VDU code (which is in C, and is just enough to get characters on the screen for testing – it is not the RISC OS Vdu code). Got arrays mostly working today – have a few bugs in string arrays to sort. Also added colour as well (via the ‘proper’ methods – using the BASIC COLOUR command, which does VDU 17, (colour) to set the colour!

(this is on a Pi5, and is an ARM64, except for the VDU code, and IO, which goes to some C code using a ‘SWI’ macro in the assembly language code – meaning I am using SWIs that conform to RO32 standards (except 64-bit registers), also meaning the code isn’t tied to any particular OS – as long as a few simple SWIs are implemented in C, then it will run fine :)

I don’t know C#

Isn’t that heavily tied into the .Net runtime?

Doesn’t it compile to an intermediate form that is often JITed into it’s final form as necessary (akin to mobile platform apps)?

Not saying it’s bad, just that maybe it’s not the best choice for RISC OS?

Quick and dirty example

That’s objects and such isn’t it? (spot the plain C programmer! ;) )
Can’t C++ do that as well?

I don’t know C# – how much of an advance on C is it? I’m imagining something that makes C look very, very primitive… sadly, only imagining it, not even visualising what it might look like…

I didn’t think I’d ever like anything coming out of Microsoft, but C# is actually a pretty good language, it’s what C++ should have been if they were able to ditch some of the C legacy and not bodge everything on top. It’s got a very clean syntax, and a large standard class library with lots of useful collection classes. It’s main disadvantage is is semi-compiled and requires a runtime instead of compiling to raw native code, and it is a managed language with garbage collection which can cause latency issues. I mainly used it on Linux with the mono runtime, although I did write it on Windows with Visual Studio. I think they were up to version 4 back then, they are on 12 now, and shovelled far too much unnecessary crap in to it.

I would not want to go back to using plain C now, particularly manipulating strings with just stdio and without a string classes. You can get most of the functionality of C# using C++14 onwards with the Standard Template Library, but it’s oh so painful compared to the simplicity of C#. Now I mainly just use Python, which has an even richer set of support libraries than C# and a nice syntax, as long as you just get over the use of white space. The only things it doesn’t do well is multi-threading and absolute performance.

Just going back to Brandy Basic, I seem to remember that Sophie Wilson was very complementary about it and it’s speed of execution on a RiscPC.

!Zap running on ARM 64 :-))