Time for a Qemu port

QEmu also comes in a variant for lower-level virtualisation called KVM, which allows greater support for the execution of code using native execution.

IMHO, this is not a variant, but the same Qemu using KVM as a backend,

However, this requires the native system to be able to execute your code – and the Pi5 doesn’t support Aarch32 for non-user mode so privileged mode execution would not benefit from this. As the vast bulk of the OS by necessity executes in a privileged mode, the benefit here to the Pi5 is minimal.

I would say it could be true or not, since HYP mode should support 32bit. And even if it’s not the case, it’s possible to combine KVM with TCG, for example with the Cortex-A57 CPU type. But does it really work? Needs some testing. Anyway, all of this will only be possible up to Cortex-A78. Then, it’ll be TCG only.

KVM is included in standard on the current Raspberry Pi OS 64bit. So it’ll be easy to test all the options, with for example the 32bit version of Raspberry OS as a VM.

Nota: for me, RPCEmu is one option, but not the best option. Capable, but not modern. Fast, but slow on ARM Linux. Cool, but with devs reluctant to implement more integration features.

IMHO, this is not a variant, but the same Qemu using KVM as a backend,

Yeah, you’re right these days. I’m showing my age – it used to be a completely separate work which was subsequently rolled in.

I extracted the device tree from QEMU 6.2.0’s ‘virt’ target (defaults to Cortex A15 CPU)

It doesn’t look too bad: a single Cortex A15 core, some RAM (I chose 128MiB in the VM config, but could be anything), 256MiB ROM, a PL011 UART (Pi uses that), Cortex A15 interrupt controller (I think we support that, if not change to one we do), an ARMv7 timer, PCIe and some other stuff (virtio-mmio, GPIO) that can probably be ignored. Everything else can be added via options, this is just the defaults with no flags set.

That looks like enough to get a basic console-based system booting to begin with, at predictable if not fixed addresses. Then drivers gradually brought up, eg by adding in hardware like IDE drives that RISC OS already knows how to talk to.

Yeah, you’re right these days. I’m showing my age

Not yet :)

Then drivers gradually brought up

VirtioVsock can be so powerful…

since HYP mode should support 32bit

EL0 can support AArch32, EL1 is AArch64 only, but EL2 is? No AArch32 at all? AArch32 with limitations (that part Qemu can manage)? Thst’s not so clear for me…

Anyway, new processors are 64bit only.

TCG can be very fast, but will need to iron out some bugs in system/module/apps that could harm the performance of the JIT engine. After that, Virtio drivers could really provide massive acceleration to RISC OS.

What’s interesting for me, is the size of RISC OS. With a cool use of VirtioFS and VirtioVsock, we could imagine to ba able to open a draw file directly from Linux on an Artwork session running inside its own RISC OS VM.

(Even if for me it’ll be full screen sessions of RISC OS)

EL0 can support AArch32, EL1 is AArch64 only, but EL2 is? No AArch32 at all? AArch32 with limitations (that part Qemu can manage)? Thst’s not so clear for me…

Only EL0 can have AArch32, everything else is AArch64.

Anyway, new processors are 64bit only.

Correct, so probably best to focus on the translation side.

One problem might be the need to run a Linux + QEMU which may require a bit of memory, so one recomendation (especially to run this on 4GB or 2GB PIs) is to look for a reduced size Linux Distro instead of a full blown Ubuntu etc. Obviously on more capable systems or 8GB PIs this is not a problem.

What’s interesting for me, is the size of RISC OS. With a cool use of VirtioFS and VirtioVsock, we could imagine to ba able to open a draw file directly from Linux on an Artwork session running inside its own RISC OS VM.

When I was experimenting (before starting to write my own hypervisor) I managed to have a RISC OS run on a QEMU (but no desktop, just command line and over serial), so I think there will be quite a bit of work on making it boot in full first (I haven’t looked at the details, so I may be wrong). However on most modern systems like AMD Zen3 or even more modern, reading files was already crazy fast, without VirtIO.

Only EL0 can have AArch32, everything else is AArch64.

I’d expect that QEMU compiled 32-bit (A32) running on a 64-bit Linux would have the same performance as running on 32-bit Linux, and only marginally worse than Linux+KVM. QEMU itself is a user app, so it’s only running in user mode (no special privs needed, not even sudo). In EL0 it can happily use A32 instructions (in fact, it has to, as it can’t switch modes), so running RISC OS is mostly a 1:1 translation. The supervisor mode instructions (SWI, MMU manipulation, etc) have to be emulated, but those are a small proportion of executed instructions.

If you build QEMU for 64-bit Arm (A64) then it has to translate A32 instructions to sequences of A64. Not sure of the performance of that, but since A64 is faster maybe the hit isn’t too bad.

If you’re using KVM then I suspect you can’t make use of EL0-only A32, but KVM may not be a whole lot faster than A32-on-A32 emulation which can.

When I was experimenting (before starting to write my own hypervisor) I managed to have a RISC OS run on a QEMU (but no desktop, just command line and over serial), so I think there will be quite a bit of work on making it boot in full first (I haven’t looked at the details, so I may be wrong).

Is this work available anywhere?

I do agree that a decent useable emulator would be nice indeed!

I do like RPCEmu but it has a few draw-backs like no mime map (not even optional or for dedicated extensions), no scroll wheel, asks me if I want to quite even after RISC OS shutdown, only old CPU (i.e. some new things needing VFP or the likes does not work), ini file should allow to set the IP and select the ROM and hostfs etc. to allow faster swap of different OSses…

But the idea to just have some RISC OS app window inside the host window which AFAIK QEmu can offer – so Pyromaniac is the most interesting one since it supports different host OSses – so that I can take it with me on my laptop e.g.

But for now I am absolutely happy to use RPCEmu when travelling and is some other cases where I want to try things and not mess with my main system.

Only EL0 can have AArch32, everything else is AArch64.

Perhaps. Probably. To be honest, nothing is clear in ARM v8.3 documentation. And nothing is clear in Qemu documentation, where it’s told that you can use KVM AND Cortex-A15 CPU model (who is… 32bit) at the same time.

Could it use KVM for 32bit in EL0 AND TGC for SVC translation? Or something is missing in the documentation? Perhaps. Anyway, as David Gees told in another thread, we don’t care, as ARM is/will be 64bits only.

Reading files was already crazy fast, without VirtIO

For once, you don’t get the point Paolo :) VirtioFS is a good way to get easier integration with an IXFS like shared filesystem. And VirtioVsock is a perfect solution for fastest as light host/guest interaction…

See my point? To profile RISC OS for heavy modules and to implement then on the host side. AKA, not to translate RISC OS to 64bit, but to begin converting some modules to C and 64bit. And then/if a 64bit kernel will be available to use a VirtioVsock like interface for 64bit modules.

Nota: VirtioVsock could also be used for a bridge for 64bit RISC OS code. This tool is just fantastic (if it’s a zero copy system).

Nota: JFI, Druck is right saying there are really some strange ex-Acorn ARM wizards here that can do things no one can understand :) And yes, a 32bit EL0 Aemulor could be done, and it will work on 64bit Raspberry Pi OS. But it’s not a viable long term solution… at all.

Nota 2: another work will be to profile RISC OS to adapt 32bit code for JIT emulation. Some ‘patterns’ are really not good for a JIT engine. To be able to remove them would be make RISC OS quite unique, again.

Of course, with an accelerated Virtio-VGA-GL driver, an optimized Virtio-SCSI driver (for direct access to NVMe drives!), VirtioVsock bridge for Linux printing, etc. RISC OS on Qemu – while slower for CPU – could be much faster for anything else. A new solution can only succed if (really) faster (NVIDIA CEO).

A new solution can only succed if (really) faster (NVIDIA CEO)

I think something built for something that exists, and will continue to exist and be developed, is more likely to succeed than something that is reliant on something that is no longer produced.

It will be used, but it’ll be only a clear success if – and only if – it’s much faster. Disruption, man! :)
Look at VirtualRPC: a very good product, but not enough to convince us stopping using Iyonix PC and later hardware.

What’s interesting for me, is the size of RISC OS. With a cool use of VirtioFS and VirtioVsock, we could imagine to ba able to open a draw file directly from Linux on an Artwork session running inside its own RISC OS VM.

I’m really not sure that the ‘size of RISC OS’ is especially relevant. As machines have got larger, it’s kinda irrelevant what size RISC OS things are. They’re tiny by comparison to most things, but then most of the machines that you’re likely to use these days have lots of memory and lots of disc space, so there really isn’t much benefit. Plus, if your goal is to open a Drawfile then you have a sufficiently niche usage case that you probably don’t really need to use RISC OS.

… we could imagine to ba able to open a draw file directly from Linux on an Artwork session running inside its own RISC OS VM.

Reading this, I was reminded of my much earlier experiment with running BASIC files directly from macOS – https://share.gerph.org/s/EJbJvyJO0KHQIq5 – which I created in February 2022.

So I created a small configuration file, and a set of scripts to do the same thing with Drawfiles. It’s the same sort of problem, and relatively easy to do. Only took about 40 minutes, because it’s very hacky but… yes, launching a Drawfile and running RISC OS within macOS.

https://share.gerph.org/s/QdoHi1HYWzHSeHn – regular video

https://share.gerph.org/s/p9kUU2Jzlu4pLlw – scaled down video

For those that don’t want to follow the links (or can’t see it for accessibility reasons), this shows a macOS Finder window on a directory which holds a few Drawfiles.
It has the ‘Automator’ configuration in the background, showing the configuration file that invokes the Pyromaniac system.

I select the CenterParcs,aff file, and use the menu to select the Quick Action ‘Open Drawfile’.

This then starts up RISC OS Pyromaniac using a dedicated configuration file, starts the desktop, and opens !Draw with the file requested loaded.

All this could be done better, but… that’s just a few configuration files and a little bit of fiddling and it does the job. This could almost certainly be done with other emulation systems – including RPCEmu – with some configuration. But RISC OS Pyromaniac is intended for this sort of integration and has host features to allow it.

I’m really not sure that the ‘size of RISC OS’ is especially relevant. As machines have got larger, it’s kinda irrelevant what size RISC OS things are.

My idea is to use RISC OS + emulator as a container to distribute software on Windows/macOS/Linux/whatever. When you know the size of some empty Java containers (several hundred of megabytes) and their speed (not so “closed to native”) the RISC OS choice is not so stupid. At least for desktop applications.

Plus, if your goal is to open a Drawfile then you have a sufficiently niche usage case that you probably don’t really need to use RISC OS.

My goal is to open a Drawfile from the host system in the emulator. VirtioFS could be cool here, since you can define a network FS just for the file you wan’t to pass to Qemu. So the VM will be able to read and write the file without getting access to the whole host fs. Today, I use a small tool that copies the file to the hostfs directory, and then copy it back the the host system. I use the same kind of trick to print from RISC OS. It could be much cleaner with VirtioVsock.

Nota: Automator trick is cool too :)

My idea is to use RISC OS + emulator as a container to distribute software on Windows/macOS/Linux/whatever. When you know the size of some empty Java containers (several hundred of megabytes) and their speed (not so “closed to native”) the RISC OS choice is not so stupid. At least for desktop applications.

The Windows distributable of RISC OS Pyromaniac is an executable plus directory, which you can customise to add in your filesystem requirements.

The macOS distributable of RISC OS Pyromaniac is similar, but even more contained because macOS apps are more like RISC OS ones.

The Docker distributable of RISC OS Pyromaniac contains everything you needs and can offer you an X11 environment to give you WxWidgets or GTK interfaces on to the emulator.

The RailPro demo that I produced for the presentations were all done within a single Docker container which happens to be launched with a routed VNC session – it could just as easily run locally.

So I guess I’m glad that you’ve got the same idea as what I’ve already implemented.

My goal is to open a Drawfile from the host system in the emulator. VirtioFS could be cool here, since you can define a network FS just for the file you wan’t to pass to Qemu. So the VM will be able to read and write the file without getting access to the whole host fs. Today, I use a small tool that copies the file to the hostfs directory, and then copy it back the the host system. I use the same kind of trick to print from RISC OS. It could be much cleaner with VirtioVsock.

The Drawfile in my example is being opened from the host system in the emulator using a mapped directory. The configuration that’s used is just a shell script that maps the file from the host filesystem into the emulator. It actually maps the parent directory, because I don’t have file mappings at the moment – I haven’t needed them as yet.

So if you edit the file and save it, those changes are in the host file system – there is no copying around, as the file is mapped by the directory mapping. If you saved it under a different name, it would appear in that directory with the new name – that’s something you couldn’t do with a file mapping, but it has its own handiness.

The script that does the mapping looks thus:

export PATH=$PATH:/Users/charles/scripts
for f in "$@"
do
	if [[ "$f" =~ ,aff ]] ; then
        dir="$(dirname "$f")"
        leaf="$(basename "$f" ,aff)"
        cd /Users/charles/projects/RO/pyromaniac
        pyrodev --config-file draw.pyro --config filesystem.native_directory_mapping="$dir"='$.Native' --set-variable "Native\$File=\$.Native.$leaf" > /tmp/pyro.log 2>&1
    fi
done

The Native directory mapping performed there allows the emulated system to see, at $.Native the directory from the host system. This is a relatively normal thing in Docker. By a similar token you could make printing ‘work’ by mapping the file that you’re intending to spool out of the system, eg --config filesystem.native_directory_mapping="$PWD/spool=$.HostPrinter". Alternatively the system could be extended so that the file mapping spooled the file to the host filesystem, before invoking lpr on the file when it’s closed. That’s probably not so difficult in RISC OS Pyromaniac.

The configuration file loads up a number of the modules and sets up how I wish the system to boot end then ends with:

execute:
  - command: desktop -file $.demo.!LaunchDraw

Where demo.!LaunchDraw contains:

Do Run $.Demo.!Draw <Native$File>

Which opens the Native$File that was set in the emulator environment.

The system could be suitably cut down so that nothing other than Draw was present and you had not other applications accessible on the system. And you could compile Draw to close when the window was closed, which would then exit the desktop and thus exit the emulator. These are all refinements, but the prototype works, and is doing – essentially – what you want.

It’s reassuring to hear that the functionality that I have implemented because I feel it is useful for cases like this, is seen by yourself to be useful as well. But also remember that there is no reason why similar could not be done with the existing RPCEmu codebase. If you wanted to provide a more integrated solution with the RPCEmu system, you could.

Yes, by all means think about how you could do better with the different emulator, but don’t believe for a moment that you need to create another emulator to get this behaviour – adding it to the existing RPCEmu is possible, and the actual behaviour you requested is easily provided by RISC OS Pyromaniac. There are many reasons for which QEmu might be suited to running RISC OS, but the sort of behaviour you are asking for does not require it.

If Pyromaniac is ever released ¹ it’s worth discussing, until then however.

Whereas a port to one of QEMUs virtual platform seems a plausible proposition. It’s about the level of complexity as a port to any other arm32 bit of hardware, of which there’s been about a dozen people contributing to half a dozen ports already.

¹ In a form that allows people to do all that is claimed, under a licence that permits continued development regardless of whom is working on it.

Given the option of something open source with a large contributing user base that’s in constant development or something else, I’d generally plump for the large-user-base-open-source thing if you want it to have any kind of traction long term. It may not do everything you want it to, but so many things have just died through letting perfect be the enemy of the good enough. QEMU is there, it’s been there for ages, and it will continue to be there for ages as many people are invested in it.

Given the option of something open source with a large contributing user base that’s in constant development or something else, I’d generally plump for the large-user-base-open-source thing if you want it to have any kind of traction long term. It may not do everything you want it to, but so many things have just died through letting perfect be the enemy of the good enough. QEMU is there, it’s been there for ages, and it will continue to be there for ages as many people are invested in it.

While I agree in general, it is not clear that this strategy generally leads to an acceptable outcome. Open Source projects, no matter if small or large, sometimes go into directions that are contrary to user’s interests, especially for niche users. RISC OS users are so small in numbers that they will always be niche users of such Open Source projects almost by definition.

See GCC and its direction for the ARM backend. At the end of the day, you have to fork-and-maintain-your-own-stuff if the direction changes into something incompatible with your needs. At this point in time, it likely gets more expensive than if you would have started to roll-your-own instead. Because if you roll-your-own, you only implement what you really need, which is – depending on the size of the project – one or two orders of magnitude less complex. Which also makes changes to suit your needs a lot easier. Don’t forget that it is sometimes nearly impossible to get changes for niche needs into upstream.

It’s basically a facette of the old discussion “make or buy”. You only know much later if your decision was right or wrong.

If Pyromaniac is ever released 1 it’s worth discussing, until then however.

It is available in the RISC OS build service and the RISC OS shell. Those are released and available to anyone as a service I provide for free.

1 In a form that allows people to do all that is claimed, under a licence that permits continued development regardless of whom is working on it.

If you want a copy, request access. When I feel confident in having people use it, I’ll make it more generally available. There currently isn’t any interest in it, from anyone who has requested access, so I genuinely believe this to be niche and not something that people actually care about. But that won’t stop my talking about the fact that it can do these things, and reiterating when people say “wouldn’t it be great if…” that “yes it’d be great, and that’s what I’ve done”.

I know you don’t want to use something unless it is open source, but it is available on request. All you have to do is ask. To say that it’s not released because you have to ask for a copy is … denying the existence of something that clearly does exist and can be used by people. Everyone has their own opinion but dismissing things because they don’t conform to your viewpoint… well, I guess you’re entitled to that view.

RISC OS Pyromaniac provides a way to try out and experiment with a variety of ideas quickly and easily. The example above for the DrawFile was a ‘this is what I’d like to be able to do’, and RISC OS Pyromaniac was, after a small amount of configuration to demonstrate what that behaviour might look like in a real system, doing the real thing.

But that won’t stop my talking about the fact that it can do these things

Or “oh lookee, another OS bug and I can give you so much detail on what’s going on that your head will spin.”

😂

dismissing things because they don’t conform to your viewpoint…

…is the nonsense of religion.

If you want a copy, request access.

I’m not interested not because I’m not interested, it’s a very interesting achievement. But I don’t speak Python and my PC is about able to run XP. Asking it to do much more would involve smoke and unpleasant smells.
So, mentally I think it’s an impressive achievement.
Technologically… I’ll get back to you if I ever get a machine made in the 21st century.

and RISC OS Pyromaniac was, after a small amount of configuration to demonstrate what that behaviour might look like in a real system, doing the real thing.

Which is absolutely the best way to prove a point. 👍

I’m still reading up what Pyromaniac is all about, and definitely finding it interesting. I’m not sure yet whether I’m interested in using it, but I’ll continue reading and may have some questions before biting any bullets. I do have an M1 Mac mini that I presume I could run it on. If I can have a RISCOS desktop running on the Mac, able to run !Draw, !Zap, and my own BASIC desktop apps, at any reasonable speed, I shall be very interested indeed.

I know you don’t want to use something unless it is open source, but it is available on request. All you have to do is ask.

As an observer rather than interested party, I can see where people might be coming from.
Another viewpoint is that saying something is freely available and then saying all you have to do in order to take a real look is ask, is like opening a library and then telling people they need to ask for a list of books and select some before you pass them out through a hatch.
I can see why some people have expressed less than ecstatic viewpoints.

See GCC and its direction for the ARM backend. At the end of the day, you have to fork-and-maintain-your-own-stuff if the direction changes into something incompatible with your needs.

I don’t think this is a particularly good comparison. Here we’re talking about implementing a generic architecture in QEMU – basically a “Pi-something” – that’s easy to target with 32bit RO. All manner of machines are implemented in QEMU – some more niche than others, they don’t just fall by the wayside. Think about MAME as an example of something that’s open source that maintains a HUGE number of architectures, some of which are incredibly niche… Anyway. The requirements for a compiler that targets RO are rather different from just implementing a platform that it can run on. GCC isn’t maintained with compiling for RO in mind. QEMU is maintained with providing machine platforms in mind.

Still, the bigger problem is if people don’t get off their butts and do something it’ll all go down the drain anyway, so you (not you personally!) may as well pull the thing off the shelf that mostly works rather than trying to maintain yet another fork of something… Again, don’t let perfect be the enemy of good enough.

So I guess I’m glad that you’ve got the same idea as what I’ve already implemented.

:)

I know you don’t want to use something unless it is open source

It’s a bit more complex than this. I must secure the bits I distribute with my commercial work.

I tried to use RPCEmu, but my integration tools are more tricks than tools.
I asked for some missing integration features and faced a big NO.

So I decided to use BBC Basic to make two versions of each application. One for RISC OS, one for Windows.
Problem: BBC Basic for Windows has an huge unsolved problem with Windows Defender.

So I finally switch only to RISC OS.
Just before it war nearly impossible to find any Raspberry Pi…

Not an easy problem.

I don’t think this is a particularly good comparison. Here we’re talking about implementing a generic architecture in QEMU – basically a “Pi-something” – that’s easy to target with 32bit RO. All manner of machines are implemented in QEMU – some more niche than others, they don’t just fall by the wayside. Think about MAME as an example of something that’s open source that maintains a HUGE number of architectures, some of which are incredibly niche… Anyway. The requirements for a compiler that targets RO are rather different from just implementing a platform that it can run on. GCC isn’t maintained with compiling for RO in mind. QEMU is maintained with providing machine platforms in mind.

I’d say it’s easier than maintaining a port to a ‘Pi-something’:

1. QEMU supports a large variety of targets. You get to choose which is easiest to port to. It is quite possible that easiest is also the simplest and the one likely to stick around the longest (QEMU generally doesn’t drop targets, but picking a popular platform makes that less likely, and the generic platforms are more popular)
2. Porting to a specific QEMU platform is easier than hardware because you can just look at the source code (and its logging/tracing output), rather than hoping the silicon vendor’s manual makes sense after being translated from Chinese
3. QEMU has integral gdb support, so you can use that to debug RISC OS code, rather than just having to flail around in the dark when you can’t see anything on the UART. I managed to dump out the device tree upthread by booting QEMU and running 3 gdb commands.

Basically targeting a QEMU-supported platform is a bit like porting to the ‘PC’ platform if we used x86. It’s so popular that there is not much chance of it going away for a long long time.

While the various API reimplementations (Pyromaniac, the ‘Linux port’, various microkernel/hypervisor ideas) are interesting, they’re all one-person projects and without an obvious roadmap for adoption by both users and developers. That may become clearer over time, but there isn’t an obvious candidate for everyone to jump on right now.

Meanwhile, a QEMU port is ‘just another ARM board’ where the ‘hardware’ is maintained by other people, who do it better than we ever could. Users and developers can carry on using RISC OS just how they did before, only with the limitations of running on the right machine removed.