What software would you like to see on RISC OS?

Considering the work involved and hit to performance, I wouldn’t waste time going that route, stick with full CPU Emulation.

Yeah, I’d definitely start off by working on optimising the interpreter engine. But I’m not sure how much performance it would be possible to get, at least without painstakingly rewriting the whole thing in optimised assembler. If I need to rewrite it all to get acceptable performance, the time would probably be better spent working on an ARM JIT, since the work might also lead to ways to improve the other JIT backends.

The minute a JIT is added into the equation, things like MMU, VIDC and IOC timer emulation become a major headache, not to mention the complications of managing the JIT and CPU instruction caches.

Timer emulation doesn’t worry me much. All post-Archimedes machines have CPU caches, and there’s a wide range of CPU models and clock speeds, so programs which rely on precise CPU timing should be rare. As long as VIDC+IOMD timing is derived from a common clock source (so VIDC+IOMD are consistent with each other), and the rate of that clock adjusts dynamically so that on average it’s running at real-time speeds, there shouldn’t be any major problems.

The only headache I can see with the CPU instruction cache is that there’s lots of software (OS/kernel included) which fails to perform cache maintenance correctly! So the emulator can’t rely on the CP15 cache maintenance operations as a hint for when the JIT codelets need to be invalidated. Instead the best course of action might be to just ignore the cache maintenance ops and make any writes to code areas invalidate the codelets immediately. This would have the potential to slow down the code quite a bit, but it ensures compatibility with pre-StrongARM code (unless the code is reliant on instruction prefetch). However most code should be StrongARM-compatible, so the JIT can probably designed under the assumption that writes to code areas are rare – i.e. the emphasis should be on making sure writes to data areas can be handled as quickly as possible.

Writes to the page tables can probably be handled in a similar manner – ignore the CP15 TLB maintenance ops because they’re often used incorrectly, and instead have a special write handler associated with the page table pages so that any writes can update the emulator’s translation tables immediately. Ideally, JIT codelets associated with unmapped pages should be kept, helping to boost emulator performance when running Wimp tasks.

You say “major headache”, I say “interesting optimisation problem to solve” :-)

This would have the potential to slow down the code quite a bit, but it ensures compatibility with pre-StrongARM code (unless the code is reliant on instruction prefetch). However most code should be StrongARM-compatible, so the JIT can probably designed under the assumption that writes to code areas are rare – i.e. the emphasis should be on making sure writes to data areas can be handled as quickly as possible.

Reading with interest.
I initially thought “why worry about pre-StrongARM code when there are other emulators for pre-RPC machines” – then I remembered that you’re talking about an emulator/JIT that should run on all future machines, which those existing emulators may not because the OS changed at a level that breaks it.

I initially thought “why worry about pre-StrongARM code when there are other emulators for pre-RPC machines” – then I remembered that you’re talking about an emulator/JIT that should run on all future machines, which those existing emulators may not because the OS changed at a level that breaks it.

I’m not worried about pre-RiscPC machines. ArcEm is designed to be portable – I’ve done builds for 64bit and big-endian CPUs in the past, and the only assembler in the RISC OS version is the SharedSound buffer fill routine and a small helper module for scanning the keyboard. It might not be the most friendly to use (needs a Wimp frontend), and it’s certainly not the fastest (~8MHz ARM2 performance on an Iyonix), but compared to the other Arc emulators we have available, it’s the one that’s least likely to run into issues on new machines/OS versions. Just ignore the fact that I haven’t got round to making an official ARMv8 compatible release yet ;-) (UnixLib SWP issue – luckily Chris Gransden has you covered)

So the pre-StrongARM code I was worried about was code which requires a RiscPC (or works best on a RiscPC) but isn’t StrongARM compatible. Or (since we’re talking full-system emulation) OS versions such as 3.5/3.6, which people might find useful for compatibility testing.

ArcEm addendum – I’ve been thinking for a while that someone should really write a SDL frontend for it. Apart from aiding portability to new platforms, it should also help solve the problem that some of the current frontends are a bit crap (e.g. Windows version has no support for command line args or sound). Maybe the RISC OS version could benefit from using the SDL frontend too.

So the pre-StrongARM code I was worried about was code which requires a RiscPC (or works best on a RiscPC) but isn’t StrongARM compatible. Or (since we’re talking full-system emulation) OS versions such as 3.5/3.6, which people might find useful for compatibility testing.

I have a strong suspicion that people using an emulator for compatibility testing might be using RPCEmu.
What is really under discussion here is how people run as many variants of old software as possible for “daily” use on their new ARM platform. It might be convenient to support those older revisions for such test style work but I’m not convinced it’s worth a special effort.

I have a strong suspicion that people using an emulator for compatibility testing might be using RPCEmu.

Under RISC OS? Yes, I would like too.

I have a strong suspicion that people using an emulator for compatibility testing might be using RPCEmu.

Good news, because that’s what I’m suggesting we bring to RISC OS. But without a lot of optimisation and/or a JIT the performance is likely to be terrible.

What is really under discussion here is how people run as many variants of old software as possible for “daily” use on their new ARM platform. It might be convenient to support those older revisions for such test style work but I’m not convinced it’s worth a special effort.

And what I’m saying is that the old OS versions which those old pieces of software require do not perform their cache or TLB maintenance correctly. So an emulator which relies on cache/TLB maintenance operations as a hint for how and when to flush JIT codelets, optimised translation tables, etc. might run into difficulties. So in a way it becomes easier to support the pre-StrongARM code than it is to ignore it.

Good news, because that’s what I’m suggesting we bring to RISC OS. But without a lot of optimisation and/or a JIT the performance is likely to be terrible.

RPC with multiple OS revisions or something with smoke and mirrors to fool the application¹. If the former then being able rapidly switch from one OS revision to another but not having to fiddle with the Boot etc (or maintain a different “disc”) would be wonderful.

¹ Some sort of modified !Run file in an app to tell the system what it expects perhaps? Or an Aemulor style record of the application requirements.

I have a strong suspicion that people using an emulator for compatibility testing might be using RPCEmu.

I don’t. I’ve found RPCEmu on my (ancient) XP box just acts weird at times, and I see utterly no point in bothering to run RO5 under emulation when I can run it natively. :-)

For RISC OS 3.5 / 3.6 / 3.7, I use RedSquirrel on the PC. The startup prompts me which one should be loaded.

For older, it’s a bit more painful, but in use !A310Em to emulate an A310 running RO3.1 / RO2 / Arthur 1.2 (sort of), but it is itself an older application so it runs under RedSquirrel.

I have, just for lulz, run !65Host under !A310Em under RedSquirrel.

I’d find it funny to run the Windows session under VirtualPC, but that means installing XP all over again inside the VPC environment, and I don’t have enough disc space free.

That said, it is only exceptional circumstances where I bother to perform any testing on quarter-century-old versions of RISC OS. I’m more interested in modern issues (32 bit good, high/low vector, SWP, etc) than “does it still work on 3.X?”.

I have a strong suspicion that people using an emulator for compatibility testing might be using RPCEmu.

I don’t. I’ve found RPCEmu on my (ancient) XP box just acts weird at times, and I see utterly no point in bothering to run RO5 under emulation when I can run it natively.

Hmmm, “compatibility testing” + “run RO5” – I don’t think that really tests the compatibility unless it was only the emulated hardware you wanted to test against.
There is a sort of implied “legacy” element in the compatibility testing.

I’d say the main reason for running RPCEmu with RO5 is running RO5 on a handy laptop, not that I’d go out and buy one for the purpose.

Timer emulation doesn’t worry me much

Until you try coding them! Implementing an accurate VIDC raster and programmable 2Mhz timers without a high precision OS timer API that can generate IRQ’s is going to be interesting. Under emulation its a non-issue as you cycle count, with code running natively however you need to interrupt the code.

If you’re only going to support well behaved RPC Wimp apps, then you could simply not implement timers, as the two that RISC OS needs to work (keyboard and serial) can be virtualised and triggered via IRQ.

The only potential issue taking this route is the POST timing test, which would cause POST to fail, but this could be patched in the OS as is done currently to get RO5 to run under Red Squirrel.

without a lot of optimisation and/or a JIT the performance is likely to be terrible.

I benchmarked the ADFFS JIT back in 2013, Pi1 performance was equivelant to a 140MHz ARM3 or 170MHz StrongARM. The lower ARM3 performance being down to self-modifying code support. To put that into perspective, a Pi1 running the same code natively is equivelant to a 285MHz StrongARM.

Those figures are based on the JIT only recoding BL and instructions that use PC, and in the case of self-modifying code support also trapping writes to pages with code in them. If recoding all memory access instructions (LDR, STR, LDM, STM, LDF, STF, LFM, SFM) I’d be suprised if a JIT could even achieve ARM610 performance.

The only headache I can see with the CPU instruction cache is that there’s lots of software (OS/kernel included) which fails to perform cache maintenance correctly…the best course of action might be to just ignore the cache maintenance ops and make any writes to code areas invalidate the codelets immediately.

You can ignore cache maintenance instructions, with a JIT you need to trap all writes to memory and clean codelets, then perform the relevent physical cache maintenance. Under a JIT there’s little difference when handling StrongARM, ARM610 or ARM3, they all require the same cache maintenance. For example, if the JIT has seen an application run at &8000 and then the OS loads another application over it, you need to clean the JIT codelet cache for the memory range overwritten, the physical cache that cover those codelets, as well as the physical cache for the range overwritten.

With regard to TLB maintenance instructions, these can also be ignored as the TLB would be virtual with multiple L1/L2PT tables to cover CPU modes – the CPU mode being Paravirtualised. You’d perform your own TLB maintenance as L1PT/L2PT are modified or during CPU mode changes.

There’s another issue that needs covering, that specific to a JIT – codelets being interrupted. If for example a codelet needs to preserve registers around itself, but is interrupted via IRQ and then reentered, you need to ensure the preserved registers aren’t corrupted.

I have a strong suspicion that people using an emulator for compatibility testing might be using RPCEmu.

For compatibility testing, Red Squirrel would be a better platform as RPCEmu’s MMU implementation has no understanding of page table access levels and doesn’t trigger memory access related Aborts correctly.

RPC with multiple OS revisions or something with smoke and mirrors

The OS would run natively within the VM, so any OS suitable for the chipset being emulated would work.

Timer emulation doesn’t worry me much

Until you try coding them! Implementing an accurate VIDC raster and programmable 2Mhz timers without a high precision OS timer API that can generate IRQ’s is going to be interesting. Under emulation its a non-issue as you cycle count, with code running natively however you need to interrupt the code.

You can still use cycle counting + soft interrupts with a JIT, you’ll just have to be a bit careful with where you insert the interrupt triggers. Checking after every instruction would ruin performance, but not checking often enough could result in soft-lock scenarios. Checking on each branch instruction and on each (block of) memory accesses (or after a PSR manipulation) would seem to be an acceptable compromise to me.

If you’re only going to support well behaved RPC Wimp apps, then you could simply not implement timers, as the two that RISC OS needs to work (keyboard and serial) can be virtualised and triggered via IRQ.

Well-behaved Wimp apps would be the primary target, yeah.

I’m not too hot on early RiscPC games, were there really that many which relied on precise IOMD/VIDC timing? I thought most of the VIDC tricks which were possible on the Arc (e.g. mid-frame palette updates) weren’t possible with VIDC20.

The only potential issue taking this route is the POST timing test, which would cause POST to fail, but this could be patched in the OS as is done currently to get RO5 to run under Red Squirrel.

The POST timing test checks the relative speed of IOMD and VIDC by doing polling reads of the VSync IRQ bit and the IOMD timer register. If a hardware emulator can’t pass that test then its timer emulation is pretty crap IMO.

If recoding all memory access instructions … I’d be suprised if a JIT could even achieve ARM610 performance.

Them’s fighting words ;-)

Honestly, I have no idea what performance I’d be able to get. Since it’s a RiscPC emulator, ARM610 would obviously be the minimum performance I’d be looking for on the target min spec machine (TBD). But I’ve never worked on an ARM backend for a JIT before, so I have no idea how much effort will be required to get there.

RPCEmu’s MMU implementation has no understanding of page table access levels and doesn’t trigger memory access related Aborts correctly.

Which is probably something I’d look to fix if I was to find the time to work on RPCEmu, in addition to other things like the sub-standard IOMD/VIDC emulation (hello, POST hack), and whatever the thing is that stops hard discs from working (RPCEmu bug? RISC OS bug? I really can’t remember any more).

What software would you like to see on RISC OS?

A screen recorder, so that desktop videos could be uploaded to YouTube, Vimeo, et al.

You can still use cycle counting + soft interrupts with a JIT

In practice it would be very complicated to implement as it would require a JIT that can handle 1:n translation (ie 1 instruction from the source can be N instructions in the JIT equivelant), requiring a memory translation table to correct branches etc. This would make self-modifying code complicated as the new code may not fit in the gap.

Checking on each branch instruction and on each (block of) memory accesses (or after a PSR manipulation) would seem to be an acceptable compromise to me.

Possibly, but it would further hit performance.

Well-behaved Wimp apps would be the primary target

Forget about supporting T0/T1 then, you could probably count the number of Wimp apps that use timers on one finger!

were there really that many which relied on precise IOMD/VIDC timing? I thought most of the VIDC tricks which were possible on the Arc (e.g. mid-frame palette updates) weren’t possible with VIDC20.

VIDC20 is identical to VIDC, the only notible difference is the PLL based clock.

The POST timing test checks the relative speed of IOMD and VIDC by doing polling reads of the VSync IRQ bit and the IOMD timer register. If a hardware emulator can’t pass that test then its timer emulation is pretty crap IMO.

The VSync IRQ bit is not easy to implement, I’ve tried various methods but still haven’t managed to get software that uses it to sync correctly. The problem being that user code needs to continue to execute in parallel to the screen blit, for it to work correctly. Quoting my post on the subject:

A solution would be to blit one raster, then allow code to continue executing until it’s time to plot the next raster, more closely emulating the hardware. As there’s no Timer API, I can’t currently implement the required IRQ code.

Something to revisit if/when a suitable API is added to RISCOS.

Aside from complicated discussions about emulation I’m getting that what people would really like is:

• Improvements to NetSurf would be welcomed by most/all people and Internet Chat applications would be nice to have.
• An IDE would be nice but most developers are OK with the current way of doing things.
• Better multimedia capabilities (media player, digital TV tuner, screen recorder, etc.,).

Would that be a fair summary?

(I’m not counting anything to do with the network stack here because that’s already got a bounty!)

Would that be a fair summary?

That looks reasonable to me.

Regarding the first item that Glen mentions, I’d comment that we have Grapevine which provides (last time I tried it) IRC, and MSN/Skype compatibility (although they may have finally broken that too), as well as nominal ICQ (but I think that protocol has now changed). This effectively “aged out” because there simply wasn’t enough interest to justify the costs of on-going development. I sank about a further 5k into the project and was seeing fingers-of-one-hand sales, so work had to shift elsewhere. There’s no reason it couldn’t be revisited with funding, but that’s a black hole I can’t really afford to repeat :(

The thing that would really help “energise” that type of application would be if people could sort out a standard for audio input. This has been missing for far too long, really. The Iyonix had it in !AudioIn, but that seems to rely on a 3rd party module (if memory serves) and the author has vamoosed. This would open the door for skype-style voice-over-IP, but without a standard way of doing it, it seems to be perpetually “missing in action”.

I suppose the answer is “So, Andrew, get it sorted!”, but I fear that the most suitable person for the task is the one who’s already most over-burdened (Mr Lee, take a further bow!). Is there anyone else who feels that they could step up to this plate? I would be happy to support such development.

It would be nice to see Pidgin (or more realistically, Finch) plus protocol plugins on RISC OS.

(I run pidgin with ICQ, Telegram, Skype, Yahoo, VK, Hangouts and SIP chat. I deleted my Facebook account on Monday, I used to use that with it too)

The thing that would really help “energise” that type of application would be if people could sort out a standard for audio input.

…

I suppose the answer is “So, Andrew, get it sorted!”, but I fear that the most suitable person for the task is the one who’s already most over-burdened (Mr Lee, take a further bow!). Is there anyone else who feels that they could step up to this plate? I would be happy to support such development.

It’s worth bearing in mind that the people who design the standard needn’t be the same people who implement it. And for most tasks coming up with the design will be quicker than the implementation and testing (especially when you have multiple hardware platforms to update/test). So although I don’t have enough time to implement everything myself, I am happy to help out with design tasks as and when needed. I’m also happy to help with guiding/mentoring people through the implementation process (e.g. dealing with the lower levels of the OS and any special considerations for writing hardware drivers)

The Iyonix had it in !AudioIn, but that seems to rely on a 3rd party module (if memory serves) and the author has vamoosed. This would open the door for skype-style voice-over-IP, but without a standard way of doing it, it seems to be perpetually “missing in action”.

Just found this link to a zip file that contains the !AudioIn module (GPL) and driver code from Dave Higton’s page.

I’ve corrected the spelling of my name in the post above.

Don’t forget that I also wrote a USB audio control module, which allows both input and out from and to USB audio devices and is also available from my web site http://davehigton.me.uk but you will also need a USB stack that supports isochronous transfer mode.

I’ve corrected the spelling of my name in the post above.

exercising your super-powers! :-)

Sounds like I need to get in touch with Dave! :)