Bounty proposal: Emulator

This is a proposal for a bounty that is IMHO absolutely essential for “plain old user acceptance” of RISC OS 5, ARMv7 and possibly later ARM variants.

In my experience, many users still operate their trusty old Risc PC (or even A5000) because their much-loved software has become incompatible with later incarnations of either ARM or RISC OS. Even RISC OS 4 suffered from this, despite being very compatible with RISC OS 3.7

Many have gone down the emulation route because th emulator is more compatible than the latest native hardware. The IYONIX suffered from this (despite Aemulor), and we see the same now with the BeagleBoard.

What I propose is the implementation of a generic RISC OS and ARM emulator built into RISC OS. The first step would be an ARM emulator for 32bit but not ARMv7 compatible software. The second would be a 26bit (ARM6 for maximum compatibility) emulator. The architecture should be open enough to allow setup of e.g. an SWI trap to allow simulation of behaviour of older RISC OS versions.

I think that “simple” and therefore slow emulation (i.e. interpreted) would be sufficient – there should still be plenty of motivation for the still active software authors to provide ARMv7-ready software, and for users to upgrade.

As I am not really an expert wrt the lower levels of RISC OS, so I am sure that the experts will be around in a minute to explain the difficulties of implementing this proposal…

But I am sure Adrian Lees would be interested in taking this bounty ;-)

edit: fixed typo

As I am not really an expert wrt the lower levels of RISC OS, so I am sure that the experts will be around in a minute to explain the difficulties of implementing this proposal…

Hello! ;-)

I’m in agreement with you that it would be good to have a RISC OS compatible emulator that allows people to run their old software. It would also be nice if it was a “user mode” emulator like Aemulor – i.e. the emulator seamlessly integrates the running program with the host OS. However I think creating such an emulator would be prohibitively difficult, and will place too many restrictions on the host system (e.g. the Wimpslot restrictions when Aemulor is loaded).

So instead I’d propose that we should aim for one (or more) full-system emulators. And as luck would have it, several such (open source!) emulators already exist, with ArcEm & RPCEmu most likely being the most prominent/feature-complete. However there are a few issues that need to be resolved with both these emulators:

• RPCEmu doesn’t have a RISC OS port yet.
• ArcEm does have a RISC OS port, but the native frontend isn’t very good. Neither are all features (e.g. sound) supported.
• Performance.
  • One of my side projects has been working on ArcEm in order to make it a viable solution for Arc emulation on new hardware (primarily ARMv7, although getting it running on the Iyonix too would be nice). Apart from some assembler bits which I ripped out, ArcEm uses an interpreter which is written entirely in C. This makes it easy to develop and port, but isn’t the best for performance, particularly on an ARM host.
  • When I started looking at it, it was getting a measly 0.8 MIPS on an Iyonix (or about 1.6MHz), which was far too slow to be usable. After about a week of hacking I was able to make it three times faster, but even on a 720MHz BeagleBoard it still wasn’t able to reach the speed of a real ARM2. I’m sure there’s still more work to be done, but by the time you take into account all the other stuff that needs doing (better video handling, audio, etc.) I’d be surprised if you’d be able to reach ARM250 speeds, let alone ARM3. (See this thread on the ArcEm mailing list for more info about my endeavours).
  • So basically, for ArcEm, we probably could get by with using an interpreter, but for RPCEmu we should most definitely be leveraging the existing dynamic recompiler framework to create an ARM-on-ARM dynamic recompiler.
  • Once RPCEmu’s recompiler is done it may be possible to port the code back onto ArcEm, but it would need some modifications in order to work correctly (Instead of relying on the cache flush CPU instructions as a hint for when code is modified, it would have to track every single memory write). Plus I’d argue that one of the benefits of ArcEm is that it tries to provide cycle-accurate emulation (important for games – arguably the only software you’d probably want to run), and any recompiler might run the risk of breaking the cycle counting code (if all my changes haven’t broken it already!)
• OS integration. Although we won’t be able to provide seamless integration like Aemulor, there are obviously things that we can do, e.g. file/printer/network sharing, drag-and-drop files into the emulator window in order to easily open them, etc. Maybe even have the emulator run a modified Wimp which allows the windows to be displayed on the desktop of the host OS.

I think the challenge with most desktop applications might be all the wide variety modules they load and the way they are expecting RISC OS to behave. I’m not sure it’s only games though, many of the most useful desktop applications for RISC OS are 26bit and everyone has a different set of those old files that they will want to access for some reason or another.

It would almost be better if those incompatible instructions could be made to abort so they can be translated on the fly? I guess this is what the re-compiler is trying to do in the only practical way?

A challenge must be distinguishing data tables from code I would think. Perhaps a code scan and replace the offending instruction with a SWI instruction so not to mess up any offsets? :-) Much too simplistic perhaps.

Certainly the fixes for TAG and TBAFS were basic status returning and mode switching. Nothing else was a problem and all the OS calls being used performed perfectly.

Could I humbly suggest that such a massive programming effort would be better directed at other projects? You’d do all that work and what would it get you? A slow and slightly clunky way to run ancient software for a niche platform on the self same niche platform which is still unable to take advantage of multi-core CPUs, can’t do pre-emptive multitasking, doesn’t have Wifi or IPv6 support, hasn’t got a usable browser with a decent JavaScript engine, can’t access multi-gigabyte files or terabyte hard drives, etc, etc, etc.

There are a million things I’d love to see RISC OS able to do, but running some unsupported, unmaintained, fifteen year old programs that only appeal to a minority of users on a minority platform is not one of them.

That’s not to say it’s not a fascinating challenge from a technical perspective. And I’m sure as much as I dislike the idea, there will be lots of people who really love it! :)

Hi Jeffrey,

of course I have followed your postings to the ArcEm mailing list closely. In fact, your results made me think that the whole system emulator idea is basically no-go for my idea of a general ARM compatibility layer for RISC OS. If using the interpreter approach, I am pretty sure that the only chance you have to get acceptable speeds would be to allow most SWIs “through” to the host OS for full host speed on these calls. ISTR that someone from Pace once posted to Usenet some results from using DDT’s ARM emulation with exactly this approach.

Could someone explain to me (in easy words preferably) why Aemulor needs to limit the WimpSlot size? Wouldn’t it be technically possible to only limit it for (a)emulated apps? I am sure I am affected by huge lack of basic RISC OS knowledge here, because if it would be easy, I am sure Adrian would have implemented it…

Hi Steve,

I agree that there are a lot of areas where RISC OS is currently lacking support – both on the OS and on the app side of things. But many of the things you mentioned are work-aroundable. We access large discs by NAS. We can use WiFi via a router or bridge or WiFi client. Nobody needs IPv6 right now. The browser issue does not affect most people because I think the people having no access to a PC for browsing are a much smaller minority than any other minority on our minority platform.

On a minority platform, it is natural that not much software is ever released. I guess that every major step in ARM/RISC OS history (3.1 → 3.5 → 3.7(StrongARM) → 32bit → ARMv7) has probably halved the amount of software available. This is a real problem for the RISC OS platform and its users. RISC OS on ARMv7 is hard to sell to the average user if you create huge up-front costs (pay for software upgrades) or force him to operate multiple machines side-by-side.

If the RISC OS software market was buzzing and thriving with new programs being announced every minute, we could ignore the problem. App A no longer runs? No problem, use app B which is better anyway. But in our situation, where many developers have abandoned their old apps forever, discarding the massive amount of old software will create a huge obstacle for people migrating away from their trusty old RPC. And it will be a recurring problem – I am sure ARM has new incompatibilities for us available in ARMv8.

By the way, wouldn’t it be cool if the emulator ran on one of the free CPU cores that RISC OS can’t make use of anyway ;-)

of course I have followed your postings to the ArcEm mailing list closely. In fact, your results made me think that the whole system emulator idea is basically no-go for my idea of a general ARM compatibility layer for RISC OS. If using the interpreter approach, I am pretty sure that the only chance you have to get acceptable speeds would be to allow most SWIs “through” to the host OS for full host speed on these calls. ISTR that someone from Pace once posted to Usenet some results from using DDT’s ARM emulation with exactly this approach.

Yes, full system emulation isn’t the best for performance, but it is the best for compatability, which is surely the aim of this idea in the first place. How would a user mode emulator cope when we fundamentally change some aspects of the OS in order to add multi threading/multi core support? We’d be creating an awful lot of work for ourselves if the OS/emulator had to maintain some translation layer for old programs. Instead, if we had a decent full-system emulator, we could even see that as giving us permission to make massive backwards-incompatible changes to the OS, because people can just use the emulator to run all their old software.

Could someone explain to me (in easy words preferably) why Aemulor needs to limit the WimpSlot size? Wouldn’t it be technically possible to only limit it for (a)emulated apps? I am sure I am affected by huge lack of basic RISC OS knowledge here, because if it would be easy, I am sure Adrian would have implemented it…

For 26bit programs to work correctly, Aemulor needs to emulate the old memory map in order to make sure all code dynamic areas (e.g. the RMA) are within the first 64MB of logical address space. As to why the limitation needs to be in place for both emulated and non-emulated programs, imagine what would happen if an emulated program was to store some data in the RMA then issue a wimp message, SWI/service call, etc. to let other programs know about it. Since Aemulor doesn’t know how every single wimp message/SWI/service call works, it’s infeasible for it to trap cases like this and patch the pointers to point to some other copy of the data. So instead it has to use the same memory map for all programs.

I agree that there are a lot of areas where RISC OS is currently lacking support – both on the OS and on the app side of things. But many of the things you mentioned are work-aroundable. We access large discs by NAS. We can use WiFi via a router or bridge or WiFi client. Nobody needs IPv6 right now. The browser issue does not affect most people because I think the people having no access to a PC for browsing are a much smaller minority than any other minority on our minority platform.

Although by that same logic, the emulator issue is work-aroundable, because people can just run an emulator on their PCs ;-)

If there’s a port to the Raspberry Pi device, emulation could be useful in education. Obviously a lot of schools will have disposed of their software at the same time as their machines. But if there were a simple means of getting old software working (without requiring recompilation) it may be attractive enough for publishers to reissue some old classics. And speed probably isn’t much of an issue in education, although sound would be.

Well I have tried out ArcEm, !Archiemu and !A310Emu on my Beagleboard and these are my findings.

- ArcEm works is is so slow as to be unusable in my view
- !A310Em doesn’t work complaining about some ADFS/IDEFS Discop SWI’s not being available.
- !archiemu works and runs at an acceptable speed and I have tried !Style on it and that seems Ok along with the new !Boot. Only issue is that you have to have exceptions off and for some reason once you have closed down any attempt to restart fails unless you put a clean set of CMOS configs in place.

Now for what it is worth my humble opinion is that most people may want to be able to use older software so some sort of simple ablility to do so like Aemulor or perhaps one of the existing emulators should be updated, either !A310 or !archiemu by Jan de Boer, if practical but I would not want lots of development time and major effort to distract from the main development of RISC OS on newer hardware and exploiting that hardware to it’s full potential. RISC OS has been hampered by too much looking back over the years and we should be encouraging the software products/developers still around to update still available products like OvationPro, Schema2, DataPower, MessengerPro etc. Without spending money with them or encouraging them to update then we’ll have some hardware but nothing new to either take advantage of future additional capabilities or add those missing features.

ArcEm, !Archiemu and !A310Emu

Also note Backwards ROS compatibility thread.

Well it’s nice to see my compatibility thread hasn’t been forgotten ;) (the suggestion of it being a core component btw was because I reckoned it would run faster).

Anyway, the way I see it would be that things would need to be split up into several different projects. Note that I don’t know of the vast improvements to ROOL since I was last on back then, so maybe some things already work.

I’m writing this looking at the consumer arm tablets on the marketplace just now, and noting how most come with only one mini-usb port, and on more expensive models also a full usb port.

1) Rool will need to support usb floppy disk drives.
2) Rool will need some way (A310Em has this, though it doesn’t work for all disks depending on anti-piracy measures) to convert floppy disks into image files.
3) Rool will require a virtual floppy device that can read these disk images. A virtual cd-rom drive for the later software may be required.
4) Rool will need a virtual onscreen keyboard ala Android, but what would be good is if only keys you wanted to view could be shown (eg. Z X / ’ and return).
5) If Rool was to enable the gyroscopic movement/rotation of tablets, then this could be used for movement in some games perhaps.

The way I look at it would be that points 1-3 would be core components of ROOL, whatever the hardware, these would remain constant for hooking into further up the line.

Relating to Doug Webb’s post, once these are implemented, then those working on the core of ROOL would not be distracted from hardware development of getting ROOL stable and capable of running on other machines, because once the base was there they’d be little more to do other than the occasional bug to fix.

This then leads on to the updating of the emulators (which would contain points 4 and 5), while this leads slightly away from opensource, I do feel that the way forward in this respect would be to take the ideas and implementations from A310Em/archiemu and combine them with those of Aemulor (which moved to http://buyit.spellings.net/). I realise one is free and one is commercial (and no doubt closed source) but it would if combined as a new emulator provide emulation accross the board for all the RiscOS versions up to 4.

So my suggestion (and some may view it as a bit of a cheek) would be that the way forward would indeed be an emulator, in that the Aemulor team and Jan de Boer work together to create one functional on ROOL, thus not distracted from ROOLs main development, but with those in the main development implementing bits into the core as required for the emulator to work effectively.

Personally I’d happily pay the nearly £30 for an emulator that allowed me to run all my old RiscOs software, I’d figure that as a sound investment.

Chief :)

1) Rool will need to support usb floppy disk drives.
2) Rool will need some way (A310Em has this, though it doesn’t work for all disks depending on anti-piracy measures) to convert floppy disks into image files.

The way the USB floppy standard has been defined rules out the possibilty of using a USB floppy drive to read anything other than DOS formatted floppies (or at least, ones which use the same sector and track layouts as a DOS floppy). So if you’re suggesting that we use USB floppy drives to read ADFS discs then I’m afraid you’re out of luck :(

Yes I was suggesting that because otherwise you’d need to either have a convertor written for windows/linux to be used on a pc with a standard floppy disk drive, or have a program which runs on riscos machines to do the same, and that would be a pain either way :(

Would it be possible to do it in some other way that I haven’t thought about?

Chief :)

Would it be possible to do it in some other way that I haven’t thought about?

The only way I can think of would be to create your own USB → floppy interface and use that. It’s probably easy enough to find a suitable microcontroller, but I’m not sure where you’d get the floppy drives from – maybe by pulling apart USB floppy drives? I’m guessing ordinary non-USB floppy drives are becoming pretty rare (new ones, at least).

I’d have no real clue where to start, and making people use micro controllers etc would somewhat reduce the ease of use such emulation would be looking for.

Well I do have my old and reliable (well old from 2005) Sony floppy disk drive, it’s just a standard internal drive.

I also have one old slim internal floppy drive which I know reads ADFS floppy disks because it made up the drive for my A4000 laptop project.

Chief :)

There is already a solution for a USB → “real floppy” interface which works at a low-enough level to support all ADFS needs: it is called KryoFlux. No write support at the moment, however. It “just” needs someone to write a driver.

KryoFlux

Interesting! €90 is a bit more expensive than a standard USB floppy, though ;-)

I had to blink when I went on their store page, thought I’d ended up at liquid silicon for a second there.

And yes 90 Euro is just a bit past my budget since I just bought an tegra2 tablet :)

Okay so I’ve been thinking more about this and was wondering, what if…

Rool hosts an !App Shop on the site, containing various free and commercial risc os open compatible programs.

Open has support for the A310Em image files on a virtual floppy disk drive (and perhaps a virtual cd-rom drive that mounts .iso images?).

The create floppy image portion of A310Em is ported to a new program, compatible with RiscOs 3 and above.

People who still have RiscOs machines convert their old software disks to floppy images.

These people upload these images to an emulator section on the !App Shop.

While the files are freely there for anyone to download, to download a file you must agree to an dialogue that:

“You agree that you own a legitimate version of this software, and are downloading this software solely for your own personal use” (or something like that).

Basically using a similar disclaimer to cd-burning software (You agree we’re not responsible if you’re doing illegal stuff with our program).

I realise that in some respects we already have disk imaging software in the shape of ADFImager, however my attempts with adf images and the windows adf creators were only partially successful, while the A310Emu images seem to be near perfect.

Or do you think we’d get cease and desist notices for hosting such files?

Chief :)

Or do you think we’d get cease and desist notices for hosting such files?

I’m not sure whether we’d get any, but ROOL almost certainly would :-)

Anyway, there’s already this initiative for setting up an archive of old software.

Shameless advertisin’, but last night I released a new version of ArcEm to the developers mailing list. It’s still very rough around the edges, but compared to the version available on riscos.info:

• Runs single-tasking (the riscos.info version was using the X frontend instead of the faster single-tasking RISC OS frontend)
• Sound support (you’ll need SharedSound if you want sound)
• Video code rewritten. The new code aims to provide scanline-accurate timing of display updates, and depalettises each scanline on the fly to a 16bpp screen mode. So games which make use of mid-frame palette swaps now work (almost) properly (Lotus II – the only game I really tried for testing – seems to be off by up to 1 scanline, but I think that’s actually better than Red Squirrel managed on my beefy PC). The build I uploaded to my site uses a 800×600 mode for everything, so you can expect lots of black space around the edge of the screen (the core code does support scaling, but at the moment the only scaling will be line doubling for the rectangular pixel modes). Eventually I should get it to select more appropriate screen modes (and after that, make use of the OMAP’s hardware display scaling).
• The host-side HostFS code has been rewritten. I forgot to check whether it was working properly in the build available on riscos.info, but when I first tried enabling hostfs I found that unixlib wasn’t attempting to undo the RISC OS → unix filename translation that arcem was doing (since the hostfs code was expecting to be run on a unix-like host). So if you saved a BASIC file from within the emulator you’d end up with a text file with a ,ffb extension. The new code just passes the RISC OS filenames & attributes straight through (if it’s running on a RISC OS host, obviously)
• The IOC timers now try to run at the correct real-time rate. This allows Lotus II to time its palette swaps (almost) properly, but I’m still puzzled by why !Alarm shows the system time as counting up too fast.
• It’s somewhat dependent on screen mode & screen activity, but performance on a standard BB-xM hovers around the 8MHz mark during gameplay (Lotus II again). This is about right for an ARM2. An Iyonix runs at about half the speed. So action games will either be a bit sluggish or will fail completely (Lotus II doesn’t like it). Also users of FX-series cards will have to put up with red & blue being swapped since I haven’t added an option to configure it yet.

I’d be interested in hearing reports of any games/software that does/doesn’t work properly. E.g. I spotted that the screen flickers every so often when playing Asylum, which is probably a bug in the new video emulation. Although I don’t expect to be doing lots of bug fixing straight away, having a list of games with known issues will surely help later on.

I’m sure I tried A310Em/ArchiEmu at some point recently, but I can’t remember how responsive they were, either on my Iyonix or BB. But I’d certainly like to think that this new version of ArcEm is the best out of the lot! (And before Steve complains at me for spending time on this instead of doing better things like adding multi-core support to RISC OS, it’s only the past week that I’ve been working on ArcEm again ;-) Once the code has made it into the ArcEm CVS it’ll be back to business as usual.)

The screen has always flickered every so often when playing Asylum, no matter what you’re playing it on – it’s in the PFAQ, in which you state quite plainly that you have no intention of fixing it! ;-P

Hmm. It’s obviously been a while since I’ve played ;-)

I don’t remember seeing any flickering at all when I played it on my A3010, and the background jumping/corrupting on StrongARM machines is/was due to self-modifying code. But I was playing the unmodified Asylum demo (perhaps it’s a demo-only bug?), running in mode 13 (I think), and it looked like every so often half the screen would be black.

Anyway, if it does turn out to be an ArcEm bug/inaccuracy then it shouldn’t be too hard to find out what’s causing it by adding some logging code.

For reference, there was a thread on comp.sys.acorn.misc in 2003 relating to Aemulor and its implementation.