1000 days of RISC OS Pyromaniac
2 posts, 2 voices
Charles Ferguson (8243)
427 posts
|
My work on RISC OS Pyromaniac reached a milestone that I thought I’d share with you. On Friday 4th March 2022, 1000 days have passed since I first started writing RISC OS Pyromaniac. Of course that doesn’t mean that I’ve been writing for it all that time (only about 640 of those days actually have commits), but it’s still a milestone. I started writing RISC OS Pyromaniac to be a debugging and development tool. If you’ve not heard of it, there’s a lot more information available on the website – https://pyromaniac.riscos.online/ – and presentations about what it does at http://presentations.riscos.online/. It was never really intended to be much more than that, but it’s developed quite a lot since then. The RISC OS build service is powered by running instances of RISC OS Pyromaniac in the cloud. I created it as a little fun service which I thought would be amusing, is surprisingly useful in actually testing builds of RISC OS components as part of their automated testing. The build service itself has been running for almost 2 years – it was released in April 2020 – and it’s been updated every few months during that time, tracking the releases of RISC OS Pyromaniac at that time. The releases of Pyromaniac itself, although only to myself, still happen at the start of each month, and reached number 33 a few days ago. The codebase now consists of nearly 100K lines of python (~90K lines of code, ~9K of comments). There are nearly 2000 tests in the project (1976 scripted integration tests + 9 python unit tests), made up of 25K lines of assembler and 7.5K lines of scripted tests. Code coverage has been tracked and it is my policy is that it should hold or go up with each new feature – currently it reports 69.5% coverage. That’s a lot of code, and that doesn’t necessarily mean it’s good, but that’s why there are tests for the code, and why the amount of code that’s actually exercised by those tests is measured. I’m pretty confident that I’m producing code that does what I expect, and that it’s probably right because of that. There are now 54 built-in modules implemented in Python 1. Some of them are very much core parts of the system’s operation, whilst others are just useful for playing with – and if it’s fun that makes it worthwhile 19. Mostly, regular modules can be loaded and used exactly as expected. Command line tools work as expected. The desktop can, with a small amount of work, be started and used – not just used, but debugged and examined. Full scale applications such as !Fireworkz work within the system 2. Games can run within the desktop and full screen 3. Customised versions of RISC OS Pyromaniac, using similar technology to the build service, have been used to run games over VNC 4 from the cloud. The system provides network access using the host system with IPv4 and IPv6 support, and provides a DCI 4 driver to be able to connect a network of RISC OS Pyromaniac systems together. Fake Freeway and ShareFS modules exist and communicate as if they were really sharing resources 5. Floppy and hard disc images are accessible through the ADFS module, and ISO images are accessible through the CDFS driver module. Sound can be output through the host’s MIDI interfaces, and plays quite passably. The tracing system can report on the execution of code, track functions and SWI use and report on memory accesses. Timings information can report how long and how many SWI calls were made, as well as other internal parts of the system. Recently, the introduction of a TCP based command system means that the behaviour of the system can be controlled from outside the system, in much the same way that the ‘monitor’ functions in QEmu/KVM 8. The debug system allows individual subsystems to be examined easily either whilst the system is running or through out boot. When aborts happen, or a part of the system is traced, or even when APIs report warnings, the system produces a dump of the registers, the related memory, recent code execution and other information related to the call. The reports even generate a C function stack backtrace if it is recognised that the system is running in a C-like environment. Debugging when something goes wrong is made easier by these reports which indicate what has happened 20. The system can run as a command line tool, allowing RISC OS command line tools to be used just like host tools with small wrappers, or as a desktop application using the user interface. The user interface using GTK and WxWidgets offers the ability to appear like a native application within your system 6. The interface offers the ability to change debug options whilst the system is running, in order to better understand what is happening, through menus. Information about parts of the system is available through menus and is being expanded as more facilities are implemented 7. The clipboard can be shared with the host system, and information about the disc and power usage of the host system are accessible. The graphics system is pretty faithful to the BBC derived interfaces, and can run many standard RISC OS applications, and even those using some of the more strange interfaces 9. Graphics operations are entirely accelerated 10, and videos of the system are easy to create, as are vectors 21. At times the system runs slowly, but this is an operating system implemented in a high level language 22, for the purposes of debugging and developing components. When I began the development it was only ever expected to be able to run the command line compiler tools and maybe some modules. It wasn’t really necessary to be able to plot fonts, render sprites, take input from the mouse, communicate over the network, or many of the other features that the system now has. RISC OS Pyromaniac was used to design and develop the controller and driver module interfaces for the FanController system 11 before the C code was even started. Tools are tested under RISC OS Pyromaniac as part of the general development process. Around 20 projects have been created which incorporate the RISC OS build service as part of their testing 12. The development of OmniDisc incorporated testing using the RISC OS build service from its inception 13. The upcoming PRM-in-XML tool will incorporate testing of the tool on RISC OS using the build service alongside the testing on other Linux distributions. RISC OS Pyromaniac runs on 3 different operating systems – macOS, Linux and Windows – and is regularly tested on 2 of them. It has been containerised for deployment in the cloud, as is used in the build service and others example services. The work has spun off other minor projects such as the PRM-in-XML work, tooling for handling RISC OS formats on non-RISC OS systems 14, VNC server for Python Cairo 15, and a markdown based slide presentation tool primarily to demonstrate the OS 16. Editor definitions for various RISC OS filetypes have been created 17, as well as numerous other released open source software 18. RISC OS Pyromaniac has been tested on, and runs happily on Unicorn 2, an updated version of the original QEmu sources, which also means it can take advantage of different CPU core configurations. During the development of RISC OS Pyromaniac many interfaces were examined in the documentation to understand how they interacted, and then within RISC OS Classic itself. At times the source was consulted to check how a particular behaviour was achieved but that was the rare exception rather than the norm. The intention was to develop the system to conform to the documented behaviour, with exceptions for actual behaviour, and to even skip out functionality that was rarely used or unnecessary. This means that some things are different, but there are fewer oddities in the system. The input system, for example, follows the spirit of RISC OS, but ignores the insanity that is the keyboard driver. That’s what Pyromaniac has become over the last 1000 days. It has massively exceeded its design goals, and whilst it is still at its heart a ‘semi-hosting environment for running RISC OS binaries on non-RISC OS systems’, it is RISC OS. I never really thought it would be able to work as well as it has, and whilst some experiments with it haven’t panned out, it’s still more than I ever expected. 1 Built in Python modules list:
2 Heap bugs which were seen in the video demonstrations laster have now been fixed and the application works fine. 3 The Great Escape has been demonstrated running, both in the desktop and running single tasking. 4 The RailPro demonstration game at https://railpro.riscos.online/ demonstrated the flexibility. 5 Although they don’t use the network, the Freeway and ShareFS modules track state and issue services as if they were actually sharing resources. https://asciinema.org/a/460199 demonstrates how this is near identical in behaviour to the full modules. 6 MacOS ‘Quick actions’ can be used to run programs within the Pyromaniac environment as windowed applications. https://share.gerph.org/s/EJbJvyJO0KHQIq5 demonstrates launching one of Jan Vibe’s graphical pograms which responds to the mouse. 7 Information windows provide some information on specific parts of the system. This is extensible and can incorporate any information used by the internal system. https://share.gerph.org/s/UTl9fTNoZZJQdYi demonstrates the module information window as an example. 8 The ‘system request’ server exposes the internal commands used to communicate with the sysem. These are easily extensible, and many of the user interface functions are provided through them. https://asciinema.org/a/467672 demonstrates using the server to query a running system in a separate terminal. 9 The Jan Vibe archive of graphical programs was published on GitHub earlier in the year. I have used RISC OS Pyromaniac to generate videos of the programs running, all of which were created unattended. https://github.com/gerph/jan_vibe_archive/tree/videos contains links to the programs in the Contents, and many have had videos added which were generated within RISC OS Pyromaniac. 10 The lack of any frame buffer accessible to RISC OS means that every operation is performed by the host as an accelerated operation. 11 FanController example code was recently released on GitHub – https://github.com/gerph/riscos-fancontroller 12 All CI based projects that I’ve created are tagged with `riscos-ci` on GitHub – https://github.com/topics/riscos-ci 13 OmniDisc is a replacement controller for FileCore Filers and was recently released on GitHub – https://github.com/c-jo/OmniDisc 14 Zip handling (https://github.com/gerph/python-zipinfo-riscos), StrongHelp extraction (https://github.com/gerph/riscos-stronghelp-python), OSLib processing (https://github.com/gerph/riscos-oslib-parser) to name a few. 15 Cairo VNC allows you to export your Cairo surface as a VNC session – https://github.com/gerph/cairo-vnc 16 Presenter was used for the slides in all the presentations I have produced – https://github.com/gerph/riscos-presenter, https://presentations.riscos.online/ 17 SublimeText definitions are present in packagecontrol.io, and a few nanorcs exist for RISC OS formats (https://github.com/gerph/nanorc-riscos) 18 Many of the open source projects not mentioned here are listed on the Technologies page – https://pyromaniac.riscos.online/technologies.html 19 Modules like Twin, which allows you to launch an editor on the host is useful but it was largely to show how integration with host tools was possible. Then there’s the Speak module which can communicate with the speech synthesis system on macOS so you can make the system talk – kinda fun to see it working, but probably not a generally useful thing. 20 The presentations linked above show some reports about the system, but a short video of the system running a development module which crashes can be found here: https://asciinema.org/a/345766 21 Creating vectors out of graphical output is relatively easy – you change the name of the image you want to create from a `.png` to a `.svg`. For example: https://jan-vibe-archive-videos.s3.eu-west-1.amazonaws.com/BAS06/dtree.svg or https://jan-vibe-archive-videos.s3.eu-west-1.amazonaws.com/DesktopAsAnSVG.svg for a slightly more complex rendering. 22 Here’s what Google has to say about whether you can write an OS in Python: https://www.google.com/search?client=firefox-b-d&q=can+you+write+an+operating+system+in+python |
Paolo Fabio Zaino (28)
1882 posts
|
Congratulations Charles, impressive work! :) |
