My RISC OS resources and Pyromaniac technologies

A couple of people had commented in a thread that they’d missed the resources that I’d published over the last few years to help with, or as part of, RISC OS Pyromaniac.

So I thought I’d point people to the technologies page – https://pyromaniac.riscos.online/technologies.html – and very quickly go through the things that have been made available on there, and referenced from other places. Plus a couple that weren’t referenced in public but have been made public on Github.

Web services:

• RISC OS build service front end – lets you write code and build binaries from an editor: https://build.riscos.online/
• RISC OS build service API – lets you build code by submitting it to the service and returns the output as text and binaries: https://build.riscos.online/api.html
• RISC OS example shell – demonstrates the RISC OS command line and the tooling that can be used within RISC OS Pyromaniac: https://shell.riscos.online/
• RailPro online – Example game using RISC OS Pyromaniac: https://railpro.riscos.online/
• robuild-client – tooling for building RISC OS code through the build service (incorporates a port of a cJSON parser and WebSockets client library) from POSIX systems or RISC OS: https://github.com/gerph/robuild-client
• Example CI repositories for 3rd party components (where not otherwise mentioned here):
  • LineEditor: https://github.com/philpem/LineEditor
  • Key Mapper module: https://github.com/gerph/ro-key-mapper
  • PrivateEye: https://github.com/dpt/PrivateEye
  • ErrorCancel: https://github.com/gerph/errorcancel
  • LanMan98: https://github.com/gerph/riscosdev-lanman98
  • CObey: https://github.com/gerph/cobey
  • Nettle: https://github.com/dpt/Nettle
  • All the GitHub CI examples: https://github.com/topics/riscos-ci

RISC OS-specific releases:

• Tools:
  • Presentation tool – Markdown based slides presentation tool, as used in my presentations: https://github.com/gerph/riscos-presenter
    • Built version, with modules: https://pyromaniac.riscos.online/binary/present.zip
  • Tools for building RISC OS Hourglass modules, and examples: https://github.com/gerph/riscos-hourglass-maker
    • Cog hourglass – Example generated by the Hourglass maker, as a port of the Pyromaniac hourglass: https://pyromaniac.riscos.online/binary/hourglass-with-percentage,ffa
  • VManage – tool for managing VersionNum files: https://github.com/gerph/riscos-vmanage
  • LineNoise – port to RISC OS of the LineNoise line editing library, and a module for ReadLine: https://github.com/gerph/linenoise
  • QR Code generator – port to RISC OS of a QR code generator program: https://github.com/gerph/QR-Code-generator
  • Pico – port to RISC OS of the pine composer, an editor: https://github.com/gerph/pico
  • RVCC – port to RISC OS of the rvcc compiler, an ARM and RISC-V compiler: https://github.com/gerph/rvcc

• Libraries:
  • Markdown parser – Port of MD4C to RISC OS: https://github.com/gerph/md4c/tree/riscos-build
  • LibXML2 – Port of LibXML2, used in PRM-in-XML: https://github.com/gerph/libxml2
  • LibXSLT – Port of LibXSLT, used in PRM-in-XML: https://github.com/gerph/libxslt

• Support code:
  • FontMap and documentation – RISC OS Select module for mapping between font names: https://pyromaniac.riscos.online/binary/fontmap-0.05.14.zip
  • WebColours and documentation – Module from my !SVG to map colour names to RGB values: https://pyromaniac.riscos.online/binary/webcolours-0.08.15.zip
  • ImageFileRender and documentation – RISC OS Select module for rendering images: https://pyromaniac.riscos.online/binary/imagefilerender-0.36.39.zip
  • DebugIt for SysLog – DebugIt is used by many parts of the RISC OS code, and this module just writes to SysLog: https://github.com/gerph/debugit
  • Parallel port joystick – Ancient implementation of the Joystick module through the parallel port: https://github.com/gerph/riscos-joystick-parallel
  • IconBorder module examples and documentation – the sources for the simple IconBorders modules used in RISC OS Select, and the interface: https://github.com/gerph/iconborders-example
  • IconBorder keyfob module – the source for an example key fob as a different type of button: https://github.com/gerph/iconborders-fob
  • Portable module SDK – the prototype for a portable module, intended to be a drop in for code to drive other hardware: https://github.com/gerph/riscos-portable-prototype
  • FanController stack – FanController and example driver module sources: https://github.com/gerph/riscos-fancontroller
  • DDEUtilsJF – implementation of DDEUtils: https://github.com/gerph/ddeutilsjf
  • JSSL – JavaScript debugger for RISC OS interfaces: https://built.gerph.org/JSSL/

• Example/test code:
  • riscos-tests – Selection of tests from RISC OS Pyromaniac which can be used to check the behaviour of RISC OS interfaces: https://github.com/gerph/riscos-tests
  • riscos-examples – Example code showing how to use some RISC OS interfaces: https://github.com/gerph/riscos-examples
  • dongutec-keypad – Example of IIC access to GPIO-based keypad – https://github.com/gerph/dongutec-keypad-mcp23008-example
  • jan-vibe-archive – Example videos and versions of the Jan Vibe graphics which work on modern systems: https://github.com/gerph/jan_vibe_archive
  • ci-templates – Templates for working with the RISC OS build system: https://github.com/gerph/riscos-ci-templates
  • jfpatch-as-a-service-examples – Example code usable with the RISC OS build service: https://github.com/gerph/jfpatch-as-a-service-examples

• Documentation:
  • PRM-in-XML tooling – RISC OS and POSIX tools for building PRM-in-XML documentation: https://github.com/gerph/riscos-prminxml-tool/
  • PRM-in-XML example files – Examples of content converted from the PRMs and others – https://github.com/gerph/riscos-prminxml-examples
  • PRM-in-XML staging area – Documentation converted from other sources to PRM-in-XML prior to integration into the main manuals (includes some Acorn specs, plus some Select documentation): https://github.com/gerph/riscos-prminxml-staging
  • StrongHelp Intenet manuals – Sources for building the manuals for the Internet StrongHelp manuals (RFCs, Media types): https://github.com/gerph/stronghelp-internet
  • Pyromaniac PRM: RISC OS Pyromaniac API documentation: https://pyromaniac.riscos.online/pyromaniac/prm/index.html

Non-RISC OS releases:

• Tools:
  • Tun/Tap JSON service – Python service to transfer ethernet packets by JSON lines over TCP to attach to real networks – https://github.com/gerph/tuntap-json-server
  • Changelog management – structured management of changelogs using the principles of https://keepachangelog.com/, with support for multiple concurrent branches of development: https://gitlab.gerph.org/gerph/changelog-management
  • Gource patches for RISC OS – patches to recognise RISC OS filetypes in Gource: https://github.com/gerph/Gource
  • Docker with RPCEmu inside – Installations of RPCEmu to run inside Docker (3.71 and 5): https://github.com/gerph/rpcemu-docker
    • RPCEmu with RISC OS 5: https://hub.docker.com/r/gerph/rpcemu-5 – docker run -it --rm -p 5901:5901 gerph/rpcemu-5
    • RPCEmu with RISC OS 3.5: https://hub.docker.com/r/gerph/rpcemu-3.7 – docker run -it --rm -p 5901:5901 gerph/rpcemu-3.7
    • Connect to VNC port 1 (5901) with password ‘password’
  • Rosetta code extractor – Extract example code from the Rosetta Code site: https://github.com/gerph/rosettacode

• Working with RISC OS files on other systems:
  • CodeMirror modes for RISC OS files – context colouring for RISC OS filetypes (JFPatch, ObjAsm, BASIC) in HTML input boxes: https://github.com/gerph/CodeMirror/tree/riscos-modes
  • Nano context colouring for RISC OS files – colouring for ARM, BBC BASIC, CMHG, C files, JFPatch and Obey: https://github.com/gerph/nanorc-riscos
  • SublimeText command file colouring – colouring for RISC OS command files (Obey files, Exec files, etc): https://github.com/gerph/sublimetext-riscoscommand-syntax
  • SublimeText CMHG colouring – colouring for CMHG files: https://github.com/gerph/sublimetext-riscoscmhg-syntax
  • SublimeText BBC Basic colouring – colouring for BBC BASIC files: https://github.com/gerph/sublimetext-bbcbasic-syntax
  • RISC OS Zip file manipulation – Zip file creation and extraction with RISC OS type information – https://github.com/gerph/python-zipinfo-riscos
  • BBC BASIC tokeniser – convert text files (&FD1) to tokenised BASIC (&FFB): https://pyromaniac.riscos.online/binary/basictokenise-0.02.10.zip
  • StrongHelp file extractor – Extracting content from StrongHelp files: https://github.com/gerph/riscos-stronghelp-python
  • StrongHelp file constructor – Alex Waugh’s StrongCopy tool: https://github.com/gerph/alexwaugh-strongcopy
  • OSLib parser – Parser for OSLib def file, in Python: https://github.com/gerph/riscos-oslib-parser

• Libraries:
  • SimpleYAML parser – single source file parsing of YAML files in Python: https://gitlab.gerph.org/gerph/simpleyaml
  • StreamedInput library – running commands in a thread and feeding the results through buffered input: https://gitlab.gerph.org/gerph/streamedinput
  • Cairo VNC library – Convert any Cairo graphics surface into a VNC accessible image/interface: https://github.com/gerph/cairo-vnc
  • Code to randomise contexts – Used within RISC OS Pyromaniac, this generates different types of context values to show up problems in RISC OS tools: https://github.com/gerph/riscos-gbpb-context-mangling
  • Turing Smart Screen library – Library for accessing a serial based display: https://github.com/gerph/turing-smart-screen-python-mit

• Implementations of RISC OS interfaces, in Python (external parts of RISC OS Pyromaniac):
  • Font qualifier parser – Parsing the qualifier on font names as required by FontManager: https://github.com/gerph/riscos-font-qualifiers-python
  • Font control code parser – Parsing the strings passed to FontManager into their components for rendering: https://github.com/gerph/riscos-font-controls-python
  • RISC OS Filer in WxPython – an implementation of the Filer like interface but for WxPython: https://github.com/gerph/riscos-filer-python
  • Memory dump rendering – an implementation of the memory presentation for command line and WxPython: https://github.com/gerph/riscos-dump-python
  • RISC OS style disassembly – disassembling RISC OS code, including FPA, as used in RISC OS Pyromaniac: https://github.com/gerph/riscos-disassemble-python
  • python-codecs-riscos – RISC OS alphabets as Python codecs: https://github.com/gerph/python-codecs-riscos

And in addition to released code and documentation, there is general information on the projects that I have worked on, or areas that I give attention to:

• Information on RISC OS Pyromaniac: https://pyromaniac.riscos.online/
  • Feature set: https://pyromaniac.riscos.online/pyromaniac/FEATURES.html
  • Ongoing changelog information: https://pyromaniac.riscos.online/pyromaniac/CHANGELOG.html
• Presentations about RISC OS Pyromaniac, testing, and others: https://presentation.riscos.online/
• Article about RISC OS documentation direction: https://www.iconbar.com/articles/RISC_OS_Documentation/index1700.html
• Article about RISC OS testing: https://www.iconbar.com/articles/Testing_on_RISC_OS/index1688.html
• Ongoing information on RISC OS Pyromaniac development: https://twitter.com/ropyromaniac
• Videos of RISC OS Pyromaniac (and others) in use: https://asciinema.org/~gerph

RISC OS example shell – demonstrates the RISC OS command line and the tooling that can be used within RISC OS Pyromaniac: https://shell.riscos.online/

Is there anywhere specific you would like bug reports for this?

As here is a simple one

*BASIC
MODE 0
COLOUR 2 (or anything higher)

produces a python backtrace pointing to a failure to cap the colour numbers to the ones available in the current mode.

In general I’m afraid I’ve not paid that much attention to Pyromaniac for a few reasons;

a) It’s not open source.
b) It’s not freely available.
c) The online tools such as the build or test service exist only as long as someone other-than-me pays their cloud-computing bills, which means I’d be unlikely to use it for more than experimenting.

I have nothing really against the project and I hope it keeps you entertained.

a) it will be open once people start using it – as I have said many times before, until people actually start using it and give me confidence in it, it’ll remain as it is.
b) the RISC OS build service is freely available. The Pyromaniac shell is freely available.
c) Feel free to start using it finding problems and encouraging the use of it so that it will become available, or if you feel that you have to pay for the bills, then I’m happy to share the cost if you’re volunteering to do so. Though… why are you using this website? it’s only here so long as someone else pays the bills?

ACE bug report though… I honestly don’t understand how that’s managed to get through :-) Thank you.
Bugs can come either to me directly by mail, or maybe on here. I haven’t really considered where people would report bugs because anyone who would be using it in anger has access to the gitlab repo, so can report them directly as issues in there.

Feel free to raise future bugs at: https://github.com/gerph/riscos-pyromaniac-issues
I shall include this in the site information in the future.

As for the bug you raised, that’s an issue with the ansitext implementation of the VDU system, failing to handle the colour selection in the correct order. The code is now fixed locally and will appear in the next release. The code was extracting the colour from the palette /before/ the derived class had vetted it and converted the number to the correct value for the mode and tint.

ImageFileRender and documentation – RISC OS Select module for rendering images: https://pyromaniac.riscos.online/binary/imagefilerender-0.36.39.zip

I’ve been trying this out and it seems to be working well for the most part. I do have a few observations, though:

• Sprites using RISC OS 5.21 mode words are always reported as unspecified, as well as sprites with newer types. Similarly, sprites with alpha channels instead of a mask are reported as not being transparent.
• I’m not sure how to get the actual resolution of sprites without it being transformed to OS or Draw coordinates.
• Draw files are reported as having a very large DPI. Is that intentional?
• Sprite files aren’t displayed on RISC OS 3.1. Draw files work, but require applications to load the DrawFile module separately.
• Are there any plans to release a version of ImageFileConvert as well?

Just had a look at the risos-filer-python as I’ve just ported something using wxPython, and I’m shocked, shocked I tell you, to find it’s Python 2 code – in this day and age!

Just had a look at the risos-filer-python as I’ve just ported something using wxPython, and I’m shocked, shocked I tell you, to find it’s Python 2 code – in this day and age!

RISC OS Pyromaniac is currently Python 2, so a lot of the components focus on working with Python 2. That said, I try whereever possible to make the code function just fine on Python 2 or 3.

Your bug, https://github.com/gerph/riscos-filer-python/issues/1, has now been fixed and the change pushed. It’s likely that the other components have similar issues with Python 3, but I’ll fix them as I come to them. Thanks for raising this – despite the fact that it took a few hours to get a working wxPython + Python 3, it’s better for at least working on Python 3. Testing was against Python 3 and wxPython 4.1.1

Screenshot of it running in Python 3 attached to the issue.

Sprites using RISC OS 5.21 mode words are always reported as unspecified, as well as sprites with newer types. Similarly, sprites with alpha channels instead of a mask are reported as not being transparent.

The newer sprite mode words aren’t supported at present as the module was written in 2002.

I’m not sure how to get the actual resolution of sprites without it being transformed to OS or Draw coordinates.

It’s a renderer, so the resolution of the image isn’t actually it’s job. It’s there to render, and to tell you how much space that the rendering takes. That said, such information might be able to be made available through a developer query to the system if it was useful. I’m not sure that it is, as some image files don’t have a resolution – they might have bounding boxes, but then they also might not. The HTML renderer has this issue, as I recall.

Draw files are reported as having a very large DPI. Is that intentional?

In the documentation you’ll see reference explicitly to the DPI that’s used. I’m not convinced that this is correct or that the module is correct. Feel free to check my logic here (and what it’s reporting) – ‘a very large DPI’ doesn’t really help me understand what you’re seeing and whether it’s right, because what you think is large might not be wrong…

Anyhow… from the PRM…

“OS units are notionally 1/180th of an inch, and are the standard units used by the VDU drivers for specifying output to the screen.”

“Internally, Draw uses a coordinate system the units of which are 1/256th of an OS unit. We shall call these internal Draw units.”

Thus it follows the the minimum size of a unit in a DrawFile is 256 * 180 DPI, which is 46080. I believe certain versions of the text had this value at 2048, but from the above, I believe 46080 is the correct DPI for a DrawFile.

Is that the value that you’re seeing? And does my reasoning for that value make sense, or is there something else that I’ve missed?

Sprite files aren’t displayed on RISC OS 3.1. Draw files work, but require applications to load the DrawFile module separately.

RISC OS 3.1 isn’t something I care about. If it works on there, great, but I don’t have any system for testing it or any interest in doing so. Obviously applications should load the resources they will need – they would also need to load the AWRender module to use the artworks renderer, the relevant ImageFileConvert modules if they want to use those resources, and Euclid if it wanted to draw the Euclid files (with the appropriate renderer files). It’s arguable that we could disable the registration if DrawFile wasn’t there, but DrawFile doesn’t have an announcement that it has arrived or left, so this could not be done reliably.

Are there any plans to release a version of ImageFileConvert as well?

Maybe, if there was any interest. The only reason for releasing IFR was because I wanted to make the presentation tool useful to people. That was 2 1/2 years ago and nobody has shown any interest in either the presentation tool or IFR until now…

Gerph wrote:

RISC OS Pyromaniac is currently Python 2, so a lot of the components focus on working with Python 2. That said, I try whereever possible to make the code function just fine on Python 2 or 3.

There’s always a few gotchas with code which is supposed to work on both, when you actually try to use it on both.

Your bug, https://github.com/gerph/riscos-filer-python/issues/1, has now been fixed and the change pushed.

Great, that’s working well under Python 3 now. Could do with an option to hide hidden files on Linux though, have to skip past lots of dot something at the top of every directory.

It’s likely that the other components have similar issues with Python 3, but I’ll fix them as I come to them. Thanks for raising this – despite the fact that it took a few hours to get a working wxPython + Python 3, it’s better for at least working on Python 3. Testing was against Python 3 and wxPython 4.1.1

I’ve just spent the last 2 months porting and pylinting 54,000 lines of Python 2, which contained about a dozen wxPython 2.8 apps which I’ve made work with 4.1.1 – it didn’t involve quite as many changes as I first thought. Only 29,000 lines to go, but no more wxPython.

By the way the pylinting is far more work than the porting (even skipping most naming niggles and loosening the design limits considerably), and I suspect that would be the case for Pyromaniac too.

There’s always a few gotchas with code which is supposed to work on both, when you actually try to use it on both.

Yeah; it’s not great when you’ve assumed things’ll be fine and didn’t even bother testing, as was the case with the filer code there. I guess the dump code will have similar issues. With things like the longtraceback (https://gitlab.gerph.org/gerph/longtraceback) I’ve at least got dedicated tests to exercise both Python 2 and Python 3, but haven’t done so for The Filer… it’s actually probably quite amenable to testing at least the virtual VS code so I should probably do that.

Great, that’s working well under Python 3 now. Could do with an option to hide hidden files on Linux though, have to skip past lots of dot something at the top of every directory.

I’m thinking the best way to do that would be for the FS enumeration to allow a filter to select what’s displayable, and either having a callback function to perform the filtering, which you can register, or making it a class member that you override. That way you can then provide a nice menu option to filter out classes of files (eg dot files, but also by filetype, size or whatever you wanted). It hadn’t been a consideration as Pyromaniac doesn’t generally need that kind of thing for its Filer display of the RO content.

I’ve just spent the last 2 months porting and pylinting 54,000 lines of Python 2, which contained about a dozen wxPython 2.8 apps which I’ve made work with 4.1.1 – it didn’t involve quite as many changes as I first thought. Only 29,000 lines to go, but no more wxPython.

That’s pretty fun (!). The wxPython side doesn’t seem too bad, but there’s always that transition from strings to unicodes that is the hardest part, IME. And for Pyromaniac the vast bulk of the code is Python 2 str-used-as-bytes, so it’s not a simple thing. Plus there’s some metaclass stuff that’s a bit tedious.

And for Pyromaniac the vast bulk of the code is Python 2 str-used-as-bytes, so it’s not a simple thing.

That is the main issue for this API stuff too. I’ve gone with treating all the things which were strings under Python2 as bytes to maintain binary compatibility, and only converting them to/from UTF-8 strings when they interact with the console, logs or user interfaces. The key thing to watch out for is when you now don’t need to use ord and where you still do.

The key thing to watch out for is when you now don’t need to use ord and where you still do.

Yup, I’ve just fixed the riscos-dump-python repo for Python3 and that was what the main problem was. The three python components – the filer, the dumper and the disassembler – should now all work in Python 3.