IOMD ROM space saving

3. Sprites

It would be nice to have integrated support of compression for Sprites.

That’s a bit of can of worms to open, a lot of software makes assumptions about the sprite format.

My point on building compression into ResourceFS would mean both sprites and text files in the ROM image would be compressed, with no API changes.

AFAIK real IOMD systems already use the ability to softload updated bigger ROM’s , aka Select, so my view is why not just do that as it was always a constraint on Select that you had to have a certain specification.

If RPCEmu can use softload then the issue is solved as well for emulation.

As to continued IOMD development then I guess it comes down to if you see that the people who still use a RPC are future software,new hardware purchasers, provide valuable feedback or are indeed developers who need a minimum cost base to continue development on.

It is a tricky question but ultimately it comes down to what are the overheads in continuing development, including what holds back new functionality developement, against the benefits.

Personally I think it is nearing a time, if not already past it, to say a fond farewell.

If we don’t do it now how about 2024 at the 30th anniversary of the RiscPC launch?

You would think wrong in this case. RO5 on RPCEmu may account for around 50% of my RISC OS usage, and that’s a mix of development and “real” use.

Since I still haven’t got round to clearing the junk room known as bedroom 4 the ARM kit is largely unreachable apart from the RPC with RO4.02
As a result the only RO use here is on RPCEmu and I haven’t pirated out a set of RO4.02, so it’s RO5 only here.

It might be a better question to ask what the ratio of RPC users to RPCEmu users.
I would have thought that most RPC users would be remaining on 3.7 or 4.02 for 26 bit software reasons and the RO5, physical ROMs were a niche of a niche market (possibly in a remote corner)

The alternative is for someone to write a new emulator.

The RPCEmu devs have made it very clear they’re only interested in emulating IOMD, so any new “RO5 for emulators” build that ROOL might produce would initially have no platform to run on. Unless there really is another dev team that are chomping at the bit for an “emulators only” build that they can write an emulator for…

I think that is allied to the experimentation Mr. Grunditz is doing with Genode, and/or Gerph’s Pyromaniac

If RPCEmu can use softload then the issue is solved as well for emulation.

Never tried it, since dropping the ROM image into the ROMS folder of RPCEmu seems a trivially easy upgrade, and it is apparently happy to accept larger ROMS

My point on building compression into ResourceFS would mean both sprites and text files in the ROM image would be compressed, with no API changes.

One thing to be wary of is that a fair number of OS/ROM components use OS_FSControl 21 to get a direct pointer to the uncompressed file content in ResourceFS. So care will be needed to avoid breaking those components (or any third-party software which does the same thing).

If we don’t do it now how about 2024 at the 30th anniversary of the RiscPC launch?

What is this, Logan’s Run? ;-)

There are quite a few upcoming changes (Pinboard2, new TCP/IP stack, partition-aware FS stack, etc.) which will add significant amounts of new code to the ROMs. So we probably do need to find some long-term solution for IOMD. But since those features don’t require modifying hardware-specific code, and it’s trivial to increase the ROM size to 5MB or more (and generate compressed ROMs for physical use), I don’t think dropping IOMD support is a sensible solution. The only question is how much effort we put into reducing how much extra RAM gets used as the OS continues to grow.

If each ResourceFS file is decompressed completely into RAM as and when it is opened, then then handle returned can point at the data just as before. The only API change necessary is to Register/DeregisterFiles to indicate that the data is compressed.

The main questions, in my mind, are how much ROM space would be saved, and how much RAM space would be used?

The only question is how much effort we put into reducing how much extra RAM gets used as the OS continues to grow.

I guess as Adjust used compressed ROM’s, 6Mb down to 4Mb, then following that type approach is sensible and it isn’t as if it is a new approach.

In addition using Adjust type minimum spec recommendations may be sensible as well.

Perhaps a ‘Farewell to ROMs’ edition is eventually in order then, keeping a 4MB ROM for the RAM-challenged systems, then moving on.

I’d still vote for punting Draw and Paint to disc, just because!

including what holds back new functionality developement

Apparently the space, but as you mention, Select softloads.

We should set our target specification to not waste time pandering to the zero people attempting to run RO5 on a 4MB A7000.

There, problem solved, move along now…

To get a rough first approximation, I compressed the whole of Resources:$ into a zip file using SparkFS. A Count of Resources:$ gave 1328 KB, the zip was 729 KB, so a rough potential saving of about 600 KB. Sprite & Text files are over 55% of the files, which as Druck said compress well. Just compressing Wimp.Sprites & Tools would save about 180 KB – and Zlib would presumably give similar but different results.

To get a rough first approximation, I compressed the whole of Resources:$ into a zip file using SparkFS.

Isn’t Squash the preferred native compression format?

I even remember a (pseudo-?) filing system based on this!

What difference would that make?

I did wonder about trying Squash … but it only does single files, not a directory structure with lots of files. So zip was easier for a first test!

If each ResourceFS file is decompressed completely into RAM as and when it is opened, then then handle returned can point at the data just as before. The only API change necessary is to Register/DeregisterFiles to indicate that the data is compressed.

And when is the memory released? The programs the expect the data to be at that address even after the file is closed.

Echoing what other people have said, the ROM apps should be dropped completely, and work just like any other app. They were useful if you had an A3010, but nowadays it’s pointless to give certain apps exalted status, particularly in light of the fact that most have superior third-party alternatives.

the ROM apps should be dropped completely,

Well, no. Draw, Paint and Edit are pretty important. If you have an error during your boot sequence, anything further on is not done. Not sure how that would feel with nothing in Resources.Apps.

Well, no. Draw, Paint and Edit are pretty important. If you have an error during your boot sequence, anything further on is not done. Not sure how that would feel with nothing in Resources.Apps.

Well if you have Harinezumi running that boot sequence in a more robust fashion than the default, most stuff still works.

As a regular user of emulation, maybe I can shed some light on the various use cases for RPCEmu.

I am a bit surprised at that, as I look at RPCEmu as a way to run some picky 26bit software now that there are computers that can be completely put together for as little as $50 USD to run RICS OS 5 much faster (Raspberry Pi Zerro W plus minimal monitor, keyboard, mouse, and power supply).

Your average PC running RPCEmu or VirtualRPC is faster than a RPi 1 or Zero (especially on the memory and storage department). And it is much more flexible to run RISC OS on the same hardware – data exchange is much easier compared to setting up networking, using RISC OS and Windows/Linux/MacOS. Easy access to all the hardware connected to the PC.

And of course the flexibility to easily have multiple emulators on the same machine, from a RISC OS 3.1 machine with Arculator as well as 3.5 to 4.x with RPCEmu – this makes e.g. software testing much much easier.

As little as $95 USD if you want a decent keyboard and monitor.

So 95$ more than installing RPCEmu on your existing PC. If you want to use your PC side-by-side with the RPi, don’t forget to add a KVM switch to your price calculation. If RISC OS had the perfect RDP client, things might be a bit different.

For everyone with an existing PC (Windows, Linux, MacOS) which is reasonably powerful, RPCEmu is more flexible and (sometimes) faster and easier than adding a second piece of hardware. Especially when considering portable solutions.

I agree with Charlotte. Take the ‘ROM apps’ out completely. They make sense on a floppy-only A3010, but not on anything made after about 1994.

A compromise would be to put Edit in Resources:$ for emergency use. This would allow the normal Draw, Paint etc. to live in Boot:^.Apps with everything else.

Critically clicking on ‘Apps’ can open Boot:^.Apps! Thus the nonsense of the ‘Apps’ icon being impossible to explain is finally resolved!

I would suggest Edit must be in ROM if you want to be able to mend a broken or missing Boot file that is stopping normal startup. Trying to load it from disc of any sort may be impossible. It has got me out of a hole several times! I have never neded Draw or Paint.

a fair number of OS/ROM components use OS_FSControl 21 to get a direct pointer to the uncompressed file content in ResourceFS

Is the “direct pointer” documented? The Wiki has OS_FSControl 21 returning an internal file handle, not an address.

For RO3.20, where I have also filled the 4MB ROM and need to remove non-essentials, I plan to do the following:

1. Remove !Alarm
2. Remove !Paint
3. Remove !Draw
4. Remove !Calc
5. Remove !Chars
6. Remove everything related to printers
7. Compress ResourceFS into a ZIP based filesystem

The removals free nearly 500K and the compression was around 600K.

I’ve not looked at the Module list in RO5 recently, but expect there are also Modules that could be moved to !System. Get rid of that Marmite GUI startup sequence for starters ;)

Something I can’t do with RO3.20, but is possible in RO5 as RAM isn’t an issue, is to compressed more Modules.

Is the “direct pointer” documented? The Wiki has OS_FSControl 21 returning an internal file handle, not an address.

I can’t see any obvious mention of it in the PRMs, so I guess Acorn were relying on internal knowledge to use the feature.

The “internal file handle” returned by OS_FSControl 21 is whatever identifier the underlying filesystem uses for the file, and for ResourceFS it just uses a pointer to the file content. I’d imagine it’s been like that forever, since it’s such a useful way to save RAM.

I can’t see any obvious mention of it in the PRMs, so I guess Acorn were relying on internal knowledge to use the feature

We have a choice then, the behaviour either needs documenting and made public, or the OS code that’s relying on this behaviour needs rewriting to use public methods to access the file contents and not direct memory access.

The later would be the preferred option, rewriting to use OS_GBPB etc but how much work is involved? This route would also allow ResourceFS to be compressed using ZLib, provided the OS also includes a ZIP based image FS.

+1

In terms of fitting things in IOMD ROM for RO5, compressing the whole ROM sounds easier.

I would expect all direct accesses are already conditional on the file actually being in ResourceFS, so that the OS_GBPB code is already there. Thus, if you did want to remove all direct accesses to ResourceFS data, search for “fsnumber_resourcefs”, and disable the checks.

How would using image directories work? Wouldn’t you lose the ability to register and replace individual files?

That is, if you have an archive with a file Resources:$.Archive.A then registering Resources:$.Archive.A has no effect since fileswitch catches the access and redirects it to the image FS before ResourceFS gets to look at it?