Sprite crunching

An observation – we could save around 75K of ROM space by reprocessing the Wimp iconsprite set to all use RISC OS 3.5 mode words (or newer) instead of numbered modes. It’ll reduce the space needed for the masks by about a quarter (most of the sprites are 4bpp, which with mode numbers means they’ll have 4bpp masks. With newer sprite formats masks are 1bpp)

The same thing could also be done for the disc-based sprites, saving about 300K of RAM for the Sprites22 set. The only downside being that we’ll lose 3.1 compatibility (not that I doubt any RISC OS 3.1 users would seriously want to load >1MB of icon sprites on startup). However the big killer with the disc-based sprites are the palettes – 360 full 256 colour palettes comes out to 700K, almost half the size of the file! Converting the sprites to a 16bpp format would save about 1K per sprite (for 34×34), especially if the RISC OS 5 15bpp + 1 bit alpha format is used, but would have to be done with care to avoid introducing banding on any which have smooth gradients. For the “small_” sprites, it would save about 2K (the overhead of the palette is much greater at the smaller image size), and could perhaps be done wholesale since banding will be less noticeable (smaller image size → steeper gradients).

Apart from saving memory (and for the disc-based sprites also disc space and load time), would there be any significant effect on the processor time taken to process and display crunched sprites?

For a simple conversion to RISC OS 3.5 sprite mode words, I think the performance will be about the same. Maybe a tiny bit faster just due to the smaller mask data.

Converting to 16bpp will be different. When rendering in true colour modes, there’ll be less cache thrashing due to not needing to use a palette. But longer code sequences due to needing to convert e.g. 16bpp to 32bpp. On a future SpriteExtend that’s capable of generating NEON code this could result in significant gains, but for the current ARM generator it’ll probably make things slower. When rendering at lower colour depths, I’d expect performance to be slightly worse than it is currently, since it’ll be going via a 32K entry lookup table rather than a 256 entry one (so more cache thrashing).

Is space an issue in the ROM? If it is, I’d say convert the ROM based sprites and leave the disk based ones for the moment. If we don’t need the 75K space, I’d say chalk this up to implement once the NEON code is in place. No point making work for yourself at this stage, when we’re close to 5.24 being released.

Going 16bit would allow for improvements to some sprites visually, so as a long term goal it’s possibly worth looking at with an artistian on board.

You mention RISCOS 3.1, where does that fit into this change? Are the disk based sprites shared with the legacy !System?

Are the disk based sprites shared with the legacy !System?

I wondered about that too. I’d wager that the majority of the stuff in !System doesn’t have sprites; after all it’s mainly stuff like CLib. The Toolbox modules might define a sprite or two though.

I imagine that it’d be possible to bring !Sprites11 and !Sprites22 up to the new format but leave !Sprites as-is for compatibility with 3.1. Does 3.1 use 22 at all? I can’t picture it ever using 11…

Does 3.1 use 22 at all?

MODE28? Anything that isn’t a glorified 50Hz TV…

Is space an issue in the ROM?

Not at the moment. But it would be a good way of saving space for IOMD, where we wouldn’t really want to switch to using compressed ROMs (although I’ve now realised that if we were to use compressed ROMs, we could go with a hybrid approach where half the ROM image is uncompressed and runs from ROM and the other half is compressed and runs from RAM – saving on the RAM overheads of using compression)

No point making work for yourself at this stage, when we’re close to 5.24 being released.

Yeah, it wasn’t something I was planning on doing now, just a note for the future.

Going 16bit would allow for improvements to some sprites visually, so as a long term goal it’s possibly worth looking at with an artistian on board.

Questionable, since the sprites are generally using custom 256 colour palettes, so are essentially 32bpp quality. And at Sprites22 resolution, each palette entry would be used on average twice, so there’ll be very little quantisation (34×34=576 pixels)

You mention RISCOS 3.1, where does that fit into this change? Are the disk based sprites shared with the legacy !System?

The boot sequence is compatible with RISC OS 3.1, and IIRC by default it’ll try loading one of the modern theme sprite sets. Which a (a) require lots of RAM and (b) are very slow to render, due to needing to map every sprite from its custom palette back to the default 16/256 colour palette. Tweaking the theme management to allow 3.1 to default to the built-in ROM sprites would be more sensible.

Does 3.1 use 22 at all? I can’t picture it ever using 11…

Yes, I think 3.1 is capable of using sprites 22. Sprites 11 is highly unlikely to be used, since there aren’t any builtin 180 DPI screen modes, and all the sprites 11 sprites are using RISC OS 3.5 sprite mode words :-)

we could save around 75K of ROM space by reprocessing the Wimp iconsprite set to all use RISC OS 3.5 mode words

saving about 300K of RAM for the Sprites11 set

all the sprites 11 sprites are using RISC OS 3.5 sprite mode words

I’m probably missing something (very little sleep last night) but don’t these statements contradict each other? If Sprites11 is already using 3.5 mode then how can we save 300k by changing them to 3.5 mode? Unless that “or newer” in your first post means something special…

Was a typo – meant to be “saving about 300K of RAM for the Sprites22 set” (now corrected)

Aha! :)

Looks like my estimate of 75KB saving was off a bit – converting the Wimp’s ROM icon sprites to use 3.5 mode words only saved 50KB.

https://gitlab.riscosopen.org/RiscOS/Sources/Desktop/Wimp/-/merge_requests/26

ON ERROR PRINT REPORT$;" at ";ERL : END

DIM sname% 13

I$="Sprites"
O$="Sprites2"
PROCcrunchfile(I$,O$)
END

DEF PROCcrunchfile(I$,O$)
 LOCAL sarea%
 SYS "OS_File",17,I$ TO ,,,,file_length%
 DIM sarea% LOCAL file_length%+8192
 sarea%!0=file_length%+8192
 SYS "OS_SpriteOp",256+9,sarea%
 SYS "OS_SpriteOp",256+10,sarea%,I$
 SYS "OS_SpriteOp",256+8,sarea% TO ,,,num_sprites%
 FOR snum%=1 TO num_sprites%
  SYS "OS_SpriteOp",256+13,sarea%,sname%,13,snum% TO ,,sname$
  SYS "OS_SpriteOp",256+24,sarea%,sname% TO ,,sptr%
  orig_size%=!sptr%
  mode%=sptr%!40
  REM Get rid of any left-hand wastage
  SYS "OS_SpriteOp",512+54,sarea%,sptr%
  REM If masked, ensure 3.5 format for smaller mask
  IF (mode% AND &FF)=mode% AND sptr%!32<>sptr%!36 THEN
   PROCconvertto35(sptr%)
  ENDIF
  SYS "OS_SpriteOp",256+24,sarea%,sname% TO ,,sptr%
  new_size%=!sptr%
  IF orig_size%<>new_size% THEN PRINT sname$;" ";orig_size%;" -> ";new_size%
 NEXT snum%
 SYS "OS_SpriteOp",256+12,sarea%,O$
 new_length%=(sarea%!12)-4
 PRINT "Final: ";file_length%;" -> ";new_length%;", ";file_length%-new_length%;" saved"
ENDPROC

DEF PROCconvertto35(sptr%)
 REM In-place conversion of old-style sprite to 3.5 sprite
 REM (Assuming no left-hand wastage)
 SYS "OS_ReadModeVariable",mode%,9 TO ,,log2bpp%
 SYS "OS_ReadModeVariable",mode%,4 TO ,,xeig%
 SYS "OS_ReadModeVariable",mode%,5 TO ,,yeig%
 pix_width%=((sptr%!16)*32+(sptr%!28)+1)>>log2bpp%
 in_ptr%=sptr%+(sptr%!36)
 out_ptr%=in_ptr%
 FOR y%=0 TO sptr%!20
  FOR x%=0 TO pix_width%-1
   in_bit%=1<<((x%<<log2bpp%) AND 31)
   IF in_bit%=1 THEN mask_word%=!in_ptr% : in_ptr%+=4
   IF (x% AND 31)=0 THEN !out_ptr%=0 : out_ptr%+=4
   IF mask_word% AND in_bit% THEN out_ptr%!-4=(out_ptr%!-4) OR (1<<(x% AND 31))
  NEXT x%
 NEXT y%
 change%=in_ptr%-out_ptr%
 REM Update mode word
 sptr%!40=((1+log2bpp%)<<27)+1+((180>>xeig%)<<1)+((180>>yeig%)<<14)
 REM Update offsets
 IF change%=0 THEN ENDPROC
 IF sptr%!32 > sptr%!36 THEN sptr%!32 -= change%
 sptr%!0 -= change%
 end%=(sarea%!12)+sarea%
 WHILE in_ptr%<end%
  !out_ptr%=!in_ptr%
  out_ptr%+=4
  in_ptr%+=4
 ENDWHILE
 sarea%!12-=change%
 SYS "OS_SpriteOp",17,sarea%
ENDPROC

I guess if we wanted more savings from the ROM icon/tool sprites, it’d be trivial to squash them. That would increase RAM usage when the Wimp starts and uncompresses them, but that’s not really an issue because 99% of the time you’re going to load in a much bigger disc-based sprite set, completely replacing the uncompressed copy of the ROM sprites.