Questions about Sprite formats

Hi,

I am currently trying to figure out all those different Sprite formats that are supported nowadays. I am working from PRM1 (“classic” Sprite format), PRM5a (RISC OS 3.5 Sprite format) and the ROOL Wiki.

Some things are confusing me and I would like some input from the experts before I start experimenting with !Paint and OS_SpriteOp to figure it out by experimentation.

Palette data confuses me. I understand 2 entries for 2 colour modes, 4 entries for 4 colour modes, 16 entries for 16 colour modes and 256 entries for 256 colour modes. But some docs seem to say that there are also 16 entries as well as 64 entries for 256 colour modes. I dimly remember technical details about VIDC1 and its palette registers, but can’t put it all together. I guess there must be a way to expand the 16/64 entries of the palette to 256 “real” colours that are finally seen?

In which situations are the “blinking colours” used/supported by RISC OS? I dimly remember that blinking was only supported up to 16 colour modes.

Then there is the mask data. I don’t understand if the “transparency data” is always part of the mask data, or if sometimes (for which sprite types?) the transparency comes directly from the sprite data. Or is that dependent if “wide mask” is active? And I also don’t understand why the Acorn bit savers decided to represent essentially 1 bit data with 8 bits in memory. Probably performance considerations.

Some unclear stuff from the docs at https://www.riscosopen.org/wiki/documentation/show/Sprite%20Mode%20Word

• sprite type 10 says “16bpp 5:6:5 TBGR” – surely that should be “BGR”?
• sprite type 16 says “16bpp 4:4:4:4” – TBGR, or something different (depending on what)?

If someone could enlighten me, that would be great. Does anyone have a collection of Sprites files which contain examples for all the variants?

Hi Steffen,

I begun to write a Sprite to Bitmap convertor for Windows, but haven’t got very far (I got as far as converting, as long as they don’t contain masks!!!). I also went through the various documentation and compiled a document, trying to make it more understandable (I tend to do that, so that when I come back to it later on, I can still understand it…hopefully).

I’ll upload it to my website at www.geraldholdsworth.co.uk in a moment.

Cheers,

Gerald.

Hi Gerald,

seems like this is no coincidence…as you may have read somewhere else, I am currently working on a Java-based tool to read content from filecore-formatted disc images. And I obviously came across your DiscImageReader tool (but not your documentation about the different disc formats – now that would have been easier than to work from the various PRM and online information!), but in the end decided to “roll my own”, because I wanted to learn how filecore works, have the tool available on all platforms I use and in a programming language I can develop without having to think about every line of code I write.

So now I have a tool that can read D/E(+)/F format floppy and harddisc images, and the sensible next step is to provide some capabilities to directly show various “native format” content – which lead to the development of a BASIC detokenizer, a Java Swing hexdump viewer and now a Java Swing Sprite viewer. Originally, I only wanted an easy way to show file and application icons in the Java Swing UI of the FilecoreImageReader, but once you have the minimum needed part of the functionality, there is some motivation to make it really complete…at the moment, my Sprite viewer supports arbitrary bpp and dpi values and “simple” palettes, but no masks yet.

Maybe I end up with a complete Java ImageIO handler for RISC OS sprites, which would be kind of cool.

And then…Squash? Spark? ZIP with filetypes? Templates? ARM code? Should keep me busy for the next few years :-)

Hi Steffen,

Yep, I did read about your Java based tool over on StarDot and thought it was a really good idea. I’m glad that my code, and documentation, has been of some use to at least two people that I know of.

The DiscImageReader, Delphi class, was written to be incorporated into the Repton Map Display (which has yet still has to be incorporated) – the tool itself is just a demo to show how to use the class. The Sprite Convertor was written as I needed Sprites, converted to Windows Bitmap, for use in my Repton Map Decoder. This also spawned the Atlas Creator, as I felt that storing the character sets as atlases to be easier.

Some more resources can be found via this wiki though it only scratches the surface.

But some docs seem to say that there are also 16 entries as well as 64 entries for 256 colour modes. I dimly remember technical details about VIDC1 and its palette registers, but can’t put it all together. I guess there must be a way to expand the 16/64 entries of the palette to 256 “real” colours that are finally seen?

I’m not sure how much support still exists for 256 colour sprites with 16/64 palette entries – so you may have to dig through the OS sources (ColourTrans, SpriteExtend, and the kernel?) to work out the answers to those questions.

In terms of VIDC1, the manual explains how the 16 palette entries are used to generate the 256 colours (and I’d assume that the sprite palette entries will directly correspond to the VIDC palette registers). http://www.home.marutan.net/arcemdocs/

In which situations are the “blinking colours” used/supported by RISC OS? I dimly remember that blinking was only supported up to 16 colour modes.

Yes, I think on Archimedes machines flashing colours were only supported up to 16 colour modes, due to the VIDC palette limitations.

On RiscPC and later, with a full 256 palette entry mode, the OS can theoretically support them, but I haven’t checked to see if it actually does.

Then there is the mask data. I don’t understand if the “transparency data” is always part of the mask data, or if sometimes (for which sprite types?) the transparency comes directly from the sprite data. Or is that dependent if “wide mask” is active?

The mode flags are the key. If bits 12-15 indicate that an alpha channel is present, then the alpha channel of the sprite data will be used for transparency information, and it’s illegal for the sprite to contain a mask.

If an alpha channel isn’t present in the sprite data, then the transparency will be taken from the mask (if present). And the “wide mask” flag indicates whether it’s 8bpp alpha or an Nbpp on/off flag.

And I also don’t understand why the Acorn bit savers decided to represent essentially 1 bit data with 8 bits in memory. Probably performance considerations.

Yes, I expect it was a performance consideration. If a mask pixel is the same size as a screen / sprite pixel then that greatly simplifies the sprite plotting (just two instructions needed to blend a word of sprite data into a word of screen data, for the best-case scenario of source and dest being at the same BPP and no palette translation needed).

sprite type 10 says “16bpp 5:6:5 TBGR” – surely that should be “BGR”?

Technically, yes. TBGR was probably chosen to match the terminology used in some other docs.

sprite type 16 says “16bpp 4:4:4:4” – TBGR, or something different (depending on what)?

The mode flags will narrow down the specific type (TBGR, TRGB, ABGR, ARGB)

Does anyone have a collection of Sprites files which contain examples for all the variants?

I’ve got examples of quite a few, from where I was testing the code for the new sprite formats. I’ll upload them when I get home.

Note that YUV format sprites aren’t supported by the OS yet, so it’s probably best to ignore those (there’s a chance the specification could change).

Thanks a lot Jeffrey, that clears many things up.

The Wiki page on Sprite Mode Word seems to suggest that the TBGR/TRGB/ABGR/ARGB information in the mode flags is only relevant for sprite type 16. That would be strange – is this correct?

When did the “wide mask” bit come into existence? PRM5a does not know anything about it, however it is listed on Sprite Mode Word page as a feature of the RISC OS 3.5 format.

Alpha channel sprites were a RISC OS Select invention, or was that already in RISC OS 4.0x?

The Wiki page on Sprite Mode Word seems to suggest that the TBGR/TRGB/ABGR/ARGB information in the mode flags is only relevant for sprite type 16. That would be strange – is this correct?

No. For “RISC OS 5” sprite mode words, the mode flags apply to all the sprite types. I’ve tweaked the wording on the page a bit to hopefully make it clearer.

When did the “wide mask” bit come into existence? PRM5a does not know anything about it, however it is listed on Sprite Mode Word page as a feature of the RISC OS 3.5 format.

RISC OS Select. It’s only listed on the “RISC OS 3.5 sprite mode word” because ROL fitted it onto that format of mode word (much like 64K colour sprites were fitted into the existing “3.5” mode word format).

RISC OS 3.5 sprite mode words have also been referred to as “new sprite mode words” in various docs (perhaps just internal OS sources/docs), but since RISC OS 3.5 is no longer new, and there’s a newer sprite mode word format available, it makes much more sense to refer to the two types as “RISC OS 3.5” and “RISC OS 5”, reflecting the major OS version the format was first introduced.

Alpha channel sprites were a RISC OS Select invention, or was that already in RISC OS 4.0x?

“wide mask” == “RISC OS Select alpha channel”

http://www.riscos.com/support/developers/riscos6/graphics/sprites/alphachannel.html

Also, “RISC OS Select alpha channel” != “RISC OS 5 alpha channel”

can be found via this wiki though it only scratches the surface.

Where on earth did they get the extension .acorn? I’ve only ever seen .ff9 and .spr.

Yes, I expect it was a performance consideration.

Remember, everybody, that the Wimp was quite functional on an 8MHz processor that didn’t have any cache whatsoever.

Also, “RISC OS Select alpha channel” != “RISC OS 5 alpha channel”

Confused yet? ;-)

Does anyone have a collection of Sprites files which contain examples for all the variants?

I’ve got examples of quite a few, from where I was testing the code for the new sprite formats. I’ll upload them when I get home.

http://www.phlamethrower.co.uk/misc2/sprites.zip

Note that some of them are technically invalid (3.5+ format sprite with left-hand wastage); I created those ones by manually editing the sprite mode word (e.g. converting a 1bpp mode number to the equivalent 3.5 & 5 sprite mode words). RISC OS still handles them, but treats them as if the left-hand edge starts at bit 0.

technically invalid (3.5+ format sprite with left-hand wastage

useful though as a screensave from an odd LH pixel created such sprites before the bug was corrected. Some apps handled them well others barfed.

Many thanks for the examples, Jeffrey.

“ASprites11” and “ASprites22” don’t work properly with my viewer – I didn’t expect the combination of 256 colours and alpha mask to be valid!

All others work fine, which gives me some confidence in my ability to “work from spec” :-)

http://www.phlamethrower.co.uk/misc2/sprites.zip

Yay! Ovation doesn’t crash or anything. ;-)

OMG – it even does alpha transparency! Here’s the alpha transparent ball over the colour swatch:

I guess Paint (and DPingScan and Draw and NetSurf) don’t support CMYK sprites properly? At least, I am seeing a “plain RGB” interpretation of the CMYK data.

I am seeing a “plain RGB” interpretation of the CMYK data.

Ditto with Ovation, I think it may be RISC OS that doesn’t (yet?) support CMYK sprites…

Correct, CMYK sprites aren’t supported yet by RISC OS 5.

http://www.riscos.com/support/developers/riscos6/graphics/sprites/cmyk.html

Is there anyway of using PNG files – in Ovation – without them being converted into sprites?

ie like EasiWriter does.
Could the JPeg file handling – be made to manage PNG files as well?

Could the JPeg file handling – be made to manage PNG files as well?

The short answer is no. JPEGs can be displayed by the OS using SpriteExtend module etc. There are no facilities in the OS at the moment to display PNGs using an analogous method. Therefore the simplest way is to convert to sprite before displaying.

Even for jpegs there are no inbuilt routines for processing the image. Eg. in OPro, a jpeg is displayed and kept as a jpeg until you do some processing on the image. Then it is converted to a sprite to enable processing options (brightness, contrast, gamma and even non- 90deg multiple rotation) to be done.

A very early release of my Java-based SpriteViewer and SpriteConverter is now available for download:

Windows:
https://www.huber-net.de/riscos/SpriteViewer_0.1.0_Windows.zip

Java generic:
https://www.huber-net.de/riscos/SpriteViewer_0.1.0_Java.zip

You need at least Java 7 installed on your computer to use this. Error handling is rudimentary. GUI is rudimentary. It is not really well-tested, but seemed to work for all Sprites (including all of Jeffrey’s test sprites) I have thrown at it. Please let me know if it works for you.

Conversion to other formats is currently only available from the command line, Gerald could have a go trying to convert to Sprites to bmp format (in theory, it should work for everything that Java ImageIO supports).

Conversion to other formats is currently only available from the command line, Gerald could have a go trying to convert to Sprites to bmp format (in theory, it should work for everything that Java ImageIO supports).

…and I am learning a lot about Java ImageIO and the various format implementations while experimenting. It turns out the JPEG written is basically broken if you give it an image with an active alpha channel (even if no alpha is used – to keep things easy, my Sprite reader always uses alpha representation and I naively thought this would not make any difference) – all colours are wrong then. It looks like the only solution is to convert to a non-alpha Image representation before writing to JPEG. Strange but true.

It also looks like the inbuilt BMP write support does not support transparency at all, which somehow ruins Gerald’s use case. While there are ImageIO extensions for many formats like many TIFF variants and even good old Amiga IFF, I have not found a freely available BMP extension supporting transparency.

I had no idea that Java’s built-in image handling was so bad. I suppose I was spoiled by Python’s de-facto standard Image library, though I was only ever interested in converting sprites to PNGs.

Sometimes I think about rewriting some of my tools in Go. Hopefully image support in the Go standard library is actively maintained.

I had no idea that Java’s built-in image handling was so bad. I suppose I was spoiled by Python’s de-facto standard Image library, though I was only ever interested in converting sprites to PNGs.

As always, the devil is in the detail. In Java’s long history, there was internal image handling (via AWT/Toolkit), “Java Advanced Imaging”, “Java ImageIO” and countless 3rd party libraries (the most prominent was Sanselan, which is now Apache Commons Imaging, which has not seen a release declared as “stable” for a very long time now). Additionally, for server-side imaging stuff, there were often native libs used, ImageMagick probably most often.

It is a mess. But once you identified the many weaknesses, it becomes halfway usable. StackOverflow is full of the same questions over and over for seemingly trivial things.

Sometimes I think about rewriting some of my tools in Go. Hopefully image support in the Go standard library is actively maintained.

I haven’t used it much, but Go’s image support feels a bit bare-bones to me.

In terms of representing sprites, one thing which it lacks is support for masks, so you’ll have to convert everything to an alpha channel format. And although the palette can contain alpha, it’s limited to 256-entry palettes, so you won’t always be able to represent the mask/alpha as additional palette entries and will have to convert to RGBA instead.

It does have support for CMYK, but with no alpha channel – so mask/alpha CMYK images would need converting to RGBA as well.

And of course there’s no concept of flashing colours in the palette (render out an animated GIF which switches between the two palette states?)

Steffen, the first sprite file I tried rendered incorrectly in your viewer. But then, I was mean and tried the most difficult I could find in my test folder for the ArtWorks sprite rendering engine: 64k colour with RGB ordering. I will put it in an e-mail to you. A few others that I tried worked fine.

Yep, I got the bit shifts wrong for 5:6:5. Good find. Now fixed.

After some good advice from Toni Reiser and Jan de Boer about the “16 palette entry 256 colour format”, I am currently finishing support for those arcane variants, after that a proper release will be done. Thanks to all who have provided feedback!