RFC: Long sprite names

I’ve been working on a cunning hack to allow all sprite files and applications to support long sprite names, but there are some repercussions and I’m interested in feedback before I waste too much time.

The intention:

• The limit of 12 chr spritenames is transparently lifted in a backwards-compatible way.
• New functionality allows intelligent use of long sprite names, falling back to shorter names automatically.
• Applications can use long leaf names and still have a unique icon (eg !ContrivedName1 and !ContrivedName2)
• Filer can display customised versions of sprites (with a long name) defaulting if not present to the shorter name.

The new functionality long sprite names allow is this: If a long sprite name is not found but contains a dot, it is truncated at the final dot and tried again. This allows the Filer for example to ask for “file_fff.selected” or “!draw.open” (compare with “directoryo”) without worrying – it would get the specialised sprite if available, or plain old file_fff or !draw otherwise.

That much is straightforward and works (don’t look behind that curtain). However, weaning applications off the firm belief that spritenames are 12 characters turns out to be much harder than expected.

The only call that returns sprite names is SpriteOp 13. Nothing uses it. I mean nothing. Nothing except *SList anyway, and it passes it a 12 byte buffer anyway! Worse, Op13 doesn’t return the actual length of the sprite name, it returns the amount it managed to fit into the buffer. D’oh!

It does this even if the given buffer length is zero, or the buffer pointer is zero (and in that case it will happily copy the sprite name into zero page at the same time). Uggh.

Even if SpriteOp 13 were useful (and were used!) there is a difference between finding sprites by long sprite name and returning long sprite names. I won’t go into the horrific implementation details at this stage.

My questions are these:

• Do people think long sprite names would be useful?
• Is auto-truncation a valuable additional functionality?
• Is it onerous to expect authors to use SpriteOp 13 instead of peeking bytes (which is what everything does)?
• Is it acceptable to slightly redefine (extend) SpriteOp 13’s behaviour so sprite name length can be detected without requiring a maximal buffer (eg ptr=0)?

Backwards compatibility is tricky, but works like this:

• Programs that directly peek sprite names (which is all of them :-[ ) see a unique truncated version of long sprite names, eg verylongspritename appears to be “verylo…”.
• Both long and short names can be used to select the sprite, but only the long name is fixed (short names can change on sprite merge for example to remain unique).
• Long names are finally truncated to 12 chrs if not found (in case applications are using unterminated sprite names).

Thoughts, suggestions and blank stares welcome.

How is the cunning hack implemented? I’m assuming you’re storing the long sprite name somewhere after the sprite header. But where exactly? Or is there a fake sprite within the sprite area which contains all the long sprite names instead of pixel data?

Programs that directly peek sprite names (which is all of them :-[ ) see a unique truncated version of long sprite names, eg verylongspritename appears to be “verylo…”.

So MS didn’t patent the longfi~1 botch then?

Is auto-truncation a valuable additional functionality?

Just plain truncation? What happens with “verylongspritename1” and “verylongspritename2”?

Is it onerous to expect authors to use SpriteOp 13 instead of peeking bytes (which is what everything does)?

I wouldn’t say onerous, but given that everybody peeks bytes, I also wouldn’t expect the world to suddenly rediscover SpriteOp 13.
Is there a byte-peek way to read the long names?

Is it acceptable to slightly redefine (extend) SpriteOp 13’s behaviour so sprite name length can be detected without requiring a maximal buffer (eg ptr=0)?

Provided that such a tweak will not alter existing expected behaviour, perhaps by the use of an unlikely magic word (akin to why we tell the Wimp that we are a “TASK”).

Both long and short names can be used to select the sprite, but only the long name is fixed (short names can change on sprite merge for example to remain unique).

In that case you will need a SpriteOp to allow the relationship between names to be determined.

Long names are finally truncated to 12 chrs if not found (in case applications are using unterminated sprite names).

I seem to remember some oddity with sprite termination and !Paint. Aren’t they space padded? I think I terminated with a null byte and !Paint blew up. I can’t be more specific, it was a decade ago. At any rate, I’d just like to say thank you to whoever fixed that annoying long-standing bug in !Paint where the last row/column on the right/top(IIRC) couldn’t be selected in a dragged-out selection, only by itself.

An additional question. Something that bothered me, but not enough to do testing.
Sprites have a limitation to the name. Directories no longer do. What sprite should be used for !VeryLongAppName?

Despite me writing “I won’t go into the horrific implementation details at this stage”, Jeffrey asked:

How is the cunning hack implemented? I’m assuming you’re storing the long sprite name somewhere after the sprite header. But where exactly? Or is there a fake sprite within the sprite area which contains all the long sprite names instead of pixel data?

OK Jeffrey, since it’s you. Everyone else can stop reading this post, I’ll answer other questions separately.

One can support long sprite names without storing any extra data, but only in a ‘write-only’ sense. If you were wondering why I was truncating long sprite names to a seven character ‘short’ name, that should give you the necessary clue.

However, as I implied, returning long sprite names is another matter, and does require storing them somewhere. Your two suggestions are the only ones possible while maintaining backwards compatibility, but both have significant problems… so I actually do both, but not at the same time:

Sprite files have an extension block after the header (using the draw-object idiom that is the de facto standard for that otherwise undocumented area), but such a block can’t be updated while in memory because of SpriteOp 24 – sprites can’t move just because you’ve renamed one. So on loading (or the first time a sprite area is seen, if loaded manually) this extension block is converted into a hidden sprite. And vice-versa on saving. (Code that saves sprite files ‘manually’ are a pain but the code copes).

Note that all of that smoke and mirrors is only required to return long sprite names. They can be used without any additional storage or data.

In practice, it is only really sprite editors that need to get at sprite names, everything else already knows the name of the sprite it wants to access, so just asks for it. That works just fine. The only exception to that might be in the light of automatic truncation, as it is conceivable the application might want to know the actual sprite name selected, rather than the one it asked for – hence if !Draw.selected is not defined and !Draw is used instead, some other form of highlight would have to be employed.

SpriteOp 13 really is useless in every regard – if you already know where a sprite is, you can’t use SOp13 to read its name, you need to know its index, which no other call returns. Sigh.

BTW, I’m currently not allowing long sprite names to be applied to a sprite area unless long sprite names have been enabled for that area via a new SpriteOp – again for backwards compatibility, as it is possible that an old task could be renaming or creating sprites with unterminated names. I suspect that the only solution to the ‘read sprite name’ requirement is to use that same SOp for that purpose too… I think SOp13 is beyond redemption.

Rick asked:

Just plain truncation? What happens with “verylongspritename1” and “verylongspritename2”?

I did write:

If a long sprite name is not found but contains a dot, it is truncated at the final dot and tried again.

And for backwards compatibility:

Long names are finally truncated to 12 chrs if not found

Rick went on:

Is there a byte-peek way to read the long names?

Go and stand in the naughty corner.

Sprites have a limitation to the [length of the] name. Directories no longer do [have such a short restriction to leafnames]. What sprite should be used for !VeryLongAppName?

Currently, whether you ask for ‘!VeryLongAppName’ or ‘!VeryLongAppTitle’ the sprite code pretends you asked for ‘!VeryLongApp’ in both cases. This modification allows us to have sprites called ‘!VeryLongAppName’ and ‘!VeryLongAppTitle’ without confusion. For backwards compatibility though, existing editors such as !Paint would think they were called ‘!VeryL…’ and ‘!Very…1’ (and they can also be accessed with those names, or !Paint wouldn’t work).

Another of the things this scheme allows, and again it’s Filer related:

If HostFS::SomeDisc.$.Utilities.Example is a directory, then Filer builds this sprite name:

directory.example.utilities.$.somedisc.hostfs

The sprite system automatically looks for the following sprites, in order:

directory.example.utilities.$.somedisc.hostfs
directory.example.utilities.$.somedisc
directory.example.utilities.$
directory.example.utilities
directory.example
directory

Which is a neat way of having unique directory icons without requiring filing system support, additional metadata or hidden files.

Since SpriteOp 24 can be used to return a pointer to the actual sprite selected, the overhead of the incremental search doesn’t need to impact upon redraws.

Another Filer/Wimp hack I’d worked on was Hover, which does things like opening directories if you hover over them while dragging a file icon. I’m indicating the hover action with a directory icon with a yellow arrow pointing into it (called directoryh), but autotruncation would allow the principle to be extended – spritename.!hover! for example.

Despite me writing “I won’t go into the horrific implementation details at this stage”

Ah, I must have missed that bit. Sorry!

I think the intention of OS_SpriteOp 13 is to allow you to enumerate all the sprites in a sprite file/area. Most of the other calls either accept a sprite name or a sprite pointer; unless you traverse the sprite area manually the only way to get a pointer is via the sprite name (e.g. OS_SpriteOp 24), and unless you know the sprite name already the only way of getting a list of names (short of traversing the area manually) is via OS_SpriteOp 13. So although it’s mostly useless, it’s not entirely useless, and well-written software (or at least stuff which has no reason to do low-level interaction with sprites) should be using it to enumerate the sprites in the area.

I think the intention of OS_SpriteOp 13 is to allow you to enumerate all the sprites in a sprite file/area

Indeed. But nothing seems to use it except *SList.

The problem with SOp13 as API is that there is no way to detect how long the buffer is required to be. This is a misdesign. In effect you have to do this:

length%=16
REPEAT
  buffer%=FNalloc(length%)
  SYS"OS_SpriteOp",&10D,area%,buffer%,index%TO,,,fitted%
  IF fitted%=length%-1:PROCfree(buffer%):buffer%=0:length%=length%*2
UNTIL buffer%

Nothing does, and why would it?

Say sprite 1 is named “examplename1”, SOp13 currently behaves in the following unhelpful ways:

SYS&2E,&10D,area%,buff%,0TO,,,L%:REM buff unchanged, L%=0
SYS&2E,&10D,area%,buff%,1TO,,,L%:REM buff contains "\0", L%=0
SYS&2E,&10D,area%,buff%,4TO,,,L%:REM buff contains "exa\0", L%=3
SYS&2E,&10D,area%,buff%,12TO,,,L%:REM buff contains "examplename\0", L%=11
SYS&2E,&10D,area%,buff%,13TO,,,L%:REM buff contains "examplename1\0", L%=12
SYS&2E,&10D,area%,0,0TO,,,L%:REM L%=0
SYS&2E,&10D,area%,0,4TO,,,L%:REM "exa\0" written to address 0, L%=3

I think we could safely change the last two of those to return actual name length. However, my tentative SpriteOp 23 (Long sprite names) is currently:

> R0 = &x17
  R1 = sprite area if R0 >= &117
  R2 = 0 to disable long sprite names for the area
       1 to enable long sprite names for the area
       ptr (>4095) to sprite name to read actual sprite name
       ptr to sprite if R0 = &217 to read actual sprite name
  R3 = buffer for sprite name if R2 > 4095, or 0 to read length
  R4 = length of buffer if R2 > 4095 and R3 <> 0
< R4 = length of spritename excluding terminator

I think I’d prefer SOp13 to be useful, as that behaviour could be built-in now, allowing code to do:

SYS"OS_SpriteOp",&10D,area%,0,0,index%TO,,,length%
IF length%=0 THEN length%=16
buffer%=FNalloc(length%)
SYS"OS_SpriteOp",&10D,area%,buffer%,length%,index%TO,,,length%

Which is backwards compatible.

The problem with SOp13 as API is that there is no way to detect how long the buffer is required to be. This is a misdesign.

Its hardly a misdesign the sprite name is defined as fitting in a 12 byte buffer so you can safely use that.

Edit: except of course you pointed out that it returns a zero terminated string so you need 13 bytes :-). Easy enough to allocate 13 bytes on the stack in C.

You’ve really missed the entire point of this thread haven’t you?

Go and stand in the naughty corner.

Aren’t I already there? ;-)

Thing is, though, that those who are used to extracting sprite names directly (not me, I hasten to add, I find a SWI call to be less hassle than a byte peek routine) will suddenly change their ways. You know it’ll happen…

Currently, whether you ask for ‘!VeryLongAppName’ or ‘!VeryLongAppTitle’ the sprite code pretends you asked for ‘!VeryLongApp’ in both cases. This modification allows us to have sprites called ‘!VeryLongAppName’ and ‘!VeryLongAppTitle’ without confusion.

…reading literally, this means that while you can have sprites called …name and …title! it will ignore the suffix? I must have misunderstood.

Otherwise, it sounds like a feasible way to provide support for long sprite names without creating a completely new sprite format.

You’ve really missed the entire point of this thread haven’t you?

Well I often do that -I did post under the ‘blank stare’ category – but you are using the ‘illogical’ behaviour of SO13 as an argument for changing it and its not. How SO13 works is irrelevant. It does work as specified.

You can’t use a zero page buffer with any length value other than 0 because if the code finds itself on an old machine it will crash.

So your question boils down to can you think of a situation where a buffer,len pair of 0,0 would be used and expect a length of 0.

No

What happens if I don’t know about long spritenames create a spritefile on a machine with longsprites ensuring all the spritenames are 12 chars

I then move to a machine without long spritenames will I get the correct spritenames.

If the answer is in your post to Jeffrey I didn’t read it – you told me not to.

What happens if I don’t know about long spritenames create a spritefile on a machine with longsprites ensuring all the spritenames are 12 chars
I then move to a machine without long spritenames will I get the correct spritenames.

You should. I don’t know how nemo’s code works, however I would expect it to be smart enough not to attempt to shorten a name that would fit naturally. It may need a shadow long name entry, but this shouldn’t make any difference to what you are asking.

The what-the-hell-dude question (just to test nemo ;-) ) is if a person creates a sprite file on a machine with long names, edits it on an older machine (including lots of renaming), plus merging in a few extra sprites, then tries to access the sprite file on a machine with long names…..how badly will it blow up?
Not as psychotic a question as it might seem – given the large installation of RISC OS, it is quite likely that sprites may be edited on a mixture of capable and non capable machines. The capabilities of this system will be defined not just by how it works but also by how it fails, and yes, it is allowed to fail gracefully. It should not fail disgracefully (that’s what Real Life is for).

Rick exclaimed:

…reading literally, this means that while you can have sprites called …name and …title! it will ignore the suffix? I must have misunderstood.

In the absence of my hackery, you can’t have a sprite with a name longer than 12 characters. You can have applications with a name longer than 12 characters though (whether that’s a good idea or not is Somebody Else’s Problem right now), so we have a problem.

Exacerbating the problem is the fact that the API has always silently truncated longer sprite names – if you ask for !VeryLongAppName you will get the sprite !VeryLongApp – not a “not found” error.

Therefore, though I now have the ability to create a sprite called !VeryLongAppName, there would be difficulties if that functionality were introduced naively. For example, it is currently permissible for an application to create a sprite called “ThisWillAnnoy” (ie pass a pointer to that test when creating the sprite) and then use it by asking for “ThisWillAnno”. If long sprite names were introduced, that contrived case would now fail.

Aside: My one-man crusade against implementation limits led me to extend the length of filetype names – ie *Set File$Type_AFF Draw Vector Graphic File. You wouldn’t believe the amount of trouble that caused.

So I am having to exercise extreme caution – long sprite names are only allowed in sprite areas that have been ‘enabled’ to have long sprite names. Sprite names longer than 12 characters that cannot be found are truncated to 12 chrs and tried again, just in case. It’s still possible to contrive failure cases, such as adding a sprite with a long name to a sprite file which a badly behaved task is going to manipulate with incorrect length sprite names… but if you worry too much about what might happen you’d never get out of bed.

Colin replied:

How SO13 works is irrelevant. It does work as specified.

It does. But it is badly specified. To use it you need to know how large a buffer is required. That’s an implementation detail, and should not be part of the API.

Acorn were very good at not putting implementation details into APIs on the whole, but the sprite support is absolutely ancient (and predates ARM of course).

Rick asks the important question:

if a person creates a sprite file on a machine with long names, edits it on an older machine (including lots of renaming), plus merging in a few extra sprites, then tries to access the sprite file on a machine with long names…..how badly will it blow up?

Like my other modules, LongSprites will be distributable, so software that needs it can include it, and people can retrofit it. So there’s actually four different cases:

• Long name aware editor with LongSprites

Long sprite names work by magic!

• Legacy editor with LongSprites

Editor will display only short names. As long as it only manipulates the sprite area via SpriteOp, everything will be fine. If it fiddles with the area directly, the worst that will happen is that the ability to return long names will be lost – it will STILL be possible to access the correct sprites by their long name.

• Long name aware editor without LongSprites

What are you doing!

• Legacy editor without LongSprites

You probably aren’t going to be able to edit the spritefile. Any sprites with long names will not be displayed, can’t be edited, and the errors that SpriteOp returns due to not finding sprites the editor believes are there are likely to cause it to quit. This is down to the disparity between how naughty applications “peek” the sprite name, and how the sprite code itself actually finds them by name.

So the only limitation is that you can’t use sprites with a long name on a system without the LongSprites module… but you wouldn’t expect to be able to would you?! Important point: You can use a sprite area containing long sprite names, you just can’t access those that have a long name. Those with short names will still be fine.

So passing around spritefiles will fail the invisibility test. If you have to go to the lengths of distributing a module why not create a new spritefile format.

Incorrect. Your sprite file can contain “!VeryLongApp”, “!VeryLongAppName” and “!VeryLongAppTitle”. In the absence of the module the first sprite will be used, which is all that was ever possible anyway.

If you have to go to the lengths of distributing a module

Applications do this all the time. You include it in the application directory and add an RMEnsure to the !Boot. What “lengths” is that?

Spritefiles are not applications. The only way you know how to decode them is by their type. Applications can carry decoders around with them files can not.

I know what you are trying to do and you obviously feel its dodgy else you wouldn’t have asked about it. I’d prefer being explicit and having a new sprite format but I don’t feel that strongly about it. Only time will tell if hijacking the sprite filetype for long filenames was a good idea.

My concern wasn’t that it was “dodgy”, but that it would be a waste of my time.

The thunderous silence on the question of whether it would be useful confirms my suspicions.

:-) Disinterest of the seven people who frequent this forum is hardly representative of RISC OS as a whole. The other 10 RISC OS users who don’t frequent this forum may be very enthusiastic.

I agree though that it can be disheartening when you ask something and tumbleweed drifts across the screen. I did try to engage but don’t have a use for long spritenames.

Neglect ye not the silent, lurking majority :-)=

OK. I have been silent because it would hardly affect me. I am not uninterested, but disinterested.

Do people think long sprite names would be useful?

Undoubtedly, if they were the standard from the start. I can think of a handful of occasions I have had to truncate or adapt spritenames to fit.

Is auto-truncation a valuable additional functionality?

Yes, in that it would seem to be neccesary for sensible backward compatibility.

Is it onerous to expect authors to use SpriteOp 13 instead of peeking bytes (which is what everything does)?

On the one occasion I needed to find sprite names I used SpriteOp 13. So it is not only *SList that uses it. ;-)

Is it acceptable to slightly redefine (extend) SpriteOp 13’s behaviour so sprite name length can be detected without requiring a maximal buffer (eg ptr=0)?

This would have no bearing on anything I have done.