Clipboard API Compatibility?

Having found myself spending a bit of time reading Wimp specs recently, I find myself with a question regarding the new Clipboard support in writable icons.

Over on the other branch, when they added cut and paste to icons, they added a new C flag to the K validation command, which prevented clipboard operation in an icon where it was set.

I’ve had a read through our spec and despite most of the API looking fairly similar (however, see below), I can’t see a mention of it.

That said, is it me, or is the extended register usage on Wimp_GetCaretPosition and Wimp_SetCaretPosition inverted between the two implementations? Over there, R0 = “TASK” for extended functionality, while R2 holds the reason code; over here, “TASK” is in R2 and R0 holds the reason code. Then add to that the fact that over here, the new Wimp_GetCaretPosition block for selection data seems to contain a superset of the data returned over there, but in a different order so apps can’t even just rely on the subset of data being present in all implementations.

I realise that we were implementing Acorn’s spec (allegedly), but have we really prevented applications from using one set of code on both platforms?

Having found myself spending a bit of time reading Wimp specs recently

Also worth digesting the docs in the sources, there’s a mention of validation strings there, and might just mean the other is out of date with the implementation.

That’s a helpful document; thanks! If correct, it makes it possible to document the Kc option for any Wimp with clipboard support, which is a simpler message to explain.

Interesting that it doesn’t mention the “TASK” API for Wimp_Get/SetCaretPosition at all. I might have a look at the sources if I’m feeling bored, although I don’t actually need that bit for what I’m doing – it was just the divergence that seemed odd.

Interesting that it doesn’t mention the “TASK” API for Wimp_Get/SetCaretPosition at all.

Huh? You did notice there were two links didn’t you?
That’s what the second link points to.

Steve – having looked at both ROL and ROOL clipboard specs (quickly), my reading matches yours on the caret SWIs – the two specs seem quite at odds, and it is difficult to see how code could work on both without separate code paths.

I’m not sure how much code is using either API, but agree that it should probably be standardised where possible. There may be political reasons for preferring one way over another, but RO5 has the fortune to be “fluid” (ie. actively developed), whereas the ROL spec will never change, so I think the onus is probably on us to avoid clashes except where fundamentally necessary.

Huh? You did notice there were two links didn’t you?

Yes… because I originally supplied one of them in my first post.

That’s what the second link points to.

You mean the link that I originally posted at the top of the thread, mentioning the API which appears to be reversed from the ROL one?

The second document, which Sprow highlighted amongst the source, doesn’t mention this incompatible use of “TASK” passing at all (as far as I can see). Which raises the question of whether the document in the Wiki (the one I originally linked to, and you’ve just kindly told me to read) is real or aspirational. That is, did the API for Wimp_SetCaretPosition in fact end up being implemented as here, with no use of “TASK” as a mystic rune.

Here’s my vague recollection, if anyone cares.

Clipboard support within the desktop was one of the top requests repeatedly from people, within applications. The clipboard protocol within the desktop had been defined by Acorn some years earlier and had low but not insignificant uptake. The protocol was tedious, however – I’d implemented in a number of my applications and I wasn’t especially happy with how to use it. Unlike (let’s say) the Throwback system, which had a protocol which you could use and SWI calls to make it easy to work with, you had to do a lot of the work yourself, and that included keeping hold of the data maybe even after your application had closed the document that things were copied from ¹. It’s all pretty sensible really, because you can convert to a suitable format for different users, but the barrier to entry was high – small applications suffer.

Added to this, the protocol doesn’t allow for the means by which an application can exit and the user would still expect the clipboard contents to be present. This was an issue that had been encountered by the NC project, and Acorn created the Clipboard module to provide a mechanism by which the clipboard content could be transferred to the Clipboard module when the application exits. This then allowed the applications to be exited on the low-memory NC systems, but to retain their data as the user might expect. Since the user interface was entirely the web browser, the implications of the Wimp not having clipboard support were pretty irrelevant to that project.

Coming back to RISC OS for desktop, the Clipboard module was not one of the ones that was part of RISC OS 4. As such, I created the ClipboardHolder module, whose function was similar (in that it holds the clipboard for applications that exit), but more specifically it allows a simple SWI interface to access the clipboard, most of the time. There’s limitations, and for full support you may need to fall back to the complete protocol, but the barrier to entry for most users is low when you only have a single SWI to call to hand off the content to the clipboard. ²

Adding support to Edit, Draw and Paint (does Paint have it? I can’t remember) for the clipboard was therefore significantly easier, because essentially you just serialise the content in memory, and call one SWI – job done. Which was the point of those calls. To make it simpler to handle those cases where you don’t need to mess around. Of course, I should have added the full protocol support, but that was a job for another day as the base implementation was good and began to make the built in applications more useful and in line with the standards.

Let’s jump back to the Clipboard protocol itself for a moment. There was a second protocol released at the same time which was related – the drag and drop protocol. The drag and drop protocol was relatively nice, and quite well thought out. Datapower used it, and I think you could see it interact with some other applications (Writer? Impression? Sorry, I don’t really remember). If the clipboard uptake was low, the drag and drop protocol was next to non-existant. I’d tried to use it, but the things that I was working with didn’t lend themselves to the protocol, and it wasn’t easy for me to experiment with. The two systems are distinct but cover similar things – the transfer of data between applications by the user.

With the standard applications supporting the Clipboard, and support to lower the barrier to entry for developers in place, the main thing that was missing was the UI itself – specifically the Wimp.

Patches had existed for clipboard for some time. Using FilterManager, WimpSWIve, and other techniques, it was possible for modules to provide clipboard support in regular icons. Some applications, of course, provided their own clipboard support, and this would complicate any implementation that was going to happen. But I’ll come to that. My own drag and drop module for icons (TextCopy, I think it was) had a support module that had to change all the text icons to be write-click-drag, because otherwise only some (and the user wouldn’t know which ones until they tried) would have drag support. IconClipboard (another module whose actual name I forget) handled the operations with a filter, I believe for keyboard copying. None supported highlighting of icons or text, as I recall.

Ben Avison, I believe it was, had written a proposal for the way in which the Wimp could be extended to handle the clipboard. It was involved and combined some of the features of the Drag And Drop protocol with the clipboard – the concept of ghost regions in particular, and a number of more complex operations. Forgive me if I have got that wrong Mr Avison; I’m going from hazy recollections of the documentation. It used some interesting extensions to the caret system, and required a lot of reworking of the innards of the Wimp.

Time’s always a problem, and getting a viable solution without overstretching is important. I put off implimenting the proposal for a long time – it was involved, it had lots of risk, and it wasn’t obviously easy to deliver in stages. Getting the drag and drop part to work, and ghost regions was very daunting ³ and I didn’t necessarily feel that building that into the Wimp was the right thing ⁴ for right at this second. I wanted to farm out the rendering of the icons to another component so that it could worry about the drawing and user interactions. That was almost immediately a non-starter.

So, stalled (or putting it off, depending on how you look at it) for some months (years?), I decided ‘ it, think differently…’ What do I actually need to get done, regardless of what might have been decided and nobody had actually implemented or done anything with? We need a way to select text. For the user to copy it. For the user to paste it. That’s it. For that to happen, we need the concept of a selection. We need keyboard and mouse selection of text. And we need a presentation that shows the text as selected. Just going ahead and implementing the presentation of the selection was easy. The interface in the clipboard extensions document was awkward and as I recall, didn’t allow for the user to be able to tell that their operatiosn were effective or not – there were no obvious markers to show that a different operation was being performed when the selection was being requested, and still be safe to use it on an old system without being destructive. Again, I could be misremembering, but that’s what I recall.

The protocol document used (I think) some markers that would (in the new version) be recognised. That’s not how most of the other Wimp SWIs were extended. Most used the TASK mechanism for recognising different operations. If the TASK word were put into a high register which wasn’t otherwise used, then old Wimps would not see it and the operation would be destructive. So the difference in operations needed to be in the existing registers so that users could recognise that the operation they performed was wrong for the system, because it reported an error. Thus, putting TASK into R0 was reasonable, and handy.. R0 contained the window handle, and the window handle is constrained by certain rules – it must have bit 0 set, and it must not be negative unless it is -1 or -2. The word TASK is not negative and doesn’t have bit 0 set, so it would be flagged on older systems as being a non-window handle, and thus rejected.

The interface then takes a parameter block in r1 (like most of the Wimp calls do), and the operation to perform in R2, which is a little different to most, but watayagonnado… The data in the block in r1 is identical to the parameter block used for Wimp_GetCaretPosition, for both get and set calls, so that simplifies understanding what’s going on for the basic calls. The register allocation is therefore simple; just the data content of the block changes with the operation type. Similar to the way that the contents of the block change on SendMessage, I guess, but I don’t think that was the intention.

The layout of the operations was kept the same (I believe) as the original specification, with a gap left for ghost selections. When I finally got around to it, the interface could be extended either in the direction of the original document, or other ways, beceause the interface was relatively flexible.

There’s also an extended operation allowed in the Wimp_DragBox interface which is actually just exposing the internal interface to ‘start an icon selection operation’ ⁵.

This also caused quite a bit of back-and-forth for me on ‘applications which had already implemented their own clipboard support which would find they didn’t work with the new handling of the keys’. I tried to work out ways in which those operations could be made to coexist, maybe by passing the control key on even though it was acted on, or seeing whether the application changed the icon and operating after it had completed. In the end, and after a few tries at finding a way to work around that, I decided ⁶ it wasn’t possible to be safe all the time if I left the application to do its thing. So I chose to entirely mask the control key presses from the application’s sight, but to keep the insertion of text happening as expected. The application doesn’t see the ctrl-c or ctrl-v operation, but it does see a whole bunch of key press operations and the icon updating with the text. That way if it wants to on paste, it can post process the content. I’m thinking of applications which have number fields that don’t let invalid things in – typing “2.3.” would be valid if the icon had a validation allowing the numbers and dots, but the application could handle the second ‘.’ and stop it being left in the icon because that’s invalid. By inserting keys for the clipboard operation, that ensures that those cases keep working. Why’s that matter? Because some people do it. Because the Toolbox does it. Because applications expect to be told what’s going on, and not to have things change underneath them.

But some applications will want to do it their own way. I can’t detect them up front, and if they want to go against the consistent interface of the system then that’d be bad, but I don’t want to dictate to them that they must work that way – because they might be industrial control machinery ⁷ and not want to use the same methods that are sensible for the Desktop. For them, there’s an override option ⁸ so that they can say ‘Yeah, I know I just don’t want you screwing with things, ok?’

Why go a different way to the original proposal? As I’ve said, I’d procrastinated on implementing it because of the larger amount of work it needed. But also, it was many years later now, and no progress had happened at the time (for a variety of reasons, I’m sure) and none was likely to happen. Being based on the work of the Drag And Drop protocol, which had near-zero take up, might be the right thing to do in the long run, but it wasn’t going to gain much in the short term. So implementing an excessive (for the initial purposes) proposal with an API which I felt was unsafe for use by applications on earlier systems, wasn’t going to happen. But, implementing what I could of the intent, with a safer API, and with the ability to extend in the future… yup, that met the goals and gave user and developer features that were functional.

After it was implemented, there were interesting questions about presentation. The highlighting of the text should follow the caret colour, but be obvious. To meet the goals of the OS (and style guide requirements) that applications be able to work together seamlessly, it was absolutely necessary for the highlighting colour of the selection to be readable by the application. All applications should be able to know about the things that the Wimp is doing to fit in with it. If the highlighting of the text is readable, it should probably be writable by the user too. That fits with the goals of having a themable desktop which doesn’t force colour or styling on the user (q.v. window tool changes, 24 bit colour, icon greying, button shapes). And of course, different systems use different ways of selecting icons – some automatically highlight the text in an icon when you click in it, and some just place the caret. Obviously RISC OS’s default had been to just place the caret, but with the existance of the highlighting changes, this could be changed. Similarly, do we allow the caret to be only at the end of the icon, and if you type into the icon should it delete the selection (that’s what some systems do), or insert at the current caret position? These all became configurables, partly for flexibility, and partly because I couldn’t decide between keeping the original default behaviour and being less friendly to people (do you really want everyone to use click-ctrl-a-delete to delete a selection, or click-type? if the latter, you break old users). I decided to make some choices that made sense and leave the way that people want to use it up to them with a bunch of configurables.

And then there’s the other extensions that were expected to come – specifically being able to use
an invisible caret with the the selection mechanism to highlight non-writeable icons, to be able to select them. Most of the heavy lifting had been done for that, and whilst there’s a lot of checks to prevent the caret being moved by keypresses into a non-writeable icon (oh so many bugs with those things), those could be updated to allow such things in the future.

Oh, and I just remembered that the system font isn’t supported. Notwithstanding my ‘industrial control machinery users should be able to do what they want’ comment, I didn’t care about the system font. IF the Wimp panics, or you configure no font, you don’t get icon selections in that implementation. Sorry, I just didn’t care enough at that time, but I still wanted to come back and address it.

I suspect I’ve just written some very similar content to that which was in one of the Rambles. It wasn’t meant to be that long, but meh…

¹ Let’s just set aside the whole unsafe data and Message_DataSaved (which also required state to be remembered about transfers) because next to nobody implemented that at all.

² Incidentally, I implemented and registered *-commands for the ClipboardHolder module a couple of months ago, so that JFPatch-as-a-service can use the Clipboard as its communication of the completed build. I was surprised that there were no *-commands defined for that.

³ Operating as a producer and consumer of Wimp messages within the Wimp itself is fraught in my experience. I’m not sure if I’d done the work for scrolling over icons before the icon highlighting work, but I suspect that if I had, it’d put me offer further interactions there, or if I hadn’t done that work it was just not reasonable.

⁴ The scrolling icons interface was a similar example of trying to keep the Wimp to one thing and farm out the heavy lifting – basic message passing is handled in the Wimp for scrolling icons, but WindowScroll does the actual work of deciding what should scroll and how (and yes it does send messages to the Wimp for Menus and the menu windows have to respond as if they’re tasks to the scroll requests, which is part of the pain from ³).

⁵ I’m pretty sure it’s just a stub, like the ‘drag window’ is just a stub to start moving a window
around.

⁶ “Bat’s dos! I can’t do it!” to quote Zaphod. Maybe. Might not be quite the right line.

⁷ Dear god, don’t do this with RISC OS.

⁸ Validation code ‘C’ I think.

Thanks for the comprehensive summary! Regarding the take-up of the DragAndDrop protocol, there is at least Easi/TechWriter, which implements both the clipboard protocol and the DragAndDrop protocol. IIRC even its internal drags go through the DragAndDrop protocol (which is not that surprising, because when you start dragging, you do not yet know whether the drag will become an internal or an external drag).