Alpha sprites and Tinct

It would be great to have alpha-channel sprites on RO5, not least for the possibilities it would offer for making the Desktop look nicer (simple drop-shadows, properly anti-aliased icons, transparency for dragged objects, etc.).

The best solution would be for someone to write an equivalent to Select’s system. But NetSurf already uses the Tinct module. Would there be any way to use this to give the desktop transparency? I’m assuming there’ll at least be licencing reasons why this isn’t easy, not to mention the technological hurdles. But I’d be interested to hear if this was feasible/desirable.

Should transparency become possible on the ROOL desktop, I’d be keen to extend my desktop themes to cope with them.

Yes, it’s certainly desirable. The main problem here is maintaining some kind of coherency with Select’s handling, and tie in with IFR, etc. I think Richard Wilson (Tinct author) would be best placed to comment on this, but I don’t know if he reads these forums. How exactly it would have to tie into RO5 would determine the precise licensing issues.

We are given to believe that IFR is highly integrated into Select, although that’s at odds with parallel claims of modularisation; so whether a Tinct-like solution could be its own module which is completely separate from other other stuff, I don’t know.

We’ve done a LOT of thinking within ROOL about transparency. There are all kinds of related issues that you get into when you think about it – transparency in sprites is just the start of it. I’m not sure how Tinct would perform compared to a more ‘native’ implementation using SpriteExtend and the like? I don’t really know anything about how Tinct works.

One thing which it is safe to say is the idea of (semi-)transparent windows is quite a long way off right now. We reckon the best solution, if and when we were to support it, would be to replace the Wimp altogether. The new Wimp could support pre-emptive multithreading, better redraw algorithms, transparency, double-buffering, hardware-accelerated transitions and effects, etc. It would have to provide a legacy API for existing (not updated) applications.

It’s a huge lump of work though, so it’s hard to visualise how it could be done as an unpaid spare-time activity by someone like Ben (who’s done this type of thing before). I think he estimates it’s about a man year of work!

Maybe ROOL should start collecting donations into a specific fund for this type of development?

Interesting stuff. I’d be interested to hear from Richard about licensing issues, etc., if he’s inclined to comment.

The comments on a thorough-going overall are fascinating. Personally, I think there’s a lot that could be done with Select’s level of transparency support (IMHO, transparent windows don’t work all that well). But obviously a proper replacement for the Wimp would be preferable to another patch-up job, if the time (and money) can be found.

This is taking the thread off-topic, but now that the ROM images are becoming complete, is it time to open consultation on a road-map for ROOL’s RISC OS? I think some prioritisation of tasks needs to take place. People are, I think, much more likely to contribute to a specific project if they can see it working as part of a coherent whole. For too long, improvements to the OS have been piecemeal and at the whim of whoever happens to have the time and inspiration. Maybe now is the time to open discussion on how to take things forward.

If people think that re-branding and overhauling the look of the OS is one of those priorities, I’d be happy to offer my services (in conjunction with any others who’ve expressed an interest, such as Michael Drake, etc.). But there’ll obviously be other, more fundamental tasks, many of which would take priority over that.

Firstly, I wish ROL hadn’t called them “alpha channel sprites”. What they support are sprites with ‘greyscale’ masks. This is quite a separate thing from the alpha channel in the framebuffer on hardware that supports multiple video/graphics planes like all the RISC OS STBs, or the Beagleboard. And sprites are fundamentally the same thing as a framebuffer.

We’ve done a LOT of thinking within ROOL [...] Ben (who’s done this type of thing before)

Aargh, Steve’s blown my cover. Yes, I spent my first few years at Acorn mostly working on the Wimp, and yes in the years since I’ve effectively rewritten the window-compositing side of it for a contract job – scroll offsets, nested windows, non-rectangular and semi-transparent windows, delayed updating of windows owned by sleeping tasks, mutiple plane support, the whole McCoy. Sadly that wasn’t an open-source job, so I can’t reuse any of it. And yes, primitive sprite plotting operations that support various types of blending are definitely a prerequisite for transparent windows, though it’s just the tip of the iceberg when it comes to the redraw management code in the window manager.

So yes, I’ve had a whole lot of ideas regarding the Wimp for many years – and it encompasses a lot of what people perceive to be wrong with RISC OS.

• Multitasking model – yes that old chestnut. But I believe before all that long multiple cores are going to become significant and we’ll finally be forced to confront this. We need some way at the very least to schedule tasks on more than one core simultaneously. I know this is a subject of almost religious feeling, but in theory I don’t see why we shouldn’t be able to maintain support for existing applications while at the very least permitting non-GUI tasks a fair stab at the CPU, allowing big chunks of the system like the Internet stack to move to USR mode from SVC mode. I’d argue that task switching needs to be be decoupled from the act of fetching the next GUI event to achieve this – bodges like TaskWindow and Wimp2 just aren’t good enough. The ability to have event-driven application code has been largely lost since Arthur (RISC OS 1 for the uninitiated), because the OS’s underlying language-neutral OS-to-application signalling system, the UpCall and Event environment handlers, were never properly dispatched by the RISC OS 2 Wimp, and this has never been fixed. As an ultimate extension of this, any Wimp event could optionally be delivered in an evert-driven way via an environment handler in this way. And threading – why not (except for languages like BASIC where the stack structure fundamentally prohibits it).

• Eye candy. So yes, non-rectangular windows, translucent windows. Much greater themability – going beyond even ROL’s rounded buttons and repositionable window controls. Animations. Yes, even 3D support.

• Usability. Rethink keyboard navigation from scratch. Clipboard and drag-and-drop support in all icons. Full IME integration. Think through alternative ways of controlling the user interface (since RISC OS will run on a whole range of gizmos) – so things like touchscreens, multi-touch trackpads, games controllers (think Pandora), remote controls, etc.

• Double buffering. Not just for smoother updates; this would permit the “soft copy” to emulate pixel depths, RGB/BGR channel ordering and so on that aren’t supported natively on the hardware, and on systems with slow access to video RAM (Iyonix) would help with cacheing and compositing, especially important when permitting non-rectangular or transparent windows, or a fancy 3D window compositing system.

• Proper support for multi-head outputs – think constraining windows and the pointer when you have multiple screens of different sizes or rotations. Having one “master” screen where the icon bar lives and error boxes, command windows etc open, rather than being split down the middle between monitors. Or even devices with more than one display where the two aren’t logically adjacent (like phones – but not just limited to that).

• Support for video planes – needed to get decent speed out of any video codec.

• A better alternative to Templates (which are also the basis of Toolbox resources). We could benefit from the Template replacement format working in terms of a higher-level widget concept, with a wider variety of modern widgets. Layouts should ideally be done relatively rather than using absolute coordinates (so the windows can reformat themselves when you change or resize the desktop font, and you don’t need different Templates for every new language you support). Maybe the whole thing should be vectorised and allow the entire desktop to be scaled?

And that lot’s just off the top of my head. Maybe some of it is just a pipedream, but the thing that strikes me is that the Wimp would need major surgery pretty much all over. There comes a point with these things that the amount of effort required is less just to start again from scratch (though referring to the original to avoid the worst pitfalls) than to gradually patch and bodge the original.

Yes, I would love to do the above. In fact, I’d love to be able to keep all those fun programming tasks all for myself. But experience at writing window managers suggests that I’m not going to be able to find the time unless we get a generous benefactor, unless I win the Lottery, or until I retire (and that last one is some decades off yet).

[My 100th post, hooray]

Well done Ben! A very informative post for your milestone 100.

I’d like to add a couple of minor bits to the stuff you’ve mentioned:

Template files would probably be replaced by a human-readable format. This solves a number of issues, not least storage in repositories such as cvs.

And because the changes to the Wimp would be so widespread, the sensible answer would be to throw it away and write a new one. Which would be in C rather than ARM assembly. Ben loves nothing more than writing assembler that no other human can comprehend(*), just to save a cycle but nowadays, we can manage with this core component being expressed in a more maintainable notation.

Thanks, though I was aiming for “thought-provoking” rather than just “informative” – trying to demonstrate the possible new ways of thinking about and using electronic devices in general and ones to which RISC OS might be ported in particular.

Ben loves nothing more than writing assembler that no other human can comprehend

        VLD1.64.W {D21,D22},[R6@128],R7
        VQRDMULH.S16 Q0,Q1,D9[1]

How could something that obvious possibly need public documentation?

(joke – but those are valid ARMv7 instructions)

Ben: that all sounds amazing, but as you point it, it’s a hell of a lot of work. Is there any way that this could be broken up into more manageable chunks? For example, it would be a valuable goal to have a replacement GUI layer: new format windows, vectorised contents, high level-templates, etc. Perhaps working alongside the existing system. For this to happen, would everything else on your list have to be done too (new multitasking model, double-buffering, etc.)? I’m not sure RISC OS can cope with anything more than incremental change, and there’s the issue of developer time to bear in mind too.

Well, the first chunk of work would be analysing what we want to do, how many parts it can be broken into, which parts can be developed concurrently and which can be developed on top of the existing modules, how many people could get involved and so on. API and general design work – saves time in the long run. I would like to think that the Wimp could be further modularised somewhat, at least separating the task/memory management side from the GUI side. I think it would also make sense for themable aspects of it (window furniture, icon rendering etc) to be plugins that can be replaced or added to.

One of the biggies introduced by multiprocessing is the need to remodel all access to data which might need to be shared between processors. This is particularly critical with the Wimp because the screen is probably the most complicated shared resource there is. I suspect that the only bit you can sensibly write on an existing base is a new template/widget/icon rendering system (because that’s only working within the context of a single application).

I’m also torn because there’s lots of other bits of RISC OS I could be working on. I’d like to be able to work on adding VFP support throughout because I think we’re well overdue for hardware floating point, and once that’s done (specifically the management of the shared register file) we can start using Advanced SIMD instructions to make a quick job of things like blended sprite plots. Which brings the thread in a nice circle.

Rather than creating a new template/widget/icon library, would there be any benefit in instead trying to use something like gtk+ with some RISC OS extensions.

To improve performance it may help to make modifications to the existing window manager.

The main problem I can see is that it won’t be usable from BASIC.

The thing is, NEON instructions can be done with some good macros or #defines if the compiler does not support them. But in all honesty, NEON is not really ARM is it?

To have NEON stuff it might even be nice to have a library of NEON code to use which has valid ARM counterparts. That way it will enable all variants of ARM processors to benefit. For example PXA270 has WMMX..

Rather than creating a new template/widget/icon library, would there be any benefit in instead trying to use something like gtk+ with some RISC OS extensions.

Well I’m going to show some of my own prejudices here, but I wouldn’t be interested in getting involved in such a thing due to the size of the code you’d have to drag into the build. Peter Naulls’s ChoX11 library is an impressive feat of engineering, but I wouldn’t want it underlying the entire GUI - it’s just too complicated and resource-hungry and too alien to RISC OS. Plus I spend far too much of my working life doing porting which I find (in general) deathly dull, and the opportunity to write something afresh, free from the need to support legacy x86 systems, tailored to RISC OS’s needs, would be a shame to squander – especially when there’s no commercial pressure to deliver by a set date.

And I think compatibility with BASIC is essential, given the high proportion of BASIC programmers we still have in the RISC OS scene.

The thing is, NEON instructions can be done with some good macros or #defines if the compiler does not support them. But in all honesty, NEON is not really ARM is it?

No, #defines in C cannot magically teach any compiler opcodes it does not already know about.

NEON is more ARM than any other FP/SIMD extension that’s ever existed, apart from the original FPA. Things like the Piccolo DSP and XScale’s 40-bit accumulator have been dropped from later revisions of the ARM architecture. By comparison, the VFP has been around in one form or another since the ARM10, and both VFP and Advanced SIMD were initiated by ARM, not by one of ARM’s licensees. Last I heard from ARM, the VFP and Advanced SIMD extensions had been implemented by all ARMv7 licensees, which they said meant that they are likely to become a mandatory part of the ARM architecture from ARMv8.

Also, to be pedantic, “NEON” is stricly speaking just one implementation of the VFP/Advanced SIMD instruction set (though you can see why people use the name for brevity!) For example, Qualcomm’s “VeNum” is the equivalent implementation in the Snapdragon family of chips.

To have NEON stuff it might even be nice to have a library of NEON code to use which has valid ARM counterparts. That way it will enable all variants of ARM processors to benefit. For example PXA270 has WMMX..

It’s possible that a VFP/Advanced SIMD emulator could be written – however it would be relatively slow, and for technical reasons it would be even harder on any architecture earlier than ARMv5 (ie. XScale and later versions of ARM 9).

I’ve just had a quick google on the iwMMXt – looks interesting, but it’s obviously not as capable as ARM’s Advanced SIMD - it only has 8 64-bit registers versus AdvSIMD’s 32, for a start. However, it also has a major drawback from RISC OS’s point of view that it re-uses coprocessor slots 0 and 1, which we still rely on for our FPA floating point instructions. That would require some serious unpicking from the OS and applications on a similar scale if not larger than the effort to convert 26-bit code to 32-bit code. It’s not even something you can patch up using processor vectors – you’d basically have to run all old code either under emulation or JIT recompile it.

I think I have not been clear about what I meant. When writing code for a purpose , say color space conversion , than that code can be written using NEON instructions. The same routine can be written using ‘old’ ARM code or using iWMMX for PXA chips. That way there can be 3 versions of that routine and depending on which CPU it may choose to run the specific optimized version…

I totally agree that JIT or emulation would be utterly useless.

For C code under GCC it is possible to assign variables to registers and thus have te ability to control which registers are used. That thus also allows code to have instructions compiled with the use of inline assembly. I could try this. GCC however has NEON instruction macros so they can be used too. (don’t know how they implemented them)

Apparently I am not so knowledgeable about Norcroft. So I have no idea if there are such possibilities.

For C code under GCC it is possible to assign variables to registers and thus have te ability to control which registers are used.

You can do this in Norcroft too, but that’s not the problem – what you can’t do is insert an arbitrary 32-bit value into the code area in the middle of a C function. Also, it wouldn’t help when the new instructions use a new register file unknown to the compiler (as with VFP/SIMD). And it wouldn’t make for efficient code since the compiler wouldn’t be able to peephole or schedule around the new instructions and you’re cramping its style by forcing a variable into a particular register. You’re really much better off using an assembler for that sort of task – that’s what an assembler is designed for.

Hi Ben, Sorry for the confusion. My idea was for the few instructions in SprExtend that stop things from compiling on older compilers due to a bug in the disassembler. It surely is by no means a structural solution.

If optimization done by handcoding is preferred then assembler files (.s) with routines for specific processors can be used. Note that these are not routines that implement ‘just’ one instruction but that do higher level methods implemented in optimized code. For example IDCT, FFT, IIR, YUV->RGB RGB->YUV etc.

The current method of using C to generate assembler and then putting that back in objasm seems like a very unorthodox method. The reason for that is unclear to me. Since I am used to gcc it seems that there is an interoperability problem of Norcroft C with Objasm? Or do the generated assembler files get parsed to change things in there?

So I hope you now understand that I am not advocating the use of inserting arbitrary 32 bit values in C code. sorry for the misunderstanding.

Basically, the build process for SpriteExtend is weird. I don’t fully understand it myself, but I think it’s the result of aggressive optimisation, trying to use the best features of both C and assembler – remember RISC OS was originally written for much lower power CPUs than we have today, and Acorn had to use every trick they could think of to get the best performance out of them.

I suspect that if it were being written today, SpriteExtend would be written in C with a lot of inline assembler. However, at the time it was written, C compilers didn’t generally provide an inline assembler feature.