Proposed GraphicsV enhancements

With all this amazing work that jeffrey has on his local PC, I hope he has a solid backup system in place.

Can you be more specific

What I meant to register was surprise, and delight, that we may be 2 release candidates away from a multi-core OS :-)

For those who are wondering, this is what the API currently looks like.

GraphicsV

Overlay hardware falls into two basic categories: “Z-Order” and “Basic”. Z-Order hardware allows full control of the Z order of the created overlays. Basic hardware offers no control of the Z order, and it’s expected that they’re placed somewhere between the desktop and the mouse pointer.

Currently the API restricts Z-Order overlays to be placed underneath the desktop. Alpha-blending or transparency colour key (TCK) must be used to mark out the desktop areas through which the overlay(s) should be visible.

Because Basic overlays are ontop of the desktop, care must be taken to hide them when necessary, e.g. if an error message appears which the user must be able to see.

If an overlay supports alpha formats, it’s expected that it will be alpha-blended by the hardware. TCK for overlays currently isn’t supported (only the desktop should use TCK).

To account for driver restrictions and to allow for more optimal memory management, all overlays for a driver will be destroyed when the driver changes mode.

The new calls described below should only be issued from the foreground.

For the different GraphicsV calls, bits 16-23 of R4 are expected to be used as follows:

VSync                 Head
SetMode               Head
SetInterlace          Head
SetBlank              Head
UpdatePointer         Head
SetDMAAddress         Head
VetMode               Head
DisplayFeatures       Head
FramestoreAddress     Head
WritePaletteEntry     Head/Overlay
WritePaletteEntries   Head/Overlay
ReadPaletteEntry      Head/Overlay
Render                Head
IICOp                 Head
SelectHead            -
StartupMode           Head
PixelFormats          Head
ReadInfo              -    
VetMode2              Head
CreateOverlay         -
DestroyOverlay        Overlay
SetOverlayPosition    Overlay
MapOverlayBuffer      Overlay
UnmapOverlayBuffer    Overlay
DiscardOverlayBuffer  Overlay
VetOverlay            -
SetOverlayScale       Overlay
SetOverlayZOrder      -

Because some calls accept both a driver number and a head number in bits 16-23, drivers should choose overlay numbers such that they don’t overlap the range of head numbers.

GraphicsV_WritePaletteEntry     * 10
GraphicsV_WritePaletteEntries   * 11
GraphicsV_ReadPaletteEntry      * 12

These calls have been extended to recognise a new palette type, type 3, which is used for the transparency colour key. This palette has the following entries:

• Entry 0: TCK value. This is a raw pixel value, rather than a &BBGGRRSS palette entry.
• Entry 1: TCK mask. This indicates which bits of a pixel are significant when the hardware performs the TCK check (e.g. in a 15bpp mode, the top bit of the halfword may be ignored).

Drivers can implement these as read-only values if the values can’t be changed. Drivers should ignore the call if TCK isn’t in use (e.g. alpha is being used instead).

GraphicsV_ReadInfo              * 18

This call is extended to add a new item, item 5, which is a 32bit int indicating the maximum number of overlays that the driver supports. If this value is zero or the call isn’t claimed then it indicates that the driver doesn’t support the overlay API.

GraphicsV_CreateOverlay         * 20

In: R0 = Pointer to descriptor buffer:
         Word 0: Flags:
                 bits 0-7: Max number of banks/buffers wanted, minus one
                 bits 8-9: Rotation, relative to the main (desktop) overlay
                           0 = none
                           1 = 90 degrees clockwise
                           2 = 180 degrees
                           3 = 90 degrees anti-clockwise
                 bit 10: Overlay must support scaling
                 bits 11-31: Reserved (0)
         Word 1: NColour of pixel format
         Word 2: ModeFlags of pixel format
         Word 3: Log2BPP of pixel format
         Word 4: Buffer width (pixels)
         Word 5: Buffer height (pixels)
         Word 6: Mask of which heads the overlay is to be used with
    R1 = Aspect ratio to use when calculating returned min/max scale values
         Top 16 bits = width
         Low 16 bits = height
    R4 = reason code + driver number
    
Out: R0 = Overlay number
     R1 = Flags:
          bits 0-7: Overlay type
                    0 = Z-Order
                    1 = Basic
                    2+ = Reserved
          bits 8-31: Reserved
     R2 = Minimum width overlay can be scaled to
     R3 = Minimum height overlay can be scaled to
     R5 = Maximum width overlay can be scaled to
     R6 = Maximum height overlay can be scaled to

Creates a new hardware overlay.

For rotated overlays, the aspect ratio and returned width & height correspond to the rotated form of the overlay, i.e. as it will appear on screen.

GraphicsV_DestroyOverlay        * 21

In: R4 = reason code + driver number + overlay number

Destroys a hardware overlay.

Specifying an overlay number of zero will destroy all overlays.

GraphicsV_SetOverlayPosition    * 22

In: R0 = Pointer to coordinate array:
         Word 0: Head number
         Word 1: X position
         Word 2: Y position
    R1 = Number of entries in array
    R2 = Bank to display (ignored if R1=0)
    R4 = reason code + driver number + overlay number

Displays or hides an overlay. The coordinate space is that of logical screen coordinates, i.e. (0,0) is the top-left corner of the screen from the viewer’s perspective (if the screen is rotated, (0,0) is still the top-left). Within that coordinate space, the coordinates specify the top-left corner of the overlay, as it appears on screen (e.g. if the overlay is rotated 180 degrees then it would actually correspond to the bottom-right pixel of the buffer).

The overlay will only appear on the heads which are included in the list. Therefore you can hide the overlay completely by specifying R1=0.

GraphicsV_MapOverlayBuffer      * 23

In: R0 = Bank number
    R4 = reason code + driver number + overlay number
    
Out: R0 = Base logical address
     R1 = Stride between image rows

Maps an overlay bank/buffer into logical address space, potentially allocating the memory for it if the driver hasn’t already done so.

If this is a freshly-allocated buffer, the contents are undefined. Otherwise the contents will be preserved.

Drivers may have limited logical address space available for mapping overlays, so in Wimp situations it’s recommended to unmap your overlay(s) before allowing the Wimp to swap out your task.

GraphicsV_UnmapOverlayBuffer    * 24

In: R0 = Bank number
    R4 = reason code + driver number + overlay number

Unmaps an overlay buffer from the logical address space.

GraphicsV_DiscardOverlayBuffer  * 25

In: R0 = Bank number
    R4 = reason code + driver number + overlay number

Discards a buffer, freeing the memory that was allocated to it. If necessary, the buffer will be unmapped first, and if the overlay was currently displaying it then the overlay will be hidden.

GraphicsV_VetOverlay            * 26

In: R0 = Pointer to descriptor buffer:
         Word 0: Flags:
                 bits 0-7: Max number of banks/buffers wanted, minus one
                 bits 8-9: Rotation, relative to the main (desktop) overlay
                           0 = none
                           1 = 90 degrees clockwise
                           2 = 180 degrees
                           3 = 90 degrees anti-clockwise
                 bit 10: Overlay must support scaling
                 bits 11-31: Reserved (0)
         Word 1: NColour of pixel format
         Word 2: ModeFlags of pixel format
         Word 3: Log2BPP of pixel format
         Word 4: Buffer width (pixels)
         Word 5: Buffer height (pixels)
         Word 6: Mask of which heads the overlay is to be used with
    R1 = Aspect ratio to use when calculating returned min/max scale values
         Top 16 bits = width
         Low 16 bits = height
    R4 = reason code + driver number
    
Out: R1 = Flags:
          bits 0-7: Overlay type
                    0 = Z-Order
                    1 = Basic
                    2+ = Reserved
          bits 8-31: Reserved
     R2 = Minimum width overlay can be scaled to
     R3 = Minimum height overlay can be scaled to
     R5 = Maximum width overlay can be scaled to
     R6 = Maximum height overlay can be scaled to

Allows the user to check whether an overlay with the specified properties can be created, and if so, what capabilities it will have. If the overlay can’t be supported, the driver should not claim the call.

When vetting the settings, the driver should take into account any restrictions that result from the current system configuration. E.g. if all overlays are currently in use, no Vet call should succeed. Or conversely, if a Vet call succeeds, so too should a corresponding Create call.

GraphicsV_SetOverlayScale       * 27

In: R0 = Width (pixels)
    R1 = Height (pixels)
    R2 = Aspect ratio to use if image size needs adjusting
         Top 16 bits = width
         Low 16 bits = height
    R4 = reason code + driver number = overlay number
    
Out: R0 = Actual width (pixels)
     R1 = Actual height (pixels)

Adjusts the scale of scalable overlays. For rotated overlays, the values correspond to the rotated form, i.e. as it will appear on screen. Rounding errors or other hardware limits may prevent the chosen size from being used, in which case the driver will adjust the size (while maintaining the indicated aspect ratio) and return the actual dimensions used.

GraphicsV_SetOverlayZOrder      * 28

In: R0 = Pointer to byte array containing overlay IDs
    R1 = Length of array
    R4 = reason code + driver number

Specifies the sorting order for any Z-Order overlays. The first entry in the list corresponds to the frontmost overlay. The effect on the order of any overlays not included in the list is unspecified.

VideoOverlay

The GraphicsV overlay API is a low-level interface that should not be used directly by programs. Instead, it’s expected that an overlay manager module (provisional name “VideoOverlay”) will be created which will present a higher-level SWI-based interface. VideoOverlay’s duties are expected to include:

• Rendering (or coordinating the rendering) of alpha/TCK sections to allow Z-order overlays to be visible
• Associating overlays with Wimp windows/tasks
  • Allows Z-order to be automatically updated
  • Ensures overlay is destroyed if task exits or window is destroyed
• Notifying programs of when their overlays have been lost
• Doesn’t use GraphicsV overlay IDs directly; instead a different ID system will be used, to prevent a program which has lots its overlay due to a mode change from accidentally interacting with an overlay that belongs to a different program
• Dealing with some of the complexities of Basic overlays
• Software emulation of overlays
• Allowing screen output to be redirected to an overlay

Since generic hardware overlays aren’t a critical system component, it’s expected that the VideoOverlay module will be disc-based rather than ROM-based.

Because Basic overlays are ontop of the desktop, care must be taken to hide them when necessary, e.g. if an error message appears which the user must be able to see.

A lesson Migrainesoft has yet to learn. Or, possibly, relearn.

Progress update – I’ve got OMAP4 mostly working, and a prototype VideoOverlay module. Although my initial plan was to do Pi first and then VideoOverlay, I’ve decided to swap them round so that if I need to make changes to the GraphicsV API to accommodate VideoOverlay I’ll have less driver code to rewrite.

The current version of VideoOverlay works by attaching the overlay to a window that the program specifies, but I have a feeling this is going to make things a bit messy when it comes to handling TCK/alpha redraw. So instead I’m going to try reworking it to use nested windows. It’ll still work on the principle of an overlay being attached to a window, but by using the nested wimp it’ll be possible for applications to have VideoOverlay-owned windows as part of their stack. So VideoOverlay will be able to handle redrawing TCK/alpha without any application involvement, and if an app wants to display something ontop of the overlay (e.g. subtitles or an OSD) it could do so by adding a window ontop of it with a transparent background colour.

Since people are interested in continued graphics improvements, I figured it would be worth trying to summarise the main things are on my wishlist, and any dependencies between them. Both for my own benefit, and on the offchance that it’ll entice someone else to try tackling some of them.

Main reference: https://www.riscosopen.org/wiki/documentation/show/Proposed%20GraphicsV%20enhancements

• Display selection – changes to the display manager, Configure, etc. to allow the user to select between different drivers/display outputs and configure which output should be active on boot (plus probably some associated driver work so that hardware which has multiple heads allows you to switch between those heads).
• Improved mouse pointer – larger, more colours. The GraphicsV APIs for this are defined (but not implemented). Higher-level APIs are yet to be fully defined. There’ll be some kernel code to unpick, since currently the kernel manages buffering of pointer image updates, and one of the aims of this change is to get rid of that (drivers will manage buffering the images themselves)
• Improved screen memory management – this is a big one, but is a key requirement for many features like generic multi-monitor support, software display scaling/BPP conversion/rotation, etc. There’s a reasonable chance that this task will touch more than just the graphics subsystem (e.g. changes to PMP or DA management may be required, better abort handling is desirable for trapping of screen memory writes by software buffer conversion layers, etc.)
• Production-ready DisplayLink driver – Mostly held back by the screen memory handling improvements.
• Basic multi-monitor support – Designing & implementing the changes necessary to allow systems which are natively multi-head, to be able to use those multiple heads. Essentially multi-monitor support within the context of a single GraphicsV driver. There may be some cross-over here with the virtual mode system
• Generic multi-monitor support – Building ontop of the basic support, the improved screen memory management, and (ideally) the software display scaling/rotation/BPP conversion to allow multi-monitor across different drivers.
• Display detection/hotplug support – Preferably something a bit smarter than “just poll the EDID every few seconds”, especially since not everything supports EDID
• Hardware display scaling & rotation – There are two main aspects of this, one is defining the extensions to the GraphicsV API (particularly VIDCList extensions) to allow the behaviour to be controlled, and the other is going to be all the UI/frontend changes to allow the user & apps to be aware of the features.
• Software display scaling & rotation (& BPP conversion) – Combining the above with the improved memory management to allow the OS to direct apps to a virtual/soft framebuffer which the kernel/VDU driver then converts to the real buffer on-the-fly
• Tidying up the tangled mess of mode selection code (including making sure the kernel actually vets the modes)
• Misc bug fixes and API extensions – software mouse pointer improvements (less flickering & redraw corruption), new APIs for large modes/sprites, OS_ScreenMode reasons 7-10, and everything from the “Other” section of the wiki page

There’s also the PCI heap conundrum which probably needs solving for some of the above memory management tasks to be made safer/easier.

Improved mouse pointer

If devising APIs, please consider (for the future) the possibility of pressure at the pointer location, and also multiple active pointers at the same time.

This is for a potential extension to support “multitouch” devices, and comes from the fact that there’s no clean way to support a tablet in RISC OS because the API cannot be extended to support pen pressure (so that’s added using a different protocol…which is a messy way to do it IMHO).

Rick applied

Pressure

There was a module I allocated and created some time ago called Channels which span out of the MMK Keyboard development. That keyboard had a rotary encoder as a volume control, and the support software allowed it to be used for seven other things too, when used with modifiers. I added support to a graphics program to allow a selection to be rotated as it was dragged, for example.

I considered pressure as a possible application and noted that there were two graphics tablets available at the time, both of which had their own APIs. One sent X&Y through MouseV, but provided a second API with X, Y & Pressure. I think PhotoDesk supported that API. The other also had Tilt, so needed a new API, but also implemented the XYP API for PhotoDesk and the MouseV interface for the pointer.

I didn’t see that as a scalable solution, so Channels was born.

It provides a number of virtual and physical channels – PS2 Mouse X & Y are two physical channels, pointer X & Y are two virtual channels, as are system volume and “pointer pressure”. The user could then connect physical and virtual channels together with a star command or GUI utility.

This allowed one to control “X, Y & Pressure” even if you only had a mouse and an MMK keyboard… but one could configure it as one pleased – plugging in a second mouse and using that to change pressure, for example.

There were a number of SWIs, including the ability to read multiple channels simultaneously. It was also possible to ‘tee’ and rescale channels, for advanced users.

One could therefore have multiple virtual channels for additional inputs (pointers) or touches, together with pressure, size, tilt, or whatever the consumer requires. Your input devices provide physical channels depending on their capability. The user connects them in the way that best suits them. Naturally simulated controls can be provided by small Wimp programs featuring knobs or sliders, or input fields.

One sent X&Y through MouseV, but provided a second API with X, Y & Pressure. I think PhotoDesk supported that API.

That’s the one my little tablet driver supports, so it can be used with PhotoDesk. Might work with ArtWorks too, but I don’t have that.

But, yes, clearly inventing API after API and bolting them all together is a bit of a non-starter…

I didn’t see that as a scalable solution, so Channels was born.

So, if people wanted to use that what would they need to be doing?

Steve asked

if people wanted to use that…

I’ve dug it out of the archives. It is 32bit, despite last having been touched in 2003. Channels itself (ChannelManager) is complete. There is also a ChannelMouse module which implements a new mouse type, controlled through Channels.

There’s no GUI though. There’s some sprites and a template, but I didn’t write a UI – the star command works. I’ll see if I can package it into a beta release that does something useful over the weekend.

It won’t be genuinely useful until there are ‘drivers’ for things like USB mice and more exotic HID devices. I have no hardware any more, this emulator has no USB support so I can’t even build a prototype for that.

‘Drivers’ are a trivial callback attached to a channel number:

Channel_Claim
=> R0  Channel number or 0 for dynamic allocation
   R1  Driver code or 0
   R2  Driver workspace or 0
   R3  Flags
   R4  Name (Single word, ctrl or space terminated)
   R5  Minimum (if flags b2) (ORR mask if flags b3)
   R6  Maximum (if flags b2) (AND mask if flags b3)
<= R0  Channel number

Flags:
   b0  Live read (call driver to read value)
   b1  Live write (call driver to write value)
   b2  Bounded (by R5,R6)
   b3  Bitfield (discontinuous)
   b4  Modifies (modified value passed on to next connected channel)
   b7  Persistent (channel is not deleted on Release)

Driver:
=> R0  Reason: 0=Read, 1=Write, 2=Shutdown, 3=User
   R1  Value to write (if R0=1)
   R12 Workspace
<= R1  Value (for R0=0,1)</code>
  

Does the client-facing API allow data to be correlated by timestamp? Knowing the X, Y, and pressure/buttons of a mouse input is useless if you can’t determine whether the three values were sampled at the same time (or near enough).

There are calls to read multiple values atomically, but this doesn’t change the way that the mouse works (since you use that example):

As you may recall, the kernel polls the mouse driver on VSync interrupts using PointerV,4 (RO4) or PointerV,0, then issues an expansion Event to document any change in scroll wheels (RO4); calls KeyV (multiple times) to reflect changes in mouse keys (RO4); and finally scales, clips and updates the mouse position in zero page. (For PointerV,0 claimants it is their responsibility to call KeyV)

As ChannelMouse is just a different type of mouse driver, equivalent to the PS2 or serial mouse driver, it works in EXACTLY the same way. Timestamping is done by the kernel at a higher level.

Does the client-facing API allow data to be correlated by timestamp? Knowing the X, Y, and pressure/buttons of a mouse input is useless if you can’t determine whether the three values were sampled at the same time (or near enough).

Maybe I’m looking at it from a different direction, but: pretty much the only relevant pointer devices that you can buy today are USB HIDs, so all the values are returned simultaneously by the device. (The sample interval is also very short.) So they’re all correlated at source as far as we’re concerned. What matters is that all the processed information should be available to clients via a single call. I think it’s a reasonable assumption that all the parameters are updated on every input transfer on the USB, so we don’t destroy the correlation.

Or have I misunderstood?

That’s all correct. Which is why I was a bit worried when I saw that each channel driver can only report a single 32bit value, with no timetamp.

In the multi-threaded world of our dreams, or even in the current single-threaded world (where you could get unlucky with a lengthy interrupt or callback), reading multiple channels one at a time could result in you getting samples from significantly different points in time.

The concept of indirecting a three or five dimensional mouse event through separate channels is no different to the long established model of updating the mouse buttons through three separate KeyV calls. All mouse connectivity ‘channelling’ occurs during the VSync poll to read the ChannelMouse status.

It is possible to subvert that of course, by manually changing one channel at a time, but that’s not how the mouse drivers work.

If the user is controlling the X axis of their ‘mouse’ with a keyboard knob and the Y axis with a USB-MIDI slider, then all bets are off… but that’s the whole point. More likely is the desire to control MouseX, MouseY and MouseButtons from the PS2 mouse (say) while connecting SerialMouse X&Y to MousePressure and MouseTilt for example. The lack of correlation between PS2MouseX and SerialMouseY is not important.

MouseX, MouseY and MouseButtons are (typically) read and written as an atomic tuple with interrupts disabled. Up to four at a time can be accessed atomically. There is an arbitrary length call like ReadVduVars but that isn’t atomic (at the moment). Maybe it should be.

Steve waited

If people wanted to use…

Despite my best intentions I got distracted by making something unrelated…

I’m tidying up Channels for release now.

Despite my best intentions I got distracted by making something unrelated…

Let me guess …

… it’s an animated GIF!

Ha. It’s for RO running in an emulator under Windows – it controls Media Player running in Windows from within the emulator.

This is the bit I’m most pleased with though – the new ‘middle’ iconbar position:

This is the bit I’m most pleased with though – the new ‘middle’ iconbar position:

I’m pretty sure the Style Guide has something to say about the width of icons on the iconbar.
Ever wondered whether the Guide needed any kind of rewrite?

It’s not an icon, it’s a UI.

Ever wondered whether the Guide needed any kind of rewrite?

The Style Guide has always needed a rewrite.

It’s not an icon, it’s a UI.

Sitting on the iconbar innit?

The Style Guide has always needed a rewrite.

:)

It’s not an icon, it’s a UI.

Sitting on the iconbar innit?

<Rolls sleeves up> Right then. It’s five icons, with no gap between them, taking the same amount of room as six !Paints. Happy now? ;-)

with no gap between them

Style Guide :P

Of course somepeople just pop that kind of thing just above the iconbar (like various digital clocks). Not that I particularly like those float above implementations and due to influence from the dark side the concept of those controls on the bar at the bottom is familiar and not totally unpleasurable.

Happy now?

Only if it works in RPCEmu. ;-)