Proposed GraphicsV enhancements

Goals

• Improve support for multiple drivers
• Provide an API suitable for controlling all the major features of the OMAP & VideoCore display subsystems
  • So that the video drivers in the OMAP & Raspberry Pi ROMs can expose all of the hardware’s functionality to RISC OS and user programs
• Provide support for displays being added and removed at runtime
  • E.g. for DisplayLink-based USB monitors/video adaptors, or for hotplugging of displays to static adaptors
• Allow programs to claim sole use of displays, hardware overlays, etc.
  • You don’t want to have two media players trying to use the same YUV overlay at the same time!
• Add API-level support for the new display formats proposed in the extended framebuffer specification
  • Note that this does not necessarily mean that the rest of RISC OS will be expected to understand the formats!
• Attempt to re-use any of ROL’s new APIs wherever possible

Existing documentation

• Extended Framebuffer Format Specification
• GraphicsV
• OS_ScreenMode
• RISC OS Select display changes
• RISC OS Select OS_ScreenMode changes
• RISC OS Select VDU Variables

Detail

OS_ScreenMode changes

Details of new OS_ScreenMode calls are given in each individual section of the document.

New reason codes are numbered from 64+, to avoid clashing with ROL’s allocations which are numbered from 255 down.

Support for multiple displays

(n.b. having multiple displays active at once, e.g. multi-display desktop, is not covered in this section)

Current status

• As of RISC OS 5.21, OS_ScreenMode reasons 64-67 can be used to register and deregister GraphicsV drivers
• The OS VDU driver only knows how to deal with one driver at a time; the VDU variable 192 is used to store the ID of the active driver. This is the same VDU variable as used by ROL to store their active driver number.
• All GraphicsV reason codes accept a driver number in the top byte of R4
• OS_ScreenMode 11 is used to select the active driver, or to read the current driver number (as per ROL). Switching driver will trigger a mode change, with the OS making a basic attempt at selecting a sensible startup mode for the new driver
• OS_ScreenMode 68 can be used to enumerate the available drivers
• On startup the OS will use the first driver that successfully registers with the system
• Service_DisplayChanged and Service_DisplayStatus are issued by the OS at the appropriate times.
• As of RISC OS 5.23, Teletext mode handling has been improved
  • ROL’s T, TX and TY mode string attributes are supported
  • Teletext font scaling is not implemented; if the chosen text window doesn’t fill the screen at the native teletext font size then the text window will be centred on the screen.
  • For kernel builds with high-res teletext enabled, any screen mode of 4bpp or greater can be used. Low-res teletext currently only supports 4bpp modes.
  • If the “native” mode 7 isn’t available, the OS will try a series of fallback resolutions until a suitable mode is found. For high-res teletext multiple colour depths will also be tried.

Remaining work

• Improve handling of Service_DisplayChanged by ScreenModes – e.g. loading a driver-specific MDF or reading EDID.
• Update the display manager and screen setup plugin to allow selection of driver
  • Improve Service_EnumerateScreenModes and related service calls to allow all displays to be queried for available modes instead of just the current display
    • Rather than overload existing parameters, it may make sense to introduce a replacement service call (or set of service calls) to allow the driver and head to be specified. A filtered mode list for just that driver/head combo can then be returned.
• Decide how to handle saving of settings in CMOS – simply storing driver number may be insufficient in some systems (e.g. USB displays), and a VideoGuard-like system may be required to ensure the user isn’t left without a display
• Allowing the display to be selected via the command line (e.g. as part of a mode string for *WimpMode) is also desirable
• Changes are required in other areas (particularly memory management) in order to allow multiple drivers to be active at once; see other areas of this document for details.

Pointer overlay

Overview

The pointer overlay is set by one of a number of routes:

• Via Wimp_SetPointerShape (just a wrapper for OS_Word 21)
• Via Wimp_SpriteOp 36
• Via OS_SpriteOp 36 (internally the Wimp calls this with R6=0 to have it autoscale)
• Via OS_Word 21

These APIs all enforce a maximum pointer size of 32×32 pixels and a colour depth of 2bpp. However a 32×32 pointer is likely to be insufficient for 180 DPI modes (the standard 11×22 pointer image, when scaled up, would be 22×44). Furthermore, most modern display hardware allows for true-colour, alpha-blended pointer overlays – but the 2bpp limitation of the current APIs prevents any such features from being used under RISC OS.

Internally the kernel buffers the 32×32×2bpp pointer images and allows drivers to reference the buffered data directly. This is favourable for the VIDC driver as the buffering code was designed specifically for that hardware. However for all other current drivers this is a useless feature, as the data must be converted to different hardware-specific forms before use.

Proposed changes

• Remove pointer image buffering code from the kernel and make it the driver’s responsibility
• Remove 32×32 size limit at the API level. Instead, each driver will be able to communicate its own limits
• Add support for true-colour transparency masked and alpha-blended pointer images
• Improve support for software pointers – add kernel/OS support for rendering the pointer instead of requiring the driver to do it
• Split GraphicsV 5 into two or more separate calls, one for defining the pointer image and one for moving the image around on screen. This will allow any CPU-intensive tasks related to preparing new pointer images to be kept out of the VSync handler.

Implementation details

GraphicsV 8

A new feature flag will be required to allow drivers to indicate that they use the new pointer API.

GraphicsV PointerFeatures

Entry
R0	Shape number (0-3)
R1	Pointer to request block
R4	Reason code, driver number, etc.

Exit
-	Request block updated as appropriate

This call will allow the caller to query a driver’s level of support for a given pointer image. It operates on a request block with the following format:

Word
0	Desired image type
	0 = 2bpp
	1 = true-colour transparency mask
	2 = true-colour alpha blended
1	Desired width
2	Desired height
3	NColour value
4	ModeFlags value
5	Log2BPP value

On entry, words 0-2 should be filled in. On exit, the driver will have updated words 0-2 to match its abilities, and filled in words 3-5 to describe the required pixel format of the shape data. For example, the VIDC20 driver will set the image to 0, set the image width to 32, and clamp the image height to within a sensible limit. Words 3-5 will describe a 2bpp mode.

GraphicsV BeginPointerUpdate

Entry
R0	Shape number (0-3)
R1	Pointer to request block (as above)
R4	Reason code, driver number, etc.

Exit
-	Request block updated as appropriate
R2	Pointer to image buffer
R3	Stride of buffer rows

This call is used to allocate a shape buffer. The stride value returned in R3 is the number of bytes from the start of one row to the start of the next. The caller is expected to fill in the buffer with data in the format requested, which is not guaranteed to match the format that was returned from an earlier call to GraphicsV PointerFeatures. Any pixels/bytes which lie outside the image (for if the stride in R3 is larger than width*bpp) must not be modified.

GraphicsV EndPointerUpdate

Entry
R0	Shape number (0-3)
R1	Pointer to image buffer
R2	X position of active point
R3	Y position of active point
R4	Reason code, driver number, etc.

Exit
-	All registers preserved

This call is used to return a buffer to the driver once it has been filled in. The active point of the image is also specified (as per OS_Word 21_0).

If the caller was updating the shape data for the image currently being displayed, it is the driver’s responsibility to ensure that the changes are buffered correctly such that there is an instantaneous transition from the old shape to the new; i.e. buffers which are currently being written to must not be displayed.

GraphicsV SetPointer

Entry
R0	Shape number (0-3)
R1	X coordinate
R2	Y coordinate
R4	Reason code, driver number, etc.

Exit
-	All registers preserved

This call is used to select the displayed shape number and/or move the image on the screen.

GraphicsV RemovePointer

Entry
R4	Reason code, driver number, etc.

Exit
-	All registers preserved

This call is used to hide the pointer.

Other changes

• Exact OS_SpriteOp and OS_Word changes are TBD.
• OS_Word 21 API is likely to remain as-is. However the implementation will need changing to add support for the new interface, including drivers which elect not to support 2bpp images and instead only support true-colour ones. Therefore conversion code will need adding to the kernel, or the kernel may elect to rely on OS_SpriteOp 36 for updating the image.
• OS_SpriteOp 36 is a prime candidate for automatically using true-colour/alpha-blending functionality when supplied a suitable sprite, if the hardware supports it. However additional flags to control behaviour may be desired. It’s expected that it will call GraphicsV internally in order to set up the new shape data (falling back to OS_Word 21 if support for the new GraphicsV pointer API is unavailable).
• VIDC20Video: Migrate shape image buffering code out of kernel and into the driver. Update to allow for taller images, up to at least 64 pixels.
• BCMVideo, OMAPVideo, NVidia: Add support for true-colour images, at least 64×64. Ensure changes are buffered appropriately.
• Kernel: Ideally we’d remove the code for dealing with the cursor image buffers. This would also free up some workspace. However the code may be required in order to support old drivers that don’t support the new API (e.g. if Aemulor or Geminus are in use and are hiding the true capabilities of a driver). The buffering code could potentially be split off into a support module, creating a compatibility layer to map old calls to new and vice-versa.

Potential future enhancements

Allow the scale of the pointer image to be specified.

Improved screen memory management

Overview

In order to fully support multiple active drivers, heads, and overlays, we need to overhaul how screen memory is managed.

At the moment the OS is in charge of memory management for the current driver:

• If the driver does not have its own framestore, the OS will use the screen dynamic area (DA 2) for screen memory. Internally this allocates from video-capable system memory that was identified by the HAL on startup. The dynamic area is doubly-mapped in order to support VIDC-style hardware scrolling. Given the current implementation, only one entity (driver, head, overlay) can sensibly use DA 2 at a time.
• If the driver has its own framestore, the driver will communicate its size and physical address to the kernel. The kernel will then map in this area using OS_Memory 13 and use it with the VDU drivers. Although this interface allows the driver to dictate where the buffer is located, the kernel is still in charge of things such as where each individual screen bank is located within that buffer.

Proposed changes

• Implement ROL’s OS_ScreenMode reason codes 7-10 (or something like them) to abstract screen banks away from memory layout and DA 2.
  • For OS_ScreenMode 10, an improved version of the CopyRectangle render op could be used, which allows rectangles to be copied from one screen bank to another. Consider making the call flexible enough to allow for further expansion (cross-head/cross-overlay copies, pixel format conversion, arbitrary source/destination buffers, rotation, scaling, etc.)
• Migrate screen memory management out of the kernel and into the drivers; it will be the driver’s responsibility to map in and manage its screen memory.
• GraphicsV APIs which previously used physical memory addresses will be updated to either use logical addresses, or perhaps offsets relative to the start of the driver’s buffer. This will allow the easy creation of drivers which do not require contiguous physical memory (e.g. software emulation of low colour modes, a VNC server implemented as a GraphicsV driver, USB displaylink devices, etc.)
• Generalise the code for managing DA 2 in order to allow other DA 2-like dynamic areas to be created. For example to support multiple concurrently active drivers which support VIDC-style screen scrolling.
• Within the kernel create a new dynamic area which accesses system VRAM – similar to DA 2, but singly mapped instead of doubly mapped. Drivers will be able to request blocks of memory from this dynamic area, for purposes such as display buffers (e.g. if the driver doesn’t support VIDC-style hardware scrolling and therefore doesn’t need doubly-mapped memory), or for hardware overlays.
• To avoid physical memory fragmentation, the kernel is at liberty to compact any of the VRAM dynamic areas (DA 2, DA 2-like DAs, or the singly-mapped DA) as it sees fit. A basic implementation of compacting the area after any block is freed may suffice. Drivers and clients will need to be notified of any buffers which are moved by compaction – whether they’ve moved in logical space, physical space, or both spaces.
• Add a new API (perhaps a new kind of dynamic area) which allows for physical pages to be claimed but no logical address space allocated to them. This will assist in the implementation of display rotation support for OMAP, where a private block of contiguous physical memory is required to store the frame buffer but the buffer must not be directly accessed by the CPU.
• Consideration is required for how these changes will impact software that provides multiple display support in the desktop; by putting too much control of screen memory into the hands of the drivers, it may become hard/impossible to set up screen memory in a way which makes multiple display support trivial from the OS’s perspective. I.e. it may be hard to conjoin framebuffers such that the OS can treat the two displays as one.

Potential future enhancements

• Cacheable screen memory
• Use domain IDs or similar to trap screen memory accesses. Either for implementing cacheable screen memory or for allowing the removal of the explicit syncs that the kernel currently performs after every accelerated GraphicsV operation (i.e. delay the sync until the next CPU access to VRAM)

Support for multiple heads

This topic is covered in its own proposed implementation page.

Support for hardware overlays

Overview

The primary goal is to provide a fully-featured API to allow hardware overlays (particularly YUV) to easily be used by user applications, e.g. movie players. The API must allow for both windowed and full-screen use.

Other, more complex use cases are not to be considered at this stage.

To allow for multiple display setups, the API is designed to allow for one overlay to appear on multiple displays, at different positions and scales. It is the driver’s responsibility to provide support for this feature as best it can.

Proposed changes

On some hardware (e.g. Raspberry Pi) we do not have direct access to the overlay hardware. We can only use a resource-based API which allows overlays to be allocated to us and their contents updated. Memory management is 100% under the control of the GPU. Due to this it’s proposed that the GraphicsV overlay interface uses a similar resource-based approach. Specifically, it’s expected that the following calls will be required:

GraphicsV CreateOverlay

In:

• Width, height, pixel format, rotation, transparency mode specified
• Usage pattern specified (e.g. write-only or read-write – to help driver decide on cacheability, RAM vs. VRAM, etc.)
• Flag for whether single or multiple buffers required (i.e. for flicker-free updates)
• Driver number specified in R4
• List of heads specified which the overlay is to be visible on
• Other flags as required – e.g. whether scaling may be required

Out:

• Properties updated to reflect the capabilities of the overlay that was created?
• Overlay ID returned
• Error returned if overlay can’t be created (unsupported features, no memory, etc.)

This call is used to allocate an overlay resource. The overlay is initially hidden and must be explicitly shown using the SetOverlayPosition call.

GraphicsV DestroyOverlay

In:

• Driver number specified in R4
• Overlay number specified

Out:

• All registers preserved

This call is used to free an overlay resource

GraphicsV SetOverlayPosition

In:

• Driver number specified in R4
• List of position information, one entry per head:
  • Visibility flag (on/off)
  • X, Y coords of position on head and scale factor
    • Potentially allow two ways of specifying position & scale – either as coords and scale factor or as a rectangle (with scale implicitly calculated from rect size)
  • Subrectangle of overlay to display
  • Depth value for Z sorting
  • Alpha level

Out:

• Position information updated with actual state

This call is used to update the position information of an overlay, specifying the position across all heads at once to allow the driver to reconfigure the overlay in the most efficient manner. On exit the actual used position information will be returned, to e.g. allow a window to resize itself correctly to avoid any gaps around the edge of the overlay if the driver wasn’t able to provide the exact scale factor requested.

GraphicsV LockOverlayBuffer

In:

• Driver number and overlay number specified in R4
• Usage pattern specified (R, W, RW – to allow driver to decide on cacheability, etc.)

Out:

• Pointer to buffer and row stride

This call is used to lock a buffer for update by the CPU. An overlay can only be locked by one user at a time; a subsequent lock attempt without an unlock will return an error.

GraphicsV UnlockOverlayBuffer

In:

• Driver number and overlay number specified in R4

Out:

• All registers preserved

This call unlocks an overlay buffer. However, in a multi-buffered overlay it does not schedule the new contents to be displayed.

GraphicsV FlushOverlayBuffer

In:

• Driver number and overlay number specified in R4
• Flag for memory persistency (true: ensure data is preserved ready for next lock request. false: next lock request can return uninitialised buffer)
• Flag for synchronisation? (true: block until buffer is visible, false: don’t block)

Out:

• All registers preserved

In a multi-buffered overlay this call will submit the updated buffer for display by the GPU. Even in a single-buffered overlay this call is important, as the driver may use it to e.g. flush data from the CPU cache.

Other GraphicsV calls

Calls will be needed for setting the palette and gamma tables used by the overlay. For the palette it makes sense to reuse the existing palette calls; therefore it’s proposed that overlay IDs are allocated by drivers such that they do not intersect the range of IDs used for heads. This will allow calls such as the palette ones to accept either a head ID or an overlay ID in bits 16-23 of R4.

On some hardware it’s possible to directly specify the YUV → RGB conversion matrix. Allowing this matrix to be specified at overlay creation time may be advantageous.

It may also be desirable to have the following calls available:

• A call to allow the active overlays and their state to be enumerated
• Calls to query the systems capabilities (e.g. get a list of available pixel formats)
• Allow GraphicsV_Render to be used on overlays

User-facing API

User software is to be discouraged from using the GraphicsV API directly. Instead, particularly for software running in the desktop, it’s expected that another API is to be used. Specifically the API used for the desktop must allow overlays to be associated with windows, such that the OS can automatically calculate the depth values of the overlays in response to the windows moving through the window stack. The overlay position will also be updated automatically as the windows are moved and scrolled, and the scale factor could be linked to the work area size.

A SWI call is also required to allow a window (or any other portion of the screen) to be filled with the correct colour to allow the overlay to be visible through it. For example to clear the alpha channel to zero or to write the correct colour value for systems which use a transparency colour key. For a software emulation of overlays this SWI could instead be used to directly plot the software overlay to the screen (e.g. use OS_SpriteOp calls to plot a YUV sprite).

For full-screen applications another API may be required, perhaps in the form of new OS_ScreenMode reason codes. To reduce code duplication it may make sense for user software (whether single or multi-tasking) to use OS_ScreenMode to create and update the contents of overlays. For multi-tasking software a Wimp SWI could be used to associate the overlay with a window, at which point the Wimp will automatically make SetOverlayPosition calls as appropriate. The Wimp could also be made to automatically destroy the overlay when the application dies.

Other notes

Some systems (e.g. OMAP) only have a handful of overlays available, and their abilities are further restricted by the fact that the desktop and mouse cursor are overlays as well. This means that that the overlays available can vary wildly by screen more or active head(s). Therefore it is permitted for drivers to destroy any overlays they wish when switching screen modes, or when the ClaimOS call is received. A service call or some other notification method will be required in order to allow drivers to communicate overlay destruction to the owner of said overlay – exact details TBD.

Potential future improvements

Improve screen output redirection support so that screen output can be redirected to overlay memory buffers. This may be a fairly simple API change, as redirecting output to an arbitrary memory buffer shouldn’t be much different to redirecting to a sprite – we just need the ability to use a buffer which doesn’t have a sprite header attached. The ability to redirect output to an arbitrary buffer will also ease the implementation of the new pointer API (at the moment, OS_SpriteOp 36 creates the pointer image by redirecting output to a 2bpp sprite. For the new interface, redirecting output directly to the pointer memory buffer would be desirable in order to avoid temporary memory allocations)

Add support for plotting of YUV sprites to SpriteExtend, and add a software emulation layer to the user-facing overlay API, so that software can elect to use software emulation of overlays if a suitable hardware overlay is unavailable.

Support for display detection

Overview

Provide a method to query whether a display is currently connected to a head. Provide a method to allow connection or disconnection of displays to be automatically detected and communicated to the system, e.g. via service call.

Proposed changes

Polling for connected displays

Add a new GraphicsV call to perform on-demand display detection:

• As input, the caller will provide a list of heads which he wants to check, and a buffer to place the results in
• As output, the driver will fill in the buffer with a sorted list of entries. The driver will sort the list by preference, such that the display which is most suitable for use as the default display will be first. This will allow the OS to easily decide which display to use on startup.
• Each list item will be as follows:
  • The head number
  • The connection state
• Different connection states are likely to include:
  • Connected
  • Disconnected
  • Detection not supported
  • Additional states may also be required, e.g. detection not supported due to driver being in power-saving mode

Note that under the current multi-head specification, the S-Video and Composite outputs of the Pandora are to be treated as separate heads. Display detection for these outputs is supported, but the hardware is incapable of determining which of the two outputs the display is connected to. It may therefore be desirable to be able to communicate these limitations to the caller in the result buffer, e.g. listing the connection state of multiple heads at once.

Automatic display detection

API TBD. Care needs to be taken with power saving; e.g. TV-out detection on OMAP is only possible if the TV-out circuitry is on and generating a video signal. Leaving this running all the time could place an unnecessary drain on the battery of portable systems. Therefore it’s suggested that automatic detection is performed either entirely at the discretion of the driver, or a method is available to enable/disable detection at the user’s request (i.e. discourage the development of software which enables automatic detection for long periods of time without the user’s permission). The service call which is used to notify the system of connection state changes can also be used to notify the system of automatic detection starting/stopping.

Other changes

The kernel should make use of the display detection functionality to decide which display to use on startup. The display manager and screen setup plugins can be updated to display the connection state of each display, potentially filtering out or listing separately any outputs for which no display is detected.

Support for display scaling and rotation

Overview

The overlay API will allow for scaling and rotation of overlays. However an API to allow scaling and rotation of the desktop (and implicitly the mouse pointer) is desirable.

Proposed changes

New feature flags (e.g. as part of GraphicsV 8) will be required to allow drivers to advertise that they support the feature.

Add several new VIDC control list items to allow the framebuffer size and rotation to be specified. The timing parameters of the VIDC list will remain as-is, specifying the actual timings to be used when generating the video signal.

Control list items will also be required to specify the method of scaling – i.e. the coordinates of the rectangle (in physical pixels) in which the frame buffer is to be scaled to fit.

New mode string attributes can be added to allow the user to select scaling and rotation when selecting screen modes. How this maps to mode specifier blocks is TBD – new mode variables, mode flags, or a new mode specifier block format may be required. The display manager and screen setup plugin will also need updating to allow selection of rotation. Since rotation is unlikely to be changed very often, it makes sense for the screen setup plugin to allow the user to configure the default rotation of the display. Any modes which do not explicitly specify a rotation can then have the default rotation value filled in by ScreenModes when it generates the VIDC list.

How overlay positions are specified when dealing with a scaled/rotated desktop is TBD; are they specified relative to the desktop overlay, or are they specified relative to physical monitor pixels? Specifying relative to the desktop would be the most appropriate for most software, but specifying relative to physical pixels may be desirable for some use cases. A flag at the time of overlay creation could potentially be used to control the coordinate space that it used, and for completeness it may make sense to allow any arbitrary head/overlay to be used as the parent. Care may also be required with respect to rotation; what if the user wants an overlay to be rotated on one display, but unrotated on the other?

Potential future enhancements

If a driver supports scaling, allow the system to automatically scale the display, to allow almost any mode to be used without requiring it to be listed in the MDF – i.e. as per AnyMode

If a display is able to sense its rotation/orientation (e.g. accelerometer in a mobile device), consider adding an “automatic” rotation mode to the OS where the desktop orientation will always match the physical orientation of the display.

Mode vet feedback

Overview

Improve or replace the vet mode call, to allow drivers to provide feedback on why certain modes have been rejected. Feedback should be designed such that software can interpret it and make an informed decision on how to adjust the mode timings in order to arrive at a mode which is supported. E.g. the call could return a series of flags of the form “pixel rate too high”, “complex constraint between vertical sync with and vertical front porch”, etc. This feedback would be particularly useful when generating a set of screen modes from EDID.

A call should also be available to query which control list items a driver understands, and drivers should be updated to ensure they reject any VIDC list containing items which they don’t claim to understand.

Proposed changes

TBD.

GPU accelerated sprites

Overview

The RISC OS desktop is composed almost entirely of sprites. Therefore most of the CPU time associated with redrawing the desktop is spent either generating translation tables for sprites, drawing complex sprites (translation tables/pixel format conversion required), or doing direct memcpy’s of pre-translated sprites. A method for creating and rendering sprites using the GPU should be able to significantly reduce the CPU cost involved in redrawing the desktop and as well as the overall redraw time.

Proposed changes

GraphicsV API

There are two ways the GraphicsV API could be implemented – as a cache-based system where the driver is free to release the sprite whenever it wishes (e.g. if the current palette changes, or to free up screen memory to allow for a mode change or creation of an overlay), or as a locked resource based system (where the driver must keep the sprite resident and valid until the caller allows it to be freed). If the cache approach is used then care must be taken to ensure the API isn’t designed in such a way that a user can successfully create a GPU optimised sprite only for the driver to throw it away just before the user has a chance to render it. If this happened the user would then have to try recreating it (which may fail again) or fall back to software rendering (which is what we’re trying to avoid altogether).

One key aspect feature of API should be that the driver must be explicitly told about any changes that have been made to the sprite. Direct access to the GPU optimised sprite is also likely to be forbidden; instead the API should be a one-way street where the caller passes in an OS sprite and gets a handle to a GPU sprite as a result. That same handle can be used to update the sprite (by passing in a new/updated OS sprite), but to give the driver freedom to store the GPU sprite in whatever format it wishes there should be no way for the caller to perform fine-grain actions such as updating individual pixels. The sprite update API can employ hints to reduce the amount of work the driver performs when updating the sprite. Hints should be forward-compatible such that drivers can ignore unknown hints; i.e. hints should describe what features of the sprite haven’t changed, rather than the features which have.

For a locked resource based API where the driver has to regenerate the GPU sprite after palette changes, etc., it makes sense for the driver to be able to directly reference the source sprite rather than forcing it to create its own internal copy. However this should be under the control of the caller, and the caller must notify the driver if he needs to move/modify the source sprite (for example via “DelinkSourceSprite”, “RelinkSourceSprite” calls)

User-facing API

An OS_SpriteOp-like API for interacting with GPU sprites would be desirable, and would ease transition to the new system. Whether the sprites are accelerated or not could be hidden away behind the scenes, allowing for the new API to be used irrespective of whether acceleration is currently possible or not. E.g. if the screen is being rendered to then GPU sprites would be used where possible. But if screen output is redirected to a sprite then software rendering would be used.

Care will also need to be taken as not all GPUs/drivers will be capable of all sprite operations, e.g. transparency masks, alpha blending, GCOL actions, or scaling. In particular, lack of hardware scaling can be at least be partially worked around in software, if the driver is either (a) told at sprite creation time what scale factor(s) that sprite is to be used at, or (b) allowed to generate scaled copies at render time (perhaps with a hint at creation time indicating how many scaled copies to maintain).

Changes to the desktop

To make the most of the new feature, at least the following modules will need updating to use it where possible:

• Pinboard: Backdrop sprite caching
• Wimp: Use GPU accelerated sprites for all sprites in the Wimp sprite pools
• Font manager: Remove the sprite-based font cache and use GPU sprites directly. Supremacy blending may be tricky as there’s no current equivalent in OS_SpriteOp.

Other

Analogue video formats

PAL/NTSC/etc. encoding and sending of a widescreen selection signal could be specified by VIDC control list item. See this thread for some related discussion.

Digital video formats & signalling

For HDMI sources, we need a unified way of controlling whether HDMI or DVI signalling is used, or whether the choice is automatic. HDMI also allows the source to indicate the intended aspect ratio of the picture (within the limited set of aspect ratios defined by the spec); a new VIDC control list item to indicate the aspect ratio would be advantageous, perhaps along with a new mode variable and mode string/specifier changes (using eigen values to identify pixel aspect ratio is not always accurate)

HDMI sources generally support pixel repetition, to allow display of modes where the pixel clock rate is lower than the minimum supported rate, or to boost the bandwidth available to other data such as audio. Although much of the support for this might be automatic (fully within the driver when it’s asked to switch to a low-rate mode), some external control may also be required (disabling the feature if the monitor doesn’t maintain the correct aspect ratio, or controlling whether pixel repetition will be used to ensure that high-bandwidth audio is always available, etc.)

TV offsets and overscan

TV offset (as controlled by *TV) and overscan settings (for any kind of interface – TV-out, HDMI etc.) need to be taken into account. At the moment the kernel modifies the VIDC list in order to apply the *TV setting, but it’s probably something that’s better done inside the driver. E.g. for the Raspberry Pi the overscan settings can be directly specified to the GPU, on OMAP they could be used to control the offset and scale of the desktop overlay, etc. Being able to configure these settings in the screen setup plugin would be nice.

Driver-specific configuration

The screen setup plugin could do with expanding to add support for driver-specific settings to be configured. E.g. overscan (as mentioned above), PAL/NTSC mode, compatibility options, red/blue swapping on Iyonix, etc.

Having a “configure” option in the display manager which launches the screen setup plugin for easy configuration of driver could also be nice.

Gamma

Gamma handling needs revising; at the moment the GraphicsV palette calls handle both the palette and the gamma table, with no clear indication as to whether the gamma should apply to just the desktop overlay or the pointer (and any other overlays) as well.

Splitting per-head gamma handling into a separate call would be best, along with potentially allowing gamma to be specified on a per-overlay basis, with the palette calls only used to set the palette.

Power controls

At the moment there’s no way of turning off a display once an OS/program has finished with it. The ClaimOS and ReleaseOS calls will help with this to a certain extent, but for situations where third party software is using a display separate calls to control power will be required.

Display ownership

Related to the above, an API is required to allow heads to be claimed by third-party software. Each head will have three states of ownership:

1. Free
2. System
3. Program

• Programs can claim heads only if they are in the Free or System state.
• The system may reclaim a head from a program at any time. The owning program will be notified of this, using a TBD API.
• Both programs and the system are capable of releasing the heads they own back into the ‘free’ state.
• Ownership of a head gives the owner sole control over any head-related configuration, e.g. screen mode, rotation, overlays, etc.

This system allows programs to coexist with the VDU much as they do now. Programs can use the standard OS APIs to change the screen mode, or they can claim ownership of the default head and set everything manually. If the program exits then the desktop will reclaim the head & overlays for the system, resetting the right WIMP mode.

It would also be desirable to have an API to allow for full control over whether a head has a desktop overlay and mouse pointer overlay assigned to it by the driver. For example a Geminus-like app which sets up a multiple display system would want desktop and pointer overlays on all the heads which it uses, using the existing GraphicsV calls to interact with them. But a movie player performing full-screen playback might only be interested in having a YUV overlay, accessed via the new overlay API.

Driver information

Improve OS_ScreenMode 68 or implement a new call (direct GraphicsV calls?) to allow more information about the driver to be queried. Useful information to make available includes:

• Driver version number (e.g. as per BCD-encoded module version numbers). For HAL drivers the version of the HAL component could be used.
• Driver features and other things which might otherwise be fetched via direct GraphicsV calls. Making the preferred route to read the information be an OS SWI will help shield the user from any changes to the underlying GraphicsV spec, or for things like screen output being redirected to sprites.

A call (OS_SpriteOp? OS_ScreenMode?) to check whether screen output is currently redirected to a sprite (and if so, where) would also be beneficial.

Third-party extensions

Consider how best to handle third-party (i.e. softloaded) extensions which provide additional functionality. E.g. a Raspberry Pi display which can be driven by the builtin driver, but has extra functionality (brightness, gamma, power) which requires custom code. Where GraphicsV APIs exist for controlling these features, it would be best if the display utility driver was able to provide implementations of those calls. So we need to try and make sure the APIs are designed in ways which allow utility drivers to easily implement them, without disturbing the functions offered by the builtin driver.

Document history

v1.00 – 31-Jan-2010

v2.00 – 28-Mar-2010

• Corrected some fallacies
• Attempted to make ‘default display’ discussion clearer
• Added Display Ownership and Display Capabilities & Configuration sections
• Added references to above sections in the appropriate places, deleting waffle where appropriate

v2.01 04-Jul-2012

• Amended to account for revised GraphicsV 14 work

v2.02 24-Aug-2012

• Added some notes on cursor overlays (mouse pointer)

v2.03 21-Nov-2012

• More notes on cursor overlays
• Added information about new display registration API.

v3.00 26-Jan-2014

• Rewritten to reflect current development status and my plans for other areas

v3.10 02-Mar-2014

• Added GPU accelerated sprites section

v3.20 10-Mar-2014

• Updated for current progress; Service_DisplayChanged, Service_DisplayStatus and OS_SpriteOp 65 now implemented.
• Added driver information section

v3.30 25-Jul-2014

• Remove sections on interlace handling & EDID reading
  • Interlace is now specified solely via the VIDC control list item
  • GraphicsV 3 is deprecated
  • Drivers are now required to automatically set the EDID segment pointer as appropriate when performing EDID reads
  • Support for EDID writes is discouraged

v3.40 09-Dec-2016

• Added some extra notes to the display scaling/rotation section
• Added “Digital video formats & signalling” to the “Other” section

v3.41 31-Jan-2021

• Moved the multiple display content to another page