GraphicsV documentation

Is the GraphicsV documentation up to date?

I ask because BCMVideo has reason codes 20-28 in its jump table that aren’t listed in the documentation.

It’s out of date. Those extra reason codes are used for the overlay system (which is documented in the VideoOverlay repo – https://gitlab.riscosopen.org/RiscOS/Sources/Video/Render/VideoOverlay/-/tree/master/doc). I’ll go through those docs and add them to the wiki. As far as I know that’s the only thing that’s missing from the GraphicsV docs.

I see you’ve already added them, thank you.

Overlays look to be incredibly useful, particularly for the blitter in ADFFS as I can drop software scaling and let the overlay handle the aspect ratio scaling. It also means I could ditch all the code that handles changing GPU resolutions when the VIDC20 parameters change – so a win-win.

Are there any plans to extend NVidia and PineVideo to support overlays which I would guess support them at hardware level?

Overlays look to be incredibly useful, particularly for the blitter in ADFFS as I can drop software scaling and let the overlay handle the aspect ratio scaling.

It will all depend on the king of overlays the machine you are targeting support. If they are limited to the YUV kind it will not help you.

Anyway, when I integrated Overlays in KinoAmp from the prototype made by Jeffrey, I could test the RGB overlays on the Pi3 and so that machine can serve you for your developments without having to wait for updated drivers on the these machines. As for some tips:
- the RISC OS implementation associates Overlays with windows, so for full screen mode you have to put an invisible window on top the screen area that overlay should cover.
- you can use overlay banks instead of screen banks.

Are there any plans to extend NVidia and PineVideo to support overlays which I would guess support them at hardware level?

Yes – but no promises on when it will happen. Both platforms might need a fair bit of experimentation to get it working properly due to lack of documentation.

I’ve started coding overlay support but I’m missing some information from the documentation – or I’ve just missed it.

GraphicsV 23 mentions “plane” and “stride”. What is a plane and how do I know how many there are? Is the stride specified in pixels or bytes? Do the banks start from 0 or 1?

GraphicsV 20 returns the overlay type (Z-order or Basic) but it doesn’t let you specify what type you’d prefer for the overlay?

How do you determine the current “head” for the overlay? GraphicsV 18 doesn’t have an item for available heads and OS_ScreenMode 11 only returns the current driver, not the primary head.

Did a search for ‘plane and stride graphics’ a site that might help with ‘stride’ came back
https://medium.com

You certainly like puzzles :-))

GraphicsV 23 mentions “plane” and “stride”. What is a plane and how do I know how many there are? Is the stride specified in pixels or bytes? Do the banks start from 0 or 1?

Planes define how your data is managed. For example in YUV, planes could consist of 3 “images” one for the luminance Y and 2 for chrominance (U and V planes) and since the chrominance usually covers 2 horizontal pixels or a square of 4 pixels they have a stride of half that of the luminance one.

You noticed that in the previous paragraph “planes could consist”, well YUV formats can also consist of a single plane which each word packing the info of 2 pixels (2Y bytes 1 U and 1 V). And like RGB, various ways of arranging the bytes in the word exists.

RBG formats use a single plane in classical screen layout.

The stride is defined in bytes and the banks starts from 0.

This little table extracted from KinoAmp (c.display) provides you a summary of what currently exists:

static const uint32_t overlay_modes[][4] =
// YUV first, by increasing subsampling, RGB by decreasing resolution
// NColour ModeFlags Log2BPP BPP
{ {0x36315659, 0x6000, 7, 1} // YV16, Plane 0 is 8bpp Y, Plane 1 Cb, Plane 2 Cr, 2x1 sub-sampled
, {0x59565955, 0x6000, 7, 1} // UYVY, 32bpp words of &Y1.Cr.Y0.Cb, 2x1 sub-sampled
, {0x32595559, 0x6000, 7, 1} // YUY2, 32bpp words of &Cr.Y1.Cb.Y0, 2x1 sub-sampled
, {0x32315659, 0x6000, 7, 1} // YV12, Plane 0 is 8bpp Y, Plane 1 Cb, Plane 2 Cr, 2x2 sub-sampled
, {0x3231564e, 0x6000, 7, 1} // NV12, Plane 0 is 8bpp Y, Plane 1 is 16bpp &Cr.Cb, 2x2 sub-sampled
, {0x3132564e, 0x6000, 7, 1} // NV21, Plane 0 is 8bpp Y, Plane 1 is 16bpp &Cb.Cr, 2x2 sub-sampled
, {-1,         0x0000, 5, 4} // TBGR32
, {-1,         0x4000, 5, 4} // TRGB32
, {-1,         0x8000, 5, 4} // ABGR32
, {-1,         0xC000, 5, 4} // ARGB32
, {65535,      0x0080, 4, 2} // BGR16
, {65535,      0x4080, 4, 2} // RGB16
, {65535,      0x0000, 4, 2} // TBGR15
, {65535,      0x4000, 4, 2} // TRGB15
, {65535,      0x0000, 4, 2} // ABGR15
, {65535,      0x4000, 4, 2} // ARGB15
, {4095,       0x8000, 4, 2} // TBGR12
, {4095,       0xC000, 4, 2} // TRGB12
, {4095,       0x8000, 4, 2} // ABGR12
, {4095,       0xC000, 4, 2} // ARGB12
};

How do you determine the current “head” for the overlay? GraphicsV 18 doesn’t have an item for available heads and OS_ScreenMode 11 only returns the current driver, not the primary head.

You specify the head to use when positioning the overlay with Graphics V22. In practice on the user side, since the Overlay module associates an overlay with a window handle I wonder how it can assign the correct head.

This little table extracted from KinoAmp (c.display) provides you a summary of what currently exists:

I can’t see any of that table in the documentation. Should the Mode Selector Block in GraphicsV 20 actually be an entry from that table? It currently links to Mode Selector Block which is something completely different.

I can’t see any of that table in the documentation.

The YUV part of that table comes from the ModeVars file of the VideoOverlay module doc.

Should the Mode Selector Block in GraphicsV 20 actually be an entry from that table?

Entries from that table serves to fill in the Mode Selector Block variables NColor, Log2BPP, ModeFlags of the Overlay desired by KinoAmp.

Entries from that table serves to fill in the Mode Selector Block variables NColor, Log2BPP, ModeFlags of the Overlay

I didn’t appreciate the Mode Selector Block required those values. From looking at the source code of VideoOverlay and BCMVideo I did eventually figure it out before you confirmed it. I’ll go back and revise the documention once I have it working.

I think my code is now creating a triple bank overlay, although I can’t get it to display on-screen. From looking at the VideoOverlay source I determined I need to obtain the TCK pixel value from the current driver and fill the screen with it – VideoOverlay uses OS_SetColour/OS_Plot with the palette entry returned from GraphicsV 12, but I can’t do that so am just writing the value into the current GPU framebuffer. I then transform the overlay to scale to the GPU resolution and the blitter rotates around the three banks displaying them at 0,0 but the overlay doesn’t appear.

I’m not sure if I need to set the Z-order and I’m also not sure if the overlays are being destroyed whenever the game changes resolution. I’ve assumed they are being destroyed, so I’m recreating the overlay whenever GraphicsV 6 is called. I’m hoping that’s not actually necessary as the underlying GPU resolution isn’t actually changing – it depends on what’s destroying the overlays.

EDIT: I neglected to mention that this is all occurring within a GraphicsV driver that is emulating a VIDC20. That rules VideoOverlay out, so I’m essentially replicating what it does via GraphicsV calls to the underlaying GPU GraphicsV driver

I’m also not sure if the overlays are being destroyed whenever the game changes resolution. I’ve assumed they are being destroyed

The VideoOverlay docs, GraphicsV file indeed mentions this behaviour:

To account for driver restrictions and to allow for more optimal memory management, all overlays for a driver will be destroyed when the driver changes mode.

I’ve started coding overlay support but I’m missing some information from the documentation – or I’ve just missed it.

Combination of missing + poorly organised. Sorry!

GraphicsV 23 mentions “plane” and “stride”. What is a plane and how do I know how many there are? Is the stride specified in pixels or bytes? Do the banks start from 0 or 1?

Stride is the number of bytes between each row, i.e. the address of pixel (X,Y) is given by the equation base + X*bytes_per_pixel + Y*stride (with 0,0 being the top-left pixel, positive X pointing right, positive Y pointing down).

I believe André’s answered the rest of those questions.

GraphicsV 20 returns the overlay type (Z-order or Basic) but it doesn’t let you specify what type you’d prefer for the overlay?

Correct. Either that’s an oversight, or I just decided it was best to keep it simple for the first version of the API.

The current implementations will prefer to give you Z-order if it’s possible, since that’s the most flexible. Are there situations where you’d prefer Basic to be used?

How do you determine the current “head” for the overlay? GraphicsV 18 doesn’t have an item for available heads and OS_ScreenMode 11 only returns the current driver, not the primary head.

Currently head numbers will always be zero, since multi-head support is still very rudimentary. Eventually though I’d expect it to be exposed as another VDU variable (Current GraphicsV driver is also exposed via VDU variable &C0)

This little table extracted from KinoAmp (c.display) provides you a summary of what currently exists:

See also the table of Valid mode variable combinations (which now lists the new FOURCC-based overlay modes – I accidentally missed that page when updating things last week).

I think my code is now creating a triple bank overlay, although I can’t get it to display on-screen.

Since you mentioned BCMVideo:

• If you’re in an alpha-blended screen mode, you’ll be getting a Z-Order overlay. I can’t remember offhand if you need to call GraphicsV 28 to set the Z order, but you will need to write transparent pixels to the main screen (i.e. just set the pixels to zero), and use GraphicsV 22 to make sure it’s positioned somewhere underneath those pixels.
• If you’re not in an alpha-blended mode, then you’ll be getting a Basic overlay. So to make it appear all you should need to do is call GraphicsV 22

Also note that VideoOverlay is a bit foolish and tries using the TCK colour regardless of whether it’s an alpha-blended screen mode or not (it should only be trying to use the TCK colour if the current screen mode doesn’t have an alpha channel). With the way the current drivers are implemented it will work (since the drivers will be returning with R1=0), but it should probably be made smarter and just default to zero if alpha blending is in use.

Combination of missing + poorly organised. Sorry!

No problem, I figured some of it out through trial and error and started adding to some of the pages if you want to four-eyes: GraphicsV 20 / VideoOverlay_Create

The current implementations will prefer to give you Z-order if it’s possible, since that’s the most flexible. Are there situations where you’d prefer Basic to be used?

I was originally thinking a Basic overlay as its taking over the whole screen to emulate VIDC20. However…Currently the Pointer is blitted into that overlay after the main screen blit, but with overlays it would make more sense to maintain a secondary overlay for the Pointer, so I guess I need Z-Ordered?

In the scenario where the Pointer is a separate overlay, if the driver comes back with Basic do I need to create the Pointer overlay after the screen overlay to ensure its on top. And when switching banks on the screen overlay, will that affect the order?

Since you mentioned BCMVideo

I did manage to get it displaying last night, although I need to do more digging as I’m seeing some very odd behaviour using three banks. If I bank switch from inside my GraphicsV 1 handler, I’m seeing overlays stuck on screen after they’ve been destroyed. I’ve tried discarding the banks before destroying the overlay as GraphicsV 21 doesn’t state if it discards banks.

I think this might be caused by a threading/reentrancy issue in BCMVideo.

note that VideoOverlay is a bit foolish and tries using the TCK colour regardless of whether it’s an alpha-blended screen mode or not

Yes, I did look at that code and wonder what would happen if there wasn’t a TCK. For the time being I’m just overlaying over the current desktop – I should probably check and switch to an alpha 24bpp mode if required. From a quick test, if the desktop is 256 colour and I request an alpha 24bpp overlay it hard-locks…I’m not sure what’s causing it though.

I have a question about transform scaling. Assuming I have a clip rectangle of 1360×768 (16:9) and an overlay that’s 320×256 (4:3) that I want to scale to fit it – how do I do that with GraphicsV 27 ? I’ve tried various combinations of scale and aspect ratio and none result in the overlay being scaled to the expected 960×768.

EDIT: Having looked at the GraphicsV 27 implementation in BCMVideo, it doesn’t appear to use the aspect ratio in its scaling calculations, and the scaling does not appear to scale proportionally to the requested size, so I’ve calculated the scaling myself. Perhaps you could add an explanation of what the transform is to the documentation, along with an example for clarity, so it’s clear to someone implementing GraphicsV 27 what calculation should be used?

I was originally thinking a Basic overlay as its taking over the whole screen to emulate VIDC20. However…Currently the Pointer is blitted into that overlay after the main screen blit, but with overlays it would make more sense to maintain a secondary overlay for the Pointer, so I guess I need Z-Ordered?

Z-Ordered would be the best choice, yeah.

In the scenario where the Pointer is a separate overlay, if the driver comes back with Basic do I need to create the Pointer overlay after the screen overlay to ensure its on top. And when switching banks on the screen overlay, will that affect the order?

Switching banks won’t affect the order of the overlays.

For BCMVideo, you can actually control the ordering of Basic overlays, by using GraphicsV 28 just as if they were Z-Order overlays. The only real difference is that the desktop will be behind them instead of on top of them. (I’ll probably make the API more descriptive & flexible in the future, to allow overlays to be freely moved in front or behind of the desktop where possible)

For OMAP3, there are three overlays, but two are used by the OS (desktop + pointer), and hardware restrictions mean we can only place the third behind the desktop if the desktop is in an alpha mode.

For OMAP4 there are four overlays (so two will be free for use as GraphicsV overlays). I think technically we can fully control their order, but the driver currently only allows you to do that if they’re Z-Order overlays (which requires the desktop to be an alpha mode).

So in summary, just make sure you use an alpha mode for the desktop and you shouldn’t have to worry about Basic overlays, at least for the current drivers.

I’ve tried discarding the banks before destroying the overlay as GraphicsV 21 doesn’t state if it discards banks.

GraphicsV 21 will discard the banks.

From a quick test, if the desktop is 256 colour and I request an alpha 24bpp overlay it hard-locks…I’m not sure what’s causing it though.

That’s odd – that’s something the hardware should support (on the Pi the pointer is a 32bpp alpha blended overlay), and it feels like something I would have tested.

I have a question about transform scaling. Assuming I have a clip rectangle of 1360×768 (16:9) and an overlay that’s 320×256 (4:3) that I want to scale to fit it – how do I do that with GraphicsV 27 ? I’ve tried various combinations of scale and aspect ratio and none result in the overlay being scaled to the expected 960×768.

For scaling I think this is all you need:

R0 = 0
R1 = 0
R2 = scaled_w
R3 = scaled_h
R5 = scaled_w
R6 = scaled_h
R7 = (scaled_w<<16) + scaled_h

If the desktop mode is 1360×768, and you want to scale the overlay to 960×768 (in desktop pixels), then use 960 and 768 as the scaled_w and scaled_h values (and use GraphicsV 22 to position it at (200,0) so that it’s nicely centred).

Also be careful to make sure bit 0 of R2 is set when calling GraphicsV 20 to create the overlay (otherwise you’re at the mercy of the driver whether it’ll allow you to scale it or not – OMAPVideo seems to be a bit strict and doesn’t allow you to scale the overlay unless you set that bit)

How possible would it be for the desktop and mouse overlays to be managed by the kernel and for upscaled screen modes to be handled there instead of having the individual video drivers handle it?

I’m finding Overlays to be somewhat troublesome.

Switching my existing code from using GraphicsV 2 which directly sets the mode at VSync to use GraphicsV 21/20/22/27 results in one of the following if a game is changing VIDC20 parameters that require the Overlay to be altered.

1. The system locks when the game changes resolution
2. The screen goes black and locks when the game changes resolution
3. A green box appears on the screen
4. When returning to the desktop the Mouse pointer doesn’t move
5. When returning to the desktop the Mouse pointer doesn’t appear
6. When returning to the desktop the game overlay remains

At first I thought it might be due to Destroying/Creating the overlay too often, so I changed the code to set the overlay size to the HCR/VCR size and only recreate the overlay if those changed – ie when an actual mode change occurs. In this way, games that just alter the borders are only causing the overlay position and scaling to change.

That didn’t change the situation so I tried shifting the blitter/overlay adjustment from within GraphicsV 1 to a 50Hz RTSupport thread – that just changed when it crashed.

I think Dispmanx might be getting overwhelmed. If you cast your mind back to the early implementation of BCMVideo we were seeing similar issues when I was setting the mode during VSync and altering the palette. I don’t recall what changed in BCMVideo to resolve it – possibly buffering mailbox requests or altering when they were sent?

The pointer issues remind me of the problems we were seeing when Gamma is enabled…I’m now wondering if they’re related and the issue is with the pointer overlay – not the gamma. That might explain why the Pi devs couldn’t reproduce what we were seeing with RISC OS, in Linux, but we could reproduce it simply by moving the pointer overlay position.

Another possible issue I’ve noticed is with parameter validation. I’m assuming there are restrictions on the size of an Overlay with Dispmanx, but there’s currently no way to determine what they are from GraphicsV. What for example is the minimum width/height and does BCMVideo adjust the size automatically to be within legal limits?

Caching/Buffering of mapped banks – these aren’t detailed in the documentation but from looking at the code mapped banks are buffered but not cached. Could control over these and possibly the access privileges be added to GraphicsV 22 – I would have thought the OS might want to restrict access to its overlays as an example and as I’m filling the overlay potentially every VSync, caching should reduce CPU stalls?

From a quick test, if the desktop is 256 colour and I request an alpha 24bpp overlay it hard-locks…I’m not sure what’s causing it though.

That turned out to be my code – a side effect of not being able to request the logical address of the DAG via GraphicsV when it needs to fill the screen with TCK. As my GraphicsV driver has taken over, it can’t ask the OS for the logical address as it doesn’t understand multi-GPU/heads. I figured out a workaround – use OS_ReadVduVariables to get the address/size of the screen in Service_PreModeChange but only if my GraphicsV driver isn’t active.

EDIT: I’ve created a Repro for the locks in this bug report

How possible would it be for the desktop and mouse overlays to be managed by the kernel and for upscaled screen modes to be handled there instead of having the individual video drivers handle it?

I’m not sure. There are some API & hardware quirks which will complicate things.

I think Dispmanx might be getting overwhelmed

On this point, I spent yesterday shifting the timing of the Overlay GraphicsV calls. With the screen at 50Hz and all Overlay calls made via a 50Hz RTSupport ticker, the issues all go away if the Overlay changes are made within the first few instructions. The ticker also handles the Joystick position/key mapping via a BL (to code with ~50 instructions and an SWI Joystick_Read), if I move the Overlay code to after that BL the issues come back.

That confirms the issues are very timing specific. What it doesn’t help with is root cause – so I’m still unsure if the issues are caused by BCMVideo, the mailbox(s) or Dispmanx.