Generic OpenGL interface

During the last 25 years, there have been quite a lot of attempts at porting OpenGL and/or OpenVG to RISC OS:

• Three ports of Mesa’s software renderer, one by David Boddie, one by Chris Gransden and one by myself.
• Another port of Mesa targeting early GeForce graphics cards on the Iyonix by Simon Wilson.
• A port of the older Broadcom OpenGL ES stack for the Raspberry Pi 0-3 by Lee Noar.
• Yet another port of Mesa that was mentioned as part of a talk on Iris by RISC OS Developments but has not received a public release that I’m aware of.

However none of them seem to have really taken off despite how significant such a thing would be. From my perspective, the biggest reason for this is because there isn’t a universal API to allow a single application to target multiple implementations, and that most of them require statically linking the implementation to the programs that use them. As such, I’m thinking it would be a good idea to discuss what a universal OpenGL interface would look like?

• How much inspiration should be taken from existing implementations like the Khronos module?
• Do we need to have Desktop OpenGL, or will OpenGL ES 1 and 2 suffice?
• If we use EGL for interfacing with the windowing system, what should the EGLNativeDisplayType, EGLNativePixmapType and EGLNativeWindowType types map on to?
• Some APIs support sharing resources with each other. What would be the easiest way to approach that?
• Would it be necessary to extend the Overlay API to accommodate any of this?

Any feedback or suggestions would be greatly appreciated.

Off the top of my head, what I think would be needed is something akin to DirectX that “just does stuff” (as close as possible to what the application is asking for, but if that isn’t possible it should lie and fake it). The application should not have to have a half dozen code paths for a half dozen different things.

Targeting specific machines and/or specific video devices is restricting, especially given that the ancient RiscPC is still widely supported (as it’s our only emulation solution).

I note also Lee’s comment in the ReadMe that the GL surface is always over top of the RISC OS frame buffer. That might work in a single tasking game, but could be… troublesome in a Wimp app.

Off the top of my head, what I think would be needed is something akin to DirectX that “just does stuff” (as close as possible to what the application is asking for, but if that isn’t possible it should lie and fake it). The application should not have to have a half dozen code paths for a half dozen different things.

I’d avoid inventing a brand new 3D API just for RISC OS – it’s quite likely that most 3D software will end up being ported from other platforms where OpenGL support is almost universal, and needing to write a custom renderer for every application is likely to be too much of a burden. 3D graphics APIs are well known for being very complex, so adopting a well proven standard would be better in the long run.

There are a few different variants of OpenGL (same as Direct3D as I understand it), but it should be possible to implement the older APIs on top of new ones within the driver, so from an application developer’s perspective you can target OpenGL ES 1 and support all RISC OS machines with it.

Targeting specific machines and/or specific video devices is restricting, especially given that the ancient RiscPC is still widely supported (as it’s our only emulation solution).

Personally, the RiscPC is interesting to target even without the need to support emulation. It would be necessary to have software rendering anyway since not all machines that could have hardware accelerated 3D are necessarily going to have drivers written for them.

I note also Lee’s comment in the ReadMe that the GL surface is always over top of the RISC OS frame buffer. That might work in a single tasking game, but could be… troublesome in a Wimp app.

It’s troublesome in full screen as well, since if it isn’t destroyed by the application when exiting or recovering from a crash it’ll stay in place until the next reboot. I’m hoping that will be easier to integrate now that Overlays exist, but I don’t how much knowledge of Overlay internals the Broadcom stack might need.

Off the top of my head, what I think would be needed is something akin to DirectX

There’s OpenGL, OpenGL ES, OpenCL, Vulkan, Khronos and MESA, which are all open and cross platform, some of them have been ported to RISC OS at one time or another, and you mention a proprietary Windows API?

I think Rick was speaking more of the “it just works” aspect.
The average user doesn’t want to know how, why, or specific incantation required.

Except it didn’t just work at first, until all the smaller graphics card players had been driven out of the market leaving a stitch up between Microsoft and nVidea/ATI (AMD)/Intel.

If it helps, I’ll try to explain how the existing implementation works and how it should ideally look IMHO.

The way the Khronos module works is that it provides Khronos_EGL, Khronos_GL and Khronos_OpenVG SWIs. These take an integer corresponding to one of those functions in R8, and the parameters for those functions in R0-R7. Static libraries are provided to allow calling the SWIs using the standard C interface.

The problem with the current implementation is that it relies on DispmanX functionality for the EGLNativeDisplayType, EGLNativePixmapType and EGLNativeWindowType types (which are normally OS-specific). This limits it to older Pis and makes it harder to use properly in applications. The main question here is “What should these types be on RISC OS, and how much control should the Khronos module have over them?”

When discussing the subject on Discord, Gerph mentioned the fact that using SWIs introduces quite a bit of overhead for a lot of the smaller functions. It would be nice to have a module version of eglGetProcAddress that would allow getting around that, but that would require settling on a calling convention and a bit more complexity with the static library bindings.

It’s also not clear how the module copes with with multiple instances, since OpenGL relies a lot on global state that is normally unique for each process.

The other implementations use C linking, which either links the implementation into the application statically, or places a hard dependency on the ELF loader. The publicly available ones provide a simple API in place of EGL which works for the specific implementation but aren’t versatile enough for a “one size fits all” solution.

In my opinion, the best solution would be to resemble the Khronos module as much as possible, but with the points I mentioned above addressed in some way to allow for a universal solution that works on all machines. Whether this should be done through breaking changes to the Khronos module or with a new module and SWI base is something I’m less sure about, but since only a couple of open-source games use it at the moment, either way should be viable.

I’m not an expert on how drivers work internally though, which is why I’m after feedback from other people who have more experience in these matters than I do.

I don’t know enough (yet) to properly weigh in, so ignore me if this is stupid! However, I wonder if Vulkan may have a role in the future of RISC OS 3D APIs?

We are currently in a weird situation regarding 3D APIs…
– Microsoft will be doing their thing for sure with DirectX, but Vulkan is also supported on Windows (not sure how well)
– Apple is doing their own thing with Metal. While OpenGL is supported on iOS / macOS, it is very deprecated, and will vanish soon
—> however, there is a wrapper library (MoltenVK IIRC) which allows Vulkan apps to use run on Metal (I am guessing this works as both are low level)
– Linux supports Vulkan
– Raspberry Pi supports Vulkan!
– (my MacBook Pro mid-2012 Unibody supports Vulkan too, with an ancient Nvidia GT650M, so it can run on old, as well as new, hardware)

The upside of Vulkan, therefore, is that it runs on a lot of platforms, and is becoming more popular
The downside is that it is low-level, and I have read that it takes 1000 instructions to set everything up to draw your first triangle (of course it can be blazingly fast once all the initial setup is done!)

So, I do wonder if Vulkan support is something we could consider (at the close-to-bare-metal level?). A Vulkan module could probably have SWIs supporting direct entry points to performance critical routines, and could have a way to send it unique IDs so that it can work with more than one app at a time).

Such a module could then be used by apps, including stuff being ported over to RISC OS from other platforms, but could also be used by other modules, such as SDL2 to make things easier, or even ‘RISC OS specific API modules’, which could do things like set up a simple set of shaders etc, and allow users to plot triangles and so on, with the ease of OpenGL <2.0, before things got messy!

Vulkan’s not a bad thing to have, but I left it out of this proposal for a few reasons:

• AFAIK Vulkan is independent of OpenGL and will likely have different porting requirements compared to OpenGL, so they can be investigated and implemented independently of each other.
• Of the devices that RISC OS runs on, only the Raspberry Pi 4 is known to have a complete open source driver for a specific version of Vulkan. Rockchip devices may be able to use the panvk driver in Mesa (or a derivative), but I’m under the impression that it’s not sufficiently complete yet. Many other devices (the Raspberry Pi 0-3, OMAP devices, the i.MX 6, the Titanium and the original Pinebook) are limited to GLES 2, which AFAIK means that none of them are capable of Vulkan. In comparison, OpenGL ES 2 should work on everything except the RiscPC and standard Iyonixes, while OpenGL ES 1 should work on everything including those two, which means that providing OpenGL would be a more universal solution than Vulkan.
• I’ve not encountered that many projects using Vulkan exclusively that would be feasible to port to RISC OS. For reference, I’d like to be able to use OpenGL for 3D games in ScummVM, for accelerated 2D rendering in SDL2, and for a bunch of open source games that make direct use of OpenGL. In comparison, Vulkan seems to be mainly popular with proprietary next-gen games, emulators for modern systems and a handful of non-game applications, and those tend to have additional barriers with porting to RISC OS like requiring C++20 or larger frameworks. As such, OpenGL is likely to be more useful on its own than Vulkan is.

That said, a useful benefit of having Vulkan is that it could be used to implement OpenGL using libraries like ANGLE. That way you’d get the additional benefits of Vulkan and the near-universal compatibility of OpenGL without having to write multiple drivers for the same device.

It’s something to consider later on IMHO. I’m not particularly knowledgeable about Vulkan, though.

the fact that using SWIs introduces quite a bit of overhead for a lot of the smaller functions

I’m just wondering aloud… how does the CLib “stubs” library work? Is it still calling SWIs behind the scenes (and therefore succumbing to the same overheads) or does it do something more clever? If it’s the latter, would it be practical to have an “OpenGL stubs” library as well, interfacing with a module?

I suspect that this will make killing the module “dangerous”, like when you kill a softloaded CLib so it might want the “refuse to die” behaviour that some have campaigned for in the past… but I’m getting way ahead of myself here :)

I’m just wondering aloud… how does the CLib “stubs” library work?

Regarding the overhead mentioned by Gerph, it’s more like having to perform many SWIs to proceed in a pipeline, resulting in significant overhead, which leads to what Rick mentioned:

that “just does stuff”

In other words, you prepare your data, call one, two, or a maximum of three SWIs, and the whole process gets done.

Besides that, there are a few considerations here (just my 0.5c as usual):

- First, a clarification: OpenGL is not just for 3D acceleration; it’s also used for 2D graphics and is focused solely on rendering graphics. It was originally designed to move such workloads from the CPU to the GPU, enhancing performance through hardware acceleration. OpenGL does not handle other multimedia tasks like audio, input, or window management, which differentiates it from APIs like DirectX that are more comprehensive in scope.

- Second: While this entire discussion works for games, which in most cases run outside the WIMP and in full screen, it’s a completely different story for WIMP-based applications. These would likely require such a library to be integrated (at the very least) with the WIMP itself. Why?
  - What happens when an application using OpenGL crashes?
  - How does the driver know to which graphic pipeline a new SWI request belongs?
  - How do we tell OGL to move a window (which is not a WIMP window for non-developers reading, but should be identified more like a “graphic window” inside a WIMP’s window) or to hide a portion of it because another WIMP window was moved on top of a portion of the OGL graphic window, etc.?
  - How or what should deal with the WIMP events that may influence OGL behavior?
  - How should the OGL pipelines “reconcile” with the WIMP’s cooperative nature?

In other words, so far we have seen either OSMesa ports, which have the peculiarity of a static bind to a specific app and run a user-space framebuffer on top of the RO FB. This is a clever shortcut, and given the above nature of OGL (small steps into a pipeline), it also matches well with the cooperative scheduler in the WIMP (contrary to other Unix ports that make the RISC OS Desktop look and feel bloated and very slow).

However, a module would need to deal with all the complexity above if it is to be a universal API for both 2D and 3D acceleration in both single and multi-tasking environments. It’s not impossible though, just the usual “need to think about it carefully first, because it’s RISC OS we are talking about”, which I believe is the reason for Cameron’s initial post here and for seeking guidance and feedback.

HTH

I’m just wondering aloud… how does the CLib “stubs” library work?

It’s just a list of function pointers that are populated when the library is initialised. So everything is a function call, no SWIs involved. The SCL is probably the only module that runs 99% of the time in usr mode (when called from an application).

More stuff should be like that, for performance reasons. SWIs are an ugly, slow, solution, but have the advantage that things can work well in BASIC. The Toolbox was originally intended to be a C library with direct function calls, but it was decided that BASIC was too important, hence the idea of filters. There’s some documentation these days, but trying to figure out how to extend the Toolbox was a nightmare. Plus it means it runs in supervisor mode – we ended up writing the text area module as a standalone app so it could be debugged, and later converted it to a module.

Second: While this entire discussion works for games, which in most cases run outside the WIMP and in full screen, it’s a completely different story for WIMP-based applications. These would likely require such a library to be integrated (at the very least) with the WIMP itself.

Most of the points you mention could be implemented using Wimp filters – Replay 3 managed that quite well. It’s probably not the best approach, though, but I’m certain it could be made to work. Replay 3 kept track of all its windows’ positions and generated a mask, much like you’d use with OpenGL.

A better solution would be deeper integration, and for the Wimp to composite all windows using OpenGL. Each window should be a sprite that’s plotted to, off-screen. There’s little point in accelerating one window if everything else around it is drawn in software.

I know that that is one of the reasons why RISC OS used so much less memory and was generally faster, but these days it’s daft not to do it the other way round. It could be done transparently in most cases – apps just get a sprite with origin 0,0 or whatever.

A better solution would be deeper integration, and for the Wimp to composite all windows using OpenGL. Each window should be a sprite that’s plotted to, off-screen. There’s little point in accelerating one window if everything else around it is drawn in software.

100% agree