Touch Screen support

The title says it all. As far as I can see there isn’t yet support for using touch screens in RISC OS – for example, CJE sell a touch screen monitor, but it says “no support yet” for touch use in RISC OS.

Over the years, there have been a number of touchscreens supported under RISC OS, but they tend to all have problems because RISC OS as an OS is just not touch-friendly. Whilst multi-touch / gestures could/can help, the reality is that RISC OS is hugely dependant on context-sensitive middle/right clicks and multiple window interfaces. These things don’t work very well with touch, which is why Android/iOS devices/apps tend to be full-screen affairs. A big issue is accurately clicking to pixel granularity with multi-finger touch – existing programs just aren’t really designed with touch in mind.

Of course, not all programs need that level of granularity, and if you’re writing your own applications, it wouldn’t be a problem at all, because you could design for touch from the get-go.

In the past, many touchscreen solutions emulated a mouse, or sent absolute co-ords to a module which would emulate the mouse in software (eg. serial-port connected ones). It is probably more standardised now, but there still may well be drivers needed for different types/mechanisms of touchscreen. I suspect, though, that writing a driver for one still wouldn’t guarantee wide-ranging support.

Perhaps the only useful role the OS itself could play would be a standardised mouse-type implementation for fixed-co-ordinate devices (may already exist). However, the mind boggles as to how one would work out the calibration of such devices in a generic manner, or have it co-exist with ordinary usb mice (since RISC OS’ concept of mouse type assumes a single mouse).

co-exist with ordinary usb mice

The USB bit is irrelevant. The co-existence of multiple pointing devices is an old consideration. In practice anything can move the pointer about. eg

MOUSEX%,Y%,Z%:MOUSETOX%+80,Y%-40

So don’t worry about that. The ‘menu’ click is the tricky one, and has been solved on some devices by using a physical button as a kind of shift key – touching the screen while pressing this button registers as a middle-click.

The biggest problem with touchscreen use of RISC OS is that the Desktop requires a fundamental difference between these operations:

• moving the mouse
• clicking
• dragging

…and touchscreen interfaces haven’t traditionally discerned between those concepts.

Modern multi-touch capacitive screens offer some chance of doing so, by allowing pressure (and in some cases, actual distance) to be measured – so a light brushing of the finger moves the pointer, but a firmer push is a click. Dragging might require two fingers, IYSWIM.

I don’t know how frustrating the “I didn’t mean to press I meant to move” experience would be, but if movement and clicking could be reliably discerned (and performed!) then there’s a screen-only solution to menu too – have a floating button panel at the opposite edge of the screen from the pointer, featuring ‘Menu’ and ‘Adjust’ buttons. Pressing the former generates a menu click at the pointer position. The latter toggles the effect of the next click.

It is not trivial to adapt the RISC OS GUI to a touchscreen paradigm.

There’s also the problem of hovering which is used to scroll the iconbar amongst other things. Surface pen gets around this by moving the pointer when the pen isn’t touching the screen so just hovering over the ends of the iconbar with the pen scrolls the iconbar. The pen also gives you a right click with the button on the barrel. Unfortunately the pen seems to need a double tap to register a single click when used in vnc which makes it less than useful. It’s not easy to use Draw when you have to double tap each point.

There’s also the problem of hovering

Yeah, this is the ‘moving the mouse’ bit but with the observation that if one removes ones finger from the (vicinity of) the screen, one probably does not want scrolling to continue. It might be that an ‘unfingered’ mouse in a scroll region would have to scroll with the window – so the scrolling would stop.

An interesting point Colin.

It is probably more standardised now,

That’s wishful thinking. ;-)

My tablet driver reads the size of the tablet, the size of the screen, and maps between them in various ways. Why the screen? Because if you move the pen, the driver moves the mouse pointer. Because there’s no standardised API for anything non-mouse.
Even adding pressure sensitivity is a sort of a hack. But it’s what the earlier drivers did, so it’s what I shall do.

between these operations:
• moving the mouse
• clicking
• dragging
…and touchscreen interfaces haven’t traditionally discerned between those concepts.

The touchpad on my computer does.

• Moving the mouse – tap-and-drag.
• Clicking – tap (if moving pointer, lift finger, tap, then lift finger)
• Dragging – double-tap but hold the second tap and begin to drag

As you can imagine, this will either need a smart driver to filter events before passing them to RISC OS, or changes to RISC OS so stuff like double-tap-hold (drag) is not mistaken for double-tap (run). There is some logic there somewhere, as double-click-hold can be used to open applications (rather than run them).

Perhaps more to the point, Android (and touch devices in general) do not have context sensitive menus. There’s sometimes (not always) a menu available either through a dedicated menu key (older devices) or a “…” menu graphic on-screen. Of course, when Google did away with menu keys, all the older apps that expected one (and thus had no on-screen menu graphic) suddenly had inaccessible menus.

It is not trivial to adapt the RISC OS GUI to a touchscreen paradigm.

And we haven’t even talked about the issue of multiple touches yet!

Nemo – the reason I mentioned about co-existance is that *configure mousetype (IIRC) assumes a single type of input device. If we made a *configure mousetype for an absolute input system (to assist with touchscreens and tablets), the obvious method would be as another mousetype, but that wouldn’t play nice with whatever else is plugged in.

Basically, I was trying to envision how (within RISC OS’ pre-existing framework) the OS could assist with absolute co-ordinate devices. I suspect the answer is “it can’t, so forget about mousetype”.

The best I can come up with is that some sort or mapping/translation would need to be done in the HID driver for USB-connected absolute co-ordinate devices.

That’s all.

I think that an absolute coordinate mouse system touchscreen (like the 5" Adafruit one) and a plugged in mouse can happily coexist.

There is a “touchscreen Support” for the Pi-DSI Display and any other USB-touch.
Here and Here you find it.
I use it for my PiTab. A small mpeg you find here.

Just an observation that might be useful.

Running RealVNC on my Android tablet to a RO desktop, the touch screen acts like a very large touch pad. This is surprisingly useable, with 1, 2 and 3 finger taps mapping to SELECT, ADJUST and MENU. Tap and drag works, but you cannot do ADJUST drag. For even more convenience you can overlay a transparent ‘mouse’ with three buttons and a scoll wheel.

Edit. Just to be clear, you can do ADJUST drag using the ‘mouse’ buttons. ;-)

I can confirm Steve Drains observation. RealVNC on my HP Envy (Windows 10) allows touch screen operation indeed.
Having said that, I must admit that VNC on my setup is not only terribly slow, but also completely unreliable. May I ask what server you are using on the RISC OS side? Mine is VNC, written by Steve Potts, dating from 2009.

May I ask what server you are using on the RISC OS side?

The latest vncserver module, 0.20 31 Aug 2019, is here

May I ask what server you are using on the RISC OS side?

The one mentioned by David with the VNCsvr front end.

It is reliable, but will be a slow if the network is slow, so I cannot alway use it effectively.