Bounty Proposal: Increase RISC OS Security

Why not a module? Because modules don’t use the C ABI.

Normally, no. But the SharedCLibrary manages.

Use a shared object → thousands of off-the-shelf libraries become available to be shared between applications very rapidly.

Sounds like a good plan, but…

Using “kernel modules” as shared libraries is an insane hack that was only used because we didn’t have shared objects yet.

And how do you plan to implement this? A horrible kernel hack to never map out certain bits of memory in which the library will reside? Try running code in a DA? Hijack the top bit of “page zero” for your jump table? Or maybe, god forbid, it would be implemented as one of those horrible horrible “kernel modules” (are there any other sort in RISC OS? ;-) ).

Now we do.

I feel obliged to point out that RISC OS is not Linux. I have trouble imagining how such a shared library would actually be implemented without more or less doing it like CLib…

aaaa

Rick: The SCL cannot be replaced at runtime (you can replace it exactly once; the ROM copy with a softloaded copy). I do not plan to use modules as shared objects, so I have no plans to implement it. We already have ELF shared library support. On 32 bit OSes (like RISC OS 5 and 6), it uses a DA. On 26 bit ones, it stores them in the RMA (but importantly the code is not run in supervisor mode.)

The important thing is that this stuff already exists and works on RISC OS, but people keep abusing modules.

Rick: The SCL cannot be replaced at runtime (you can replace it exactly once; the ROM copy with a softloaded copy).

True, but this isn’t unheard of. Why is it that after installing some software on Windows, it wants to be restarted? Answer – because you can’t upgrade a shared resource that is currently in use. [might be similar on Linux? I have no experience of that…]
Really, CLib (softload version) should have refused to die if anybody had called the init SWI. But it doesn’t, which means it can be freely replaced. Bang!

Can an ELF shared library be replaced on the fly at runtime?

On 32 bit OSes (like RISC OS 5 and 6), it uses a DA.

Mmmm… A Dynamic Area for holding a library. That’s an interesting idea.

(but importantly the code is not run in supervisor mode.)

Neither are calls into CLib from a user mode application, though they can be (so modules written in C can use the same functions).

To break it down, when a C program calls a function, for example “_kernel_swi()”, it will take a BL branch to the jump table. In a program as stored on disc, the jump table is a long list of “MOV PC,#0” instructions (reset vector? yikes!) followed by a set of zero words.

However, once the SWI “SharedCLibrary_LibInitAPCS_32” has been called, the jump table instruction will say “LDR PC,&xxxxxxxx”, where the address to load from is one of the zero words following the jump table, in this case &FC19C7C8 (loading PC from a constant permits us to access any address at will).
After this, we are now in the CLibrary, and the code that can be seen using *MemoryI on the address referred to will correspond with the code that is present within CLib (…castle.RiscOS.Sources.Lib.RISC_OSLib.kernel.s._k_body) which basically stacks stuff, constructs the SWI number in R12, loads the register block, calls CallASWIR12, copies the registers back to the output block, sets NULL return if there was no error, and pulls the registers to return to the caller.
The code is located in the RMA and it is capable of handling code in SVC mode… however in this instance, as with any user mode application, there was no mode change. The code executed in USR mode. It simply wouldn’t make sense to switch to SVC mode just because we are calling a routine in the RMA when such a thing is not technically necessary. This misnomer arises from the fact that modules are usually communicated with using SWIs and SWIs run in SVC mode. This isn’t an attribute of the RMA or of modules, it’s just how the processor deals with SWIs.
Because of what modules typically do, most of the entry points are with the processor in SVC mode, with the notable exception of the Start entry point. If you have ever written a module application you will notice three things:
1, Its base address is not &8000
2, It never gets paged out (as it is in the RMA, not normal application workspace)
3, It runs in USR mode, despite being in a module and in the RMA
CLib is similar, as any examination of how calls are actually made would demonstrate.

but people keep abusing modules.

Possibly because it is seen as simpler to “is module there? is it new enough? then call a SWI”. That’s more or less what the CLib loader stub inside every program does. It is simple. Maybe too simple, but you get the point.

That said – show me a dynamic library that works with “Norcroft”, then we can talk. ;-) On the other hand, a hybrid that works sort of like CLib but is located in a DA may be “doable”. Frankly, the problem as I see it is not where the code is, but how to sensibly handle requiring loading a newer library version when an older version is currently in use.

Posted as a bug in the bug tracker [https://www.riscosopen.org/tracker/tickets/405] (whoa! it took seven minutes to post this bug).
It’s one that has bothered me for ages !

It is possible to replace the ROM version of CLib with a softloaded one – because the ROM version is still in ROM so direct links, such as within !Edit etc, will still work.

It is not possible to replace a softloaded version of CLib with a newer softloaded version, because applications using CLib set up a list of branches into the C library module when the program initialises. If a newer/different version of CLib is loaded afterwards, it is likely that this may be placed elsewhere in the RMA, so the positions held within the previously set up jump tables (any applications loaded with the previous CLib) may end up pointing to fragments or rubbish.

This requires having multiple copies of CLib available. For older versions of RISC OS:

• Start system normally.
• Load an older (but newer than ROM) version of CLib.
• Load something that uses C. I loaded OvationPro, but anything that uses C will do.
• Load the newest version of CLib that you have.
• RISC OS will promptly freeze. [it did with RedSquirrel and RISC OS 3.70]

For RISC OS 5, this is a little bit harder to do because there appears to be only one slightly older CLib in !System.Modules, and loading a module on top of itself won’t change anything. So here is a method to simulate loading a newer CLib in a different part of the RMA (which is what may well happen if the newer one is larger).

• Start system normally.
• *RMLoad System:Modules.CLib
• Load some software that uses CLib – such as SparkFS.
• Go to the command line (not taskwindow) and type:
  *RMKill SharedCLibrary ¹
  *RMLoad System:Modules.BasicEdit
*RMLoad System:Modules.CLib
• Press Return and the machine will freeze just after drawing the desktop grey background.

It is clearly not safe to arbitrarily mess around with CLib when clients have jump tables pointing into it.

Therefore, softload versions of CLib should set a flag, if any programs call LibInitXXXX (as applicable for the version of CLib – LibInit[Module]APCS_32 for RISC OS 5) then subsequently CLib should refuse to die. There is no easy mechanism to determine if any clients are still using CLib, so it should just refuse to die in all cases.

This means that it would not be possible to quit all CLib clients and then load a newer module, but since forgetting something could result in a stiffed machine, this was never a particularly safe practice.

I’m leaving this as Normal severity despite the symptoms being more akin to Major because everybody and their kitten ought to know not to softload CLib twice. I am just concerned with the possibility of something providing a newer version of CLib (say, as a package or via the installer) and the application concerned then trying to RMLoad it. This should fail. It wouldn’t. The results? Unpleasant.

¹ RMKilling CLib and then RMLoading it again at this point will show no effect as it will be loaded back to the same place, and state it held in the application, not in the module… The loading of ARMBE is in order to force CLib to end up somewhere else in RAM and invalidate all the jump tables. You might argue that this is contrived (yes, it is) but really we shouldn’t be able to kill CLib at all when there are programs using it.

It looks as if CLib has no workspace of its own. Maybe softload versions should just refuse to die?

By the way, on a Pi:

*RMLoad System:Modules.RTSupport
SWI &575C5 not known

Subsequently, the mouse pointer vanishes, but mouse activites are still active (pressing Menu will call up a menu, etc). Entering SYS "Hourglass_On" shows no hourglass, either.

This is referring to loading “RTSupport” v0.11 from the !System.500.Modules directory.

The problem? Looks like related to the above, RTSupport is allowing itself to be removed when it is in use, and somewhere between being removed and being reloaded, something is trying to call RT_ReadInfo (SWI &575C5). That something, I’d imagine, is tied in with the mouse pointer in some way.

I see your logic Rick. I wonder if there is another way to stop a softloaded CLIB being replaced. Say the module loading routine in the OS check if it is CLIB being loaded and if so then check if there is a softload CLIB already and erroring if so.

Say the module loading routine in the OS check if it is CLIB being loaded and checking if there is a softload already and erroring if so.

Better the module(s) do this themselves, then those that have clients linked into them in some way can specifically refuse, rather than relying upon the OS to take care of this.

For example – I can’t give command lines as I just turned my RISC OS machine off (I’m about to go rewatch Serial Experiments Lain if my phone can cope with .ogm files!). Anyway… load NetSurf. When that is running, try to RMKill the SharedULib module. It won’t let you. Not while NetSurf is running.
It will if you quit NetSurf; but we don’t have that luxury with CLib as it doesn’t record clients, only setting up the jump tables on request.

It is possible to replace the ROM version of CLib with a softloaded one – because the ROM version is still in ROM so direct links, such as within !Edit etc, will still work.

It is not possible to replace a softloaded version of CLib with a newer softloaded version, because applications using CLib set up a list of branches into the C library module when the program initialises.

It is for this reason that the boot sequence was changed 12 years ago to always load the latest SharedCLibrary from !System before anyone else gets a look in.

That effectively makes anything doing the RMEnsure/RMLoad sequence a NOP since the RMEnsure will do nothing, unless your boot sequence is over 12 years old, in which case you get all the freezes you deserve!

It’s not much different to the Toolbox modules being loaded (previously by UnplugTBox), they’re system resources that can only be changed early on before any clients are attached.

Now I understand why modules are so much easier than shared libraries.

Presumably, if a shared library were to be updated, it would be possible for the new one to install jumps from the old one to itself, assuming that the libraries do not hold state (which is what I’d expect for a shared library).

in which case you get all the freezes you deserve!

Interesting attitude. A library that has clients should not die. Period.

I’m not actually sure what OpenSSL NetSurf’s built against

Depends on who’s building and when they built it. Builds that come from us are currently using version 1.0.1k.

The new build of NetSurf for RISC OS with the very latest OpenSSL (released today) is now up.

There’s a lot of nonsense above about shared objects vs. modules accessed via SWIs. The CLib is basically a shared object using a custom format. C functions are not accessed via SWIs (which is why you can’t replace it twice). Proper ELF shared objects allow code to be shared between applications (why is using a DA an ‘interesting idea’, Rick?), multiple different versions installed and used at once, versioned symbols, better debuggability, and ease of porting libraries from elsewhere.

Using modules and SWIs basically lets you do none of that.

And I say again, all this already exists for RISC OS and works.

There’s a lot of nonsense above about shared objects vs. modules accessed via SWIs. The CLib is basically a shared object using a custom format. C functions are not accessed via SWIs (which is why you can’t replace it twice). Proper ELF shared objects allow code to be shared between applications (why is using a DA an ‘interesting idea’, Rick?), multiple different versions installed and used at once, versioned symbols, better debuggability, and ease of porting libraries from elsewhere.

Perhaps I’m missing something significant, but isn’t it true that the paragraph above is just as true if you delete the acronym ELF? i.e. ELF is NOT a requirement for proper shared objects.

Steve: No, ELF is not a requirement for shared objects, but it is the leading open standard and what is implemented on RISC OS. (It also happens to be what most UNIXes use, and most embedded systems use for pre-linking stages.)

Steve: No, ELF is not a requirement for shared objects, but it is the leading open standard and what is implemented on RISC OS.

Might I suggest that the following style is better phraseology:

Proper shared objects allow code to be shared between applications, multiple different versions installed and used at once, versioned symbols, better debuggability, etc
This already exists for RISC OS. The following are examples of projects already released using this technology:
list examples

The following are the pre-requisites for any project using this technology:
list requirements

The other implementations (not currently available to RISC OS) are:
list implementations

Presumably, if a shared library were to be updated, it would be possible for the new one to install jumps from the old one to itself.

It would make more sense for the version number to be a part of the library name, so if a newer version needed to be loaded, it could be loaded alongside the older one. Okay, it is wasteful and defeats a lot of the point of a shared library, but I don’t think that situation ought to arise often … unless the ELF system is as broken as Linux in that code compiled to use a specific version of the shared library must use that version and not a later one (often this is worked around by using a bunch of symlinks to minimise the number of actual versions of libraries required).
I really hope that in this instance “any later version” will suffice, with backwards compatibility meaning older software can use the newer library.

There’s a lot of nonsense above about shared objects vs. modules accessed via SWIs.

Since your last post (yesterday), I very specifically pointed out that CLib does not use SWIs, plus I also pointed out that it is quite happy working in USR mode.

Shared libraries and modules with SWIs are not the same thing. They are superficially similar in that additional functionality is provided, but the big difference is that SWIs extend the operating system in some manner while being extremely inefficient to use for standard library functions.

Of course, a good many library functions will call SWIs themselves, either directly (fread() → _sys_read() → _kernel_osgbpb()) or indirectly (have fun wading through what one call to printf() involves!), but that’s neither here nor there. RISC OS works by reacting to SWI calls. But we don’t necessarily have to add more SWI calls. ;-)

C functions are not accessed via SWIs (which is why you can’t replace it twice).

C functions are set up by writing a jump table of absolute addresses into the CLib module. That is why you can’t replace it twice.

(why is using a DA an ‘interesting idea’, Rick?)

Because I had previously considered dynamic areas to be places where data is stored, not places for executable code. This is probably related to older versions of RISC OS placing DAs above 64MiB in the memory map so they couldn’t be used for executable code due to the 26 bit PSR. I never really thought much about them otherwise…

multiple different versions installed and used at once

I take it this means it is linked by version? Is it not possible for a program wanting an older version to utilise a later version?

Perhaps I’m missing something significant, but isn’t it true that the paragraph above is just as true if you delete the acronym ELF? i.e. ELF is NOT a requirement for proper shared objects.

ELF is not a requirement for proper shared objects, but it is a requirement for the use of such with GCC and as described here.
In essence, we have two executable application formats available on RISC OS – the traditional AIF (which is old and could do with some extra functionality) and the Linux-derived ELF (which does more but is not native (so needs a ‘helper’ to get it to load)).

The problem is, while the shared library (quote) “already exists for RISC OS and works”, it does not as far as I can tell, work with the DDE…

Because I had previously considered dynamic areas to be places where data is stored, not places for executable code.

True, and to use it for code will really be a problem the day someone will decide to forbid code in DA for security/stability reasons.

I take it this means it is linked by version? Is it not possible for a program wanting an older version to utilise a later version?

No. And perhaps one day it will even break ‘à la windows’ (where dynamic libraries are not shared any more, but duplicated)

Funny :)

True, and to use it for code will really be a problem the day someone will decide to forbid code in DA for security/stability reasons.

And just think about the problems we will have once someone decides to disable execution of BASIC code for security/stability reasons.

The problem is, while the shared library (quote) “already exists for RISC OS and works”, it does not as far as I can tell, work with the DDE…

Why is this a problem? You also cannot use modern C++ with the DDE. Just use the tools that work in the way you want.

From another thread:

The same with ELF (see other thread): DDE is my compiler, so find other ways (heu, I mean, not you).

This makes little sense.

The DDE doesn’t support shared libraries at all (beyond the SCL, which is a separate issue), so if you wish to use the DDE you don’t have the option to use ELF – or anything else – at present.

GCC does support shared libraries, using the standard ELF format. The system works now, and is proven, so it’s hard to see why it’s a problem unless you’re adopting a “not invented here” approach to progressing the platform.

If a GCC-using developer only needs the standard functions found in the SCL, then GCC uses the SCL just fine: all my software is compiled like this.

The problem comes when more Posix-y stuff, not found in the SCL, is required (such as with NetSurf, or anything else building on code from other platforms). Then the only options are to use ELF-based shared libraries, or to statically link the whole lot into a huge binary (as currently happens with NetSurf). Developers doing that kind of work are probably already using GCC anyway.

The only question is what to do if shared libraries were added to Norcroft. Given that the working precedent is ELF, it’s hard to see what benefit there would be to re-inventing the wheel. Ideology?

 Just use the tools that work in the way you want.

I do. DDE does what I want.

But if I was to suggest shared libraries, then with the caveat of “this version or later” in the linking phase, it would make the most sense to adopt the ELF system. Why? Because it has already been done. Wouldn’t make sense to reinvent the same thing unless there was something fundamentally different that needed to be done.
Plus, it would mean we (DDE) could use their (GCC) libraries. What’s the alternative? Duping everything?

aaaa

Forgive this diversion from deeper matters.

BASIC code is script, so it has always been possible to run it in DAs, and also in RMA. Shared libraries for BASIC have been possible for a very long time using either my Basil module or home-grown single modules. That they have rarely been used owes a lot to the individualistic nature of most RO BASIC programmers, I think.

Back to the main discussion. ;-)