RiscLua update

Back in October 2015 RiscLua 6 came out, with the new facility for dynamic linking of C modules. That was because Lee Noar gave me a lot of help with GCC. Three years later RiscLua 7, and subsequently 8, reverted to static linking, because I could not figure out how to make GCC do what I wanted. Now Paolo Fabio Zaino has visited the RISC OS world and he has shown me what I needed to do. So RiscLua has been updated to version 86. It is functionally identical to the previous version 84 but has dynamic linking, so any programs using the riscos library may need an extra line

riscos = require "riscos"

Started to port libs already :)

Good to see RISC OS software updated to the latest version!
This will be included also in our Cloverleaf RISC OS distribution.

Thanks for the new release, but as you mention the change to dynamic linking I thought you may like know about a problem I’m seeing.

After installation I get the error:

File ‘SharedLibs:lib.ld-riscos/so/1’ not found

when I try to run any of the example files.

*SOMStatus shows ld-riscos/so/2 loaded so that may be the problem?

I’m running RISCOS 5.28 with Iris 1.010. Hope this isn’t just something weird about my setup – I’m used to ld, ldd, ldconfig etc on Linux but haven’t been using RISCOS for years until very recently so am not really clear how it all works here to try to problem-solve it.

Replying to myself: you probably knew this already, but I’ve realised that if I reboot and don’t load Iris then all works fine from the RiscLua perspective… but Iris won’t then load, giving the error “There are still clients using ARMEABISupport”. So, sorry, it looks like it’s not a RiscLua problem per se.

Iris uses ‘private shared libraries’ where they’re in shared library format, but aren’t actually shared. You’re not the first person to run into issues when mixing Iris and ‘normal’ apps.

While I’m not an authority on how it all works, I believe that the idea was to not ‘pollute’ SharedLibs with the various beta libraries that Iris uses. Once Iris comes out of beta, the libraries would all be considered stable and installed into SharedLibs as usual.

Hi Daniel,
yes this is a problem unveiled by Iris, however the specific problem you are mentioning is caused most likely by something not updating right in ARMEABISupport module (which leaves the client app mapped).

Let me explain:

If you load !RiscLua and run the examples just after booting (so Iris has not messed up yet with Shared Objects) and then you open a TaskWindow and run the following commands:

*SOMStatus

You will see that SOManager is perfectly aware that RiscLua has loaded (for example) rlua:solib.riscos/so

Now, the client for that SO gets mapped in another module called ARMEBIASupport. If one simply kills SOManager (like the SOMKill script does in Iris), then there are very little chances to be able to understand what’s going on.

Libraries loaded in SOManager do have an expire time and it can be set, but as far as I have seen it seems to set the expire time for libraries that will be loaded in the future(?). However, if a library expires (aka no client uses it for like 30 minutes with the default expire time) then the library is unloaded automatically by SOManager, BUT the client in ARMEABISupport is not cleaned up :(

However the mapping in ARMEABISupport seems to be happening only once (I ran multiple tests loading and unloading RiscLua and also waiting for SOManager to unload SOs), so it may be simply that SOManager gets mapped as a client in ARMEABISupport (because the address of the mapped object never changes, so in theory it should be always the same client), but I am guessing here.

Maybe I can try to play with ARMEABISupport_Cleanup or something, to see if it may help removing the registered client when Lua exits. But this is all I can think of. GCC Support for ARMEABISupport was added recently (from 4.7.4 release 4 IIRC), so it may also be something unrelated to RiscLua specifically (not sure).

If one is wondering why RiscLua is not trying to unload SOs loaded by scripts then there are multiple reasons for this:

• RiscLua is a (obviously) programming languages, hence there are tons of chances one will run multiple Lua scripts/programs. Given that SDs are quite slow, not unloading the so makes subsequents executions of Lua scripts that requires that so much faster. In the end this is why we have DLL and SO managers in all OS ;)
• Even if I wanted to add an unload then it would be complicated, in the sense that a Shared Object may load other shared libraries that we would not be aware of in Lua Runtime. So, there is no easy solution for this (btw there is no solution for this also in Lua on Linux, just for the record).

For what concern Iris:

Yes, there should be just one SharedLibs (and it should not be killed/removed), so it should be Iris to behave like all other apps and use only one SharedLibs (the official one). However, I agree with the Iris team that it is premature right now, given the beta state of the libraries used in Iris and also the fact that !Packman tend to replace the entire !SharedLibs, so in case of an update, if Iris would have installed its SOs in the official !SharedLibs then you would “lose” them (they would end up in a backup, but yes, no longer loadable).

And yes, if one doesn’t try to load Iris, then everything works fine, I have ran tests for like a day and RISC OS on a RPi400 appeared to be very stable and no problems happened.

Hope this help clarifying the situation.

I do not know whether this is relevant. But a running Lua thread that needs to unload a module has to remove its entry in the package.loaded table.

package.loaded[foo_lib] = nil -- no more foo

yes this is a problem unveiled by Iris, however the specific problem you

Technically it’s one of many problems caused by the way !Iris is distributed.

are mentioning is caused most likely by something not updating right in ARMEABISupport module (which leaves the client app mapped).

This is due to Iris needing a later version of the SOManager and EABISupport modules then is available via PackMan.

If either SOManager or EABISupport are in use they can’t be updated.

The solution to this problem is to produce a new release of ShareLibs that includes these new versions. (say SharedLibs release 6).

used in Iris and also the fact that !Packman tend to replace the entire !SharedLibs, so in case of an update, if Iris would have installed its SOs

If Iris used the real SharedLibs there wouldn’t be any issues as there is nothing currently using the armeabihf folder in SharedLIbs. PackMan will only back things up and replace files if it is trying to update a file it doesn’t know about. There won’t be any as there are no packages currently using armeabihf.

@ Gavin

I do not know whether this is relevant. But a running Lua thread that needs to unload a module has to remove its entry in the package.loaded table.

package.loaded[foo_lib] = nil — no more foo

If you try that with riscos (C Package) it should not unload the riscos/so module from SOManager. Even if it did, if riscos/so would have had dependencies on other /so libraries from SharedLibs: those wouldn’t be unloaded.

So the above syntax, AFAIR, works only for pure Lua libraries. Also for RiscLua libraries I would be cautious, because most likely it will not remove other RiscLua libraries loaded by the NILed library, in other words I am not sure it cleans all the dependency tables, JFYI.

Beside the above, the issue that Daniel is having is being caused by Iris that is trying to reload SOManager etc., which should not be killed and reloaded, so it’s not a RiscLua issue.

@ Chris

Technically it’s one of many problems caused by the way !Iris is distributed.

I agree, I made it clear in my previous post that Iris way of being distributed is causing this issue and more, but at the same time the community doesn’t wants to wait for Iris to be stable (many have complained about the lack of Iris being available). This falls into a discussion that also involves money and politics and I do not wish to start it all over again in this thread.

If either SOManager or EABISupport are in use they can’t be updated.

This, exactly this is the reason why I used the term “unveiled by Iris”. The thing is, it’s not just Iris that may need to update them, what about updating the SharedLibs: with a new release after an app has started to use it? Maybe we should introduce (like Apple does) an upgrade process that effectively happens at boot time and requires a reboot after having downloaded all the necessary bits?

Hopefully it’s more clear now.

The solution to this problem is to produce a new release of ShareLibs that includes these new versions. (say SharedLibs release 6).

Exactly and a relatively easy update process and also make sure that SharedLibs updates what it has to and not “move everything” in a backup directory and replace it.

PackMan will only back things up and replace files if it is trying to update a file it doesn’t know about. There won’t be any as there are no packages currently using armeabihf.

Packman will cause problems when updating SharedLibs, I have experienced myself and reported it to the developer which recognised that SharedLibs update process in PackMan is not as good as it should be. For instance, if one deploys a new SO in SharedLibs and then SharedLibs has an update via Packman = boom.

With that said, again, if one doesn’t use Iris, RiscLua is totally fine, also Python works fine and so are other browsers using the regular SharedLibs.

Might be a really dumb observation, but why not take the updated modules from Iris and drop them into the regular SharedLibs, then when the machine is next started it will have the newest bits available.

effectively happens at boot time and requires a reboot after having downloaded all the necessary bits?

Don’t need to bother with such complications. The reason Windows (and possibly Apple) need to do the boot update is because resource files are held open so cannot be changed “when loaded”.
With RISC OS, modules are loaded entirely into memory, so it’s easy to replace the ones on disc, and then require a reboot in order to get them loaded.

Until then, RMEnsure the necessary version, and fail with a “please update” message.
This is what we should all be doing with CLib, instead of trying to blindly RMEnsure it, which I’m sure you can understand isn’t the greatest idea. ;)

@ Rick

Or even simpler: Keep repeating over and over that Iris is not even close to a production release yet and, as such, should be treated as “for beta testing purposes only”. This BTW is also made clear by ROD.

I would also recommend to the Iris team to add an error in their !Run that clearly tells the user to reboot and why. And to use Iris just for beta testing right now, because it also messes up with other apps, not just RiscLua.

I would also recommend to the Iris team to add an error in their !Run that clearly tells the user to reboot and why. And to use Iris just for beta testing right now, because it also messes up with other apps, not just RiscLua.

Iris works perfectly fine if everything it needs is merged into the real SharedLibs and it’s embedded one is deleted. Also needs a couple entries in it’s Run file commenting out. It then doesn’t mess up other apps.

Keep repeating over and over

Good luck.

Since pretty much everything is treated as beta software these days, it seems like some don’t quite understand what the concept actually means. ;)

Since pretty much everything is treated as beta software these days, it seems like some don’t quite understand what the concept actually means. ;)

LOL XD

Iris works perfectly fine if everything it needs is merged into the real SharedLibs and it’s embedded one is deleted.

The last famous words Chris… mine just got stuck on a website loool had to close and re-open it, so no Iris is still in beta and I am sure it will be a great browser for RISC OS when it’s stable enough and yes when the SO are merged into SharedLib! :)

mine just got stuck on a website loool had to close and re-open it

Well, it is based upon WebKit. Have you ever used Safari on an iPad? It’s a pretty accurate recreation. :-p

Iris works perfectly fine if everything it needs is merged into the real SharedLibs and it’s embedded one is deleted.

The last famous words Chris… mine just got stuck on a website loool had to close and re-open it, so no Iris is still in beta and I am sure it will be a great browser for RISC OS when it’s stable enough and yes when the SO are merged into SharedLib! :)

Looks you can all agree that Iris works just as it normally does¹ when the SharedLibs items are merged into the normal system boot and the hacks are removed from the Iris !Run file.

¹ Like any other beta software – with a few glitches.
Everyone is reporting the circumstances of their glitches, right?

Everyone is reporting the circumstances of their glitches, right?

Mines seems related to the JS engine and how it executes complex sites, Andrew is aware of the limitations. This has very little to do with /so in dedicated SharedLibs, most likely still something to do with some bug in such libs or the way they put together Iris.

One odd thing I have noticed is that, on another test, after transferring an entire SD content from a 32GB card to a 128GB card Iris on that one no longer start up. But I had no time to check what’s happening there. However the only app having issue on that SD is Iris (they only thing changed on that SD is the daily image of RO 5.29, so most likely it’s related to that).

Anyway these are the issues I am having with Iris and, for the record, the issues with some complex JS website also happens on with an SD that has only RO 5.29, !Boot and Iris (no regular SharedLibs or anything else).

Thanks for all the explanations! :-)