!Apps autoupdate and more

@ Theo

In honour of bikesheds everywhere, I’d suggest renaming the commands to PkgInstall, PkgList, Pkg… as it’s nicer to type ‘PkgInstall fooapp’ rather than ‘PkgModInstall fooapp’.

Sure, that’s easy to do.

You could look at the kind of ‘verbs’ that different package managers use, as I think there is quite a bit of commonality between them.

I already did. Basically, the whole DME project is about “importing” elements from Linux/macOS/Windows where they are a) feasible, b) useful, c) RISC OS “compatible” (see !Launchpad features for examples, oh and just for the record: I did !Launchpad UI first, before Microsoft would “copy” it on Windows 11! Hahah :D )

However, in this case let’s not forget that:

Debian (and derivatives): apt install/dist-upgrade/full-upgrade/update…

Fedora (and derivatives): yum|dnf install/update/upgrade/list/search/check-update…

Windows Package Manager: winget install/update/search/list/upgrade/uninstall…

Point is, the “standard language” in this matter, is organised so that, most likely, the “typical” RISC OSy form will have to change drammatically. In other words, if I need to follow the “defacto” standar verbiage, then it should become something like:

PkgMod install/update/search/list/upgrade/uninstall… etc.

(or just Pkg, obviously)

While, if we would really go full on RISC OS style, then it should probably behave more like a FileSystem or maybe like MOS Utilities (aka PkgInstall, PkgUpdate, PkgList, PkgUninstall etc.).

I am also pondering a PkgEnsure (so, Apps can ensure to ask the user to pull dependecies in case they have been removed or are outdated), but this is definitely a stretch goal.

Plan is, when I have something demoable, then probably ping ROOL folks to ensure choices are made to blend in with the rest of the OS etc. and then put up an official forum page with feedback request from users.

For now I am busy enough to get it to work properly XD

I don’t necessarily think we need a full blown auto-updater with a registry list that needs to keep track of all the inns and outs of all the programmes on the computer. This is probably overkill for the RISC OS way of doing things. It’s something RISC OS has pretty successfully avoided for other tasks perhaps it’s just me but I can’t see why this should be the exception.
Just stick a command in the application’s !Boot or !Run file to trigger the installer to check in background. While this means applications that are never used, never get updated. I can’t think of a situation where this would be a problem outside of the !Boot directory. A tighter more managed approach would be needed within the !Boot directory.
RISC OS doesn’t have the complected web of dependencies that many other operating systems have and I don’t see this changing to such an extent that this would become a problem.
To cover for any other situations where an install needs to be manually triggered for an application you add a check for updates option to the filer menu.

To cover for any other situations where an install needs to be manually triggered for an application you add a check for updates option to the filer menu.

When you create an empty application (i.e. an empty directory with a name starting with !) the filer provides a Help option in its filer-menu, which is greyed out. If you put a file called !Help inside it, the option remains greyed out. But if you put a file called !RunImage inside it, the Help option comes to life in the filer-menu. This is why you are likely to see empty !RunImage files sometimes. A similar mechanism might perhaps be supplied for an Update option that runs an Obey file called, say, !Update inside the application.

If you put a file called !Help inside it, the option remains greyed out. But if you put a file called !RunImage inside it, the Help option comes to life in the filer-menu.

Surely calling the executable !RunImage is a convention rather than something mandated?

Whether or not the Help entry lights up should depend upon the presence of the help file, nothing else (or if one must verify that it’s a legit app, look for the !Run file that is necessary).

Edit: The Style Guide suggests that the name is required (p115), then backtracks by saying that not all applications require all resources.
Certainly, if one had an app that was a module, the nomenclature would be eccentric.

Surely calling the executable !RunImage is a convention rather than something mandated?

No, the string !RunImage is a literal constant in the Filer module. At least it is in version 2.43 29 May 2016 at offset &6368.

Ditto RISC OS 3 (looking at the binary).

Seems that there is something odd going on with the date stamps too, does an app directory get a fake datestamp that matches the !RunImage ?

does an app directory get a fake datestamp that matches the !RunImage ?

Yup. If !RunImage has load and execute addresses, the resulting datestamp won’t be all that believable :-)

Surely calling the executable !RunImage is a convention rather than something mandated?

I think so too. You only need to call it !RunImage if you want the application directory datestamp to be fudged as that of the !RunImage file.

Just stick a command in the application’s !Boot or !Run file to trigger the installer to check in background.

Sure, the local file inside an app can work for this, in general one of the issue you’d have with completely custom solutions is, when someone stops maintaining an app and someone else wants to take over, then with completely custom solutions, they’d have to also adapt the update infrastructure.

Some folks may make it easy, others may not. So having a standardised approach would make life easier for everyone. Also having a pre-built API means less work for the single programmer making an app (all I hear in this forum is how little time people have to do things on RISC OS, so…)

RiscPkg format has a centralised “registry”, but that also offers other benefits, like caching etc. (again nothing that is not replicable in other ways, the only true advantage is that it exists and has been tested for years).

A new format will need testing and time to become stable and deal will all details (and we all know the Devil is always into the details). But, again a local descriptor file would be a nice start.

I like the decentralised approach also because of the terrible state RISC OS’s FS is in for 2023, it’s way to easy to corrupt directories and files, so a centralised registry is a potential big risk factor.

I think so too. You only need to call it !RunImage if you want the application directory datestamp to be fudged as that of the !RunImage file.

+1 , despite the style guide, what I have noticed is that the only true required file for an !App is the !Run file (so !Boot, !Help and !RunImage are all optional or “desireable”, but not necessarly a must have).

Perhaps the Filer module should be rewritten so that the Help-option only comes to life in the filer-menu when there is a file called !Help in the application? I note that the filer only scans the application once, so that if you later change or delete objects in the application it is fooled into continuing to show a Help-option.

Perhaps the Filer module should be rewritten so that the Help-option only comes to life in the filer-menu when there is a file called !Help in the application?

IMHO, no. However, in both cases, I don’t have a strong optinion TBH.

I note that the filer only scans the application once, so that if you later change or delete objects in the application it is fooled into continuing to show a Help-option.

This is true, but that happens because the implementation of the Filer doesn’t react to the specific new object being added to an !App directory, which, IMHO, is not very consistent with the desire of having independed units (aka Apps) without a centralised control.

IMHO, an App should be a treated as a selfcontained object and so, if an !Help file is added (or even if a file inside the !App level is renamed with !Help), the Filer should update the state for the !App.
IMHO, failing to consider (and mostly treat) an !App a such becomes confusing (or at least, express an inconsistent behaviour), but (again) this is just my personal opinion.

Some folks may make it easy, others may not. So having a standardised approach would make life easier for everyone. Also having a pre-built API means less work for the single programmer making an app (all I hear in this forum is how little time people have to do things on RISC OS, so)

I agree. We are at a point where we need to integrate an update/installation manager and a standard official approach is essential. Basing it on PackMan would seem the obvious choice. I am concerned relying on the installer to manage updates is likely to leave applications outdated. Moving applications around or not using the installer to install them are very common issues that currently PackMan does not cope well with. I have just tidied my hard disc!
Adding an extra line to a !Run file to trigger an update check would be simple to do and less hassle than getting PackMan to search out and log lost applications.

Basing it on PackMan would seem the obvious choice.

Yup, that’s why I started to create the PkgMod out of LibPkg (RiscPkg library that is also used by PackMan)

I am concerned relying on the installer to manage updates is likely to leave applications outdated.

So, in the end there will be a set of tools, but they really are only Front Ends. LibPkg is what does the job. So, !PackMan, PkgMod (the installer is going to be just a Windows style installer that is nothing more than a Front End to PkgMod + a scripting language to customise the installation process and !FirstBoot which is, again, nothing more than a Front End as well).

Nothing is stopping an installed App from running a request to PkgMod in the !Run file, to check for an update. But I am not sure yet if this is actually a good idea, I mean, what if a site is unresponsive? That would freeze the load for a while, so probably the best approach would be to add an option in an App to check for updates (which would then be a request to PkgMod or something).

BTW, by using this approach, when a user will run !PackMan, it should also be able to update all the installed Apps. In the end the !Packages “DB” is goign to be the same, so they’ll all work together.

The only downside of this approach, according to some comment, is that, given that they’ll all use !Packages, it means it’s effectively a centralised approach. So, not sure how many will like it.

Nothing is stopping an installed App from running a request to PkgMod in the !Run file, to check for an update. But I am not sure yet if this is actually a good idea, I mean, what if a site is unresponsive? That would freeze the load for a while, so probably the best approach would be to add an option in an App to check for updates (which would then be a request to PkgMod or something).

I think this can be largely mitigated by providing the user and App developer with a large selection of configurable options. Trigger update check when an app is run, when an app is quit, when the machine is shut down, or just log request for later manual triggering etc.
I suspect for most apps adding in app options would not be necessary. Instead it could be set up so the user can select what happens with all updates using global user configure settings.

The only downside of this approach, according to some comment, is that, given that they’ll all use !Packages, it means it’s effectively a centralised approach. So, not sure how many will like it.

An alternative for developers who don’t want PackMan to host it, would be to allow them to issue a update notification message through PackMan.
Then users can be notified of an update and download it manually from the appropriate site. This would hopefully encourage more developers to embrace PackMan or at the least make sure users get there updates.

An alternative for developers who don’t want PackMan to host it, would be to allow them to issue a update notification message through PackMan.

That is how it already works. Each developer can have their own repository on-line (on their own website). My PkgMod and !PackMan just need a pointer to their own repository. That is how it’s done already. A RiscPkg repository it’s very easy to implement on every server. I am even implementing one directly in github.com.

Then users can be notified of an update and download it manually from the appropriate site.

This sounds very comvoluted, I can already see users asking: “Why can’t this thing be updated automatically?”. Many have already reported this problem asking how hard would it be for developer X or Y to add “PackMan feature” which is the way many users consider the whole thing. While !PackMan it’s only a front end for the RiscPkg format and uses the library mentioned multiple times in this thread.

Adding an option in an App menu that state: “Check for update” and when the user click on that option make a request to PkgMod is very simple. The only true issue with in-App updates is that the App directory will be overwritten by an update and this (depending on how the App itself has been designed) could be a problem. Hence having an external update tool is always the best way, but again, if an App is written not to have internal files opened and read config only at startup etc. then in-App update it’s an easy thing to do.

Hope this makes sense.

Adding an option in an App menu that state: “Check for update”

On all my Ovation Pro applets I have a button in the info window to check for updates. It checks the users current version against the latest version number available on my website and presents a link in a browser if there’s a newer version for them to download.

At this stage it does auto-update the applet – the user has to do that manually.

That is how its done already. A RiscPkg repository its very easy to implement on every server. I am even implementing one directly in github.com.

Sorry I don’t think I described what I meant well enough. I was thinking about developers who didn’t want to use a repository at all, still being able to send an update availability notification.

This sounds very comvoluted, I can already see users asking: ?Why can’t this thing be updated automatically??.

I would totally agree with them. It’s about chipping away at the reasons not to use the system. The more Ÿexibility the more likely the system will align with a developers needs wants and desires!

Adding an option in an App menu that state: ?Check for update? and when the user click on that option make a request to PkgMod is very simple.

For the majority of apps it should be. There are situations I feel it would probably simpler just to add something to !Run though.
If it’s an old single tasking programme.
If a developer has lost the source code or has no plans to update an app but would just like to make it auto updatable in case the situation changes.
The original developer has long since vanished but someone would like to make it available and updatable through PackMan in case the situation changes.

The only true issue with in-App updates is that the App directory will be overwritten by an update and this (depending on how the App itself has been designed) could be a problem.

When I was developing my installer I came up with all sorts of weird and overly complex ways of dealing with this. In the end I went back to including an update file in the app that lists the history of all the files and directories in it, what to do with them, any movements deletions additions etc. If the list in the new version and the old version match and agree with what is actually there, allowing for any version specific change. Then the update is OK to go ahead otherwise it fails. Creating an app to largely automate the process of managing the list files would be straightforward too.

all sorts of weird and overly complex ways of dealing with this

<looks directly at ChatCube>

That software has a fancy upgrader, that (the last two times I tried) not only had zero fault tolerance but would throw in the towel over really trivial things ¹ having already removed most of the older application files, thus leaving a completely broken mess in its wake.
(which needed to be “fixed” by finding where the archive got downloaded to and copying everything by hand)

Happened twice. I gave up after that.

¹ As far as I could work out from the log, it was trying to delete ² some little avatar graphic or somesuch that didn’t exist. Does it get the file list from the new archive rather than looking to see what is actually present? That shouldn’t be a critical error in any case…

² And is it really necessary to delete everything before replacing it? Takes time and adds loads more catalogue writes (which matters on flash media).

That software has a fancy upgrader, that (the last two times I tried) not only had zero fault tolerance

Yeah, saw that too. I have the impression that CL has developers used to code on Linux where, for example, an inode doesn’t get updated until all related open-files are closed, so that type of update works fine on Linux, but won’t work on RISC OS especially if they also use things like local Fonts (that stay open durign use and will get overwritten during an update that contains the same files) etc.

But oh well…

The way I am trying to handle the update process is to have:

1) a switch (“-force”) that basically tells PkgMod “Look, I am the developer of this App and I take full responsabilities for the update, so just do it!”
2) and a switch (“-restart” which takes a path to the App) which kills the App before starting the update and then restart it , you’ve probably seen it already working with !Launchpad, it self-restart after every updates and also ensure to save user last changes before restart and migrate the DB (if needed) before the new version reads it during the restart.

That software has a fancy upgrader, that (the last two times I tried) not only had zero fault tolerance.

Yes. The utter mess that I have seen from some upgraders baffles me. Upgraders often seem to get created quickly at the end of a project with little in the way of planing. An upgrader must be as bomb proof as possible while giving the user sensible options if something does go wrong.

1) a switch (“-force”) that basically tells PkgMod “Look, I am the developer of this App and I take full responsibilities for the update, so just do it!”

Do you have any plans to provide anything to work around configuration files etc? Many programmes out there still do not use ‘Choices’.

Upgraders often seem to get created quickly at the end of a project with little in the way of planing. An upgrader must be as bomb proof as possible while giving the user sensible options if something does go wrong.

The Mac version of MS Teams will fail to update with the absurdly unhelpful “something went wrong” if you’ve had the gall to rename its icon.

anything to work around configuration files etc? Many programmes out there still do not use ‘Choices’

That shouldn’t be the upgrader’s responsibility. An app should not ship with configuration files, and should be able to cope with them not being present.

But, alas, in the real world…..
(still have not had anybody send me the config file for the NetSurf theme creator, it crashes on startup without it)

The utter mess that I have seen from some upgraders baffles me. Upgraders often seem to get created quickly at the end of a project with little in the way of planing. An upgrader must be as bomb proof as possible while giving the user sensible options if something does go wrong.

Which is a good reason to provide that one bomb-proof upgrading mechanism in the OS for all those poor application developers to use instead of being forced of reinventing the wheel one more time.

Do you have any plans to provide anything to work around configuration files etc? Many programmes out there still do not use ‘Choices’.

If there are programs out there that are regularly upgraded AND do not store config files in Choices, please contact the author.

Waiting for someone to contact me wrt CDBurn/CDVDBurn and its homebrew updater application and its insistence of storing the user settings inside the application…

still have not had anybody send me the config file for the NetSurf theme creator, it crashes on startup without it

What config file? I have NSTheme 0.02 (21 Mar 2005) which appears to run fine on RO5.28, with no sign of a config file either within itself or $.!Boot.Choices. However, I note an oddity. There are three identical copies of the subdirectory CVS: !NSTheme.CVS, !NSTheme.Resources.CVS and !NSTheme.Resources.en.CVS each containing files called Entries, Repository and Root. It looks like somebody has been having second and third thoughts about where things should go. I have not bothered to test what happens if any or all copies are deleted – nothing, I expect.