!Apps autoupdate and more

This thread is from here:

https://www.riscosopen.org/forum/forums/1/topics/17696?page=2#posts-142418

The whole messages subthread replicated here:

Rick said:

I’m surprised there was never a mechanism defined for apps to check for updates themselves . . . as is commonly done elsewhere.

My Manga, and some of my stuff for Windows, would connect to my server and fetch a tiny control file that tells what the latest available version is. If later, the user would be notified.

Paolo said:

I’m surprised there was never a mechanism defined for apps to check for updates themselves . . . as is commonly done elsewhere.

Same thoughts, never understood why Acorn or who followed never really defined a “standardized” way to have an easy way to update Apps beside having to scan the entire internet XD

And if we won’t add more code to write to a developer, all it required was adding a new text file in each App, even in INI format stating:

[app]
url=<where-is-located>
name=<zip-file-name>
release=<installed-release-number>
md5=<md5>

Where md5 is optional.

We could have called it !Info and add it insde the !App directory. Either the App itself OR an OS based tool could have used it to check for updates.

There are (obviously) other ways too, but this one is uber easy (and we already have INI file libs for BBC BASIC, C etc. and if a language is missing it, it’s trivial to implement).

Sometimes (Darth Vader Meme): “I find RISC OS lack of pragmatism, disturbing!” XD

However, in RISC OS’s defence, actually Mac OS X (until Apple Store) had the exact same problem and people had to develop 3rd party tools to solve it (such tools where using the App metadata, hence they did add something like the above to macOS Apps).

Steve Pampling replied:

We could have called it !Info and add it insde the !App directory.

Creating a set of such files to drop onto existing applications is, perhaps, something that a non-coder could do

Either the App itself OR an OS based tool could have used it to check for updates.

IIRC there’s a tool out there that scans a directory /disk checking for Help files in applications/tools which should give a clue or two how the check could be done if no one comes up with a better idea.

Edit: After a quick dig, I’m conflating two items – HelpScan (which doesn’t check files/directories, but rather checks variables) and DateSpy which checks directories, and subdirectories, for modules and reads their date info.

Edit2: “VerInfo” is probably a less likely clash with existing files

Gavin Wraith added:

Scanning a disk can take a long time. This little skeleton program, for example

-- Scan current disk for a file
local target = "!info" -- nb. lower case
local herepat, join = "^[^$]*%$", "%s.%s"
local root = (arg[0]):match (herepat)
local dir in require "riscos"
local clock in os
local N = 0
local found = { }
local search
search = \ (d)
  for leaf, ft in dir (d) do
    local this = join:format (d, leaf)
    N + = 1
    if ft == 0x1000 or ft == 0x2000 then
      search (this)
    elseif leaf:lower ( ) == target then
      found[1 + #found] = this
    end -- if
  end -- for
end -- function
-- main
print ("searching ", root)
local now = clock ( )
search (root)
print (#found, " items out of ", N, " in ",
       clock ( ) - now, " seconds")
print "------- finished"

gave

searching       SCSI::HardDisc4.$
80       items out of   240983   in     117.94   seconds
------- finished

Paolo replied to Steve:

“VerInfo” is probably a less likely clash with existing files

Another way would be, fo reach app exposing the file to create an `App$Info` variable, which would also speed up the scanning (scan vars instead of the entire disc and only check those that have the App$Info variable set).

Rick added:

Same thoughts, never understood why Acorn or who followed never really defined a “standardized” way to have an easy way to update Apps beside having to scan the entire internet

To be fair, “the internet” as a concept was quite new in the Acorn days. If you recall, there was precious little support for it in Windows 95, and Acorn gave up the ghost in 1998 in the height of the madness of The Browser Wars. Hell, back in those days one could still use ArcWeb and Webite to browse the various websites.

Following Acorn’s demise was a brief shocked hiccup, followed by… some unpleasantness.

Besides, there were sources like Poppy’s site, the Acorn Cybervillage, Drobe, and various others that I’ve forgotten. Oh, and anything of use I’d have dropped on Arcade. ;)

And if we won’t add more code to write to a developer,

More stuff in the RMA running in SVC mode, whee!

even in INI format stating:

My method did require some additional code (well, I lie, I wrote the code once and I just include it when it’s required 1). It fires off a fetch to some-url-here/path/appname/update.txt and it gets a file that contains a file version number, the reference to the update file, and a textual information string saying what’s changed. Here’s an example: https://heyrick.eu/software/verodes/update.txt

The program would check its version with that in the file, and if the file is newer, prompt if the user wants to update using the description as a reason why they might want to.

There was no MD5, but the file paths were always treated as relative, the system just won’t allow a reference to a file on another website (the Windows incarnation was stricter and wouldn’t allow outside of where the update.txt file was fetched from; this was relaxed within RISC OS because it was a load of code I couldn’t be bothered writing ;) ).
Plus, if the update file can be tampered with, so to can the MD5 in the description, so it didn’t seem to be worth bothering with. MD5 is only useful if it comes from a different source to the file (which makes sense if you download something via torrent or a third party mirror).

an OS based tool could have used it to check for updates.

The problem with that is that it requires the user to manually check. Wouldn’t it be better if the app itself could take a quick check when it starts, and notify you if there’s an update?

The other problem is that it can’t really work as a SWI call or *command because of the async nature of fetching stuff. So it really needs programmer involvement. That being said, no reason most of it can’t be stuffed into a library…

Scanning a disk can take a long time.

Yup. And it looks like your skeleton doesn’t actually open any files it finds either, which will take longer.

That sort of thing probably ought to be pushed to the background. Look at a couple of files, poll, look at a few more, poll, repeat until done.

¹ Except Manga, that was written anew because, well, it wasn’t Windows and it had some customisations due to being an optional beta-user thing. If you’re at all interested, it’s check_for_updates() towards the end of c.wrapper.

Paolo added:

Scanning a disk can take a long time. This little skeleton program, for example

True, but don’t forget that for update checks, that takes even more time (you need to query a webserver that may be down or disapeared and wait for a time out), or you may not have internet set or a very slow one.

Beside the above, one way !Launchpad can scan for things is “in background” (aka during a NULL event), so that will be completely transparet to a user, but again, users can disable it and scanning for variables is still better.

However, scanning for variables has the side effect that, if something hasn’t been seen by the filer yet, it won’t be checked.

So, as usual it’s a question of what would be the best approach.

And Gavin Wraith replied:

scanning for variables is still better

It is certainly more efficient than scanning the disk. Another possibility (did I miss it or has it been buried under a shower of aspersions?) is a registry of some sort, that is updated at installation. But that knocks on the head the ease of moving applications around afforded by the Obey$Dir mechanism – a central, feature of RISC OS, currently insecure (though Paolo may have a remedy for that). Registries fit when there is a distinction between the installer and the user of an application, a social division alien to the spirit of RISC OS, IMHO.

@ Gavin

Another possibility (did I miss it or has it been buried under a shower of aspersions?) is a registry of some sort, that is updated at installation.

That’s what !PackMan uses, (if you have !PackMan installed) you’ll find yours in:

!Boot.Resources.!Packages <----

But that knocks on the head the ease of moving applications around afforded by the Obey$Dir mechanism – a central, feature of RISC OS, currently insecure (though Paolo may have a remedy for that).

True. !PackMan uses a centralized mechanism to move Apps from one location to another, so yes, it’s no longer as simple as regular RISC OS.

However, it doesn’t needs to be this convoluted:

Having a Module that handles the registry one can provide a simple command that updates the Package position when the Package !Boot is read by the Filer, something like:

*Registry_UpdateLocation <PackageName> <Obey$Dir>

If this is executed by the !App !Boot and !Run, you’re good with the Registry AND moving Apps around RISC OS style.

Having a Module that handles the registry one can provide a simple command that updates the Package position when the Package !Boot is read by the Filer

This could get messy if the user has several older versions of an application lying about and does not boot the latest version when the machine starts. Then whichever has been “seen” first could become the registered location.

This could get messy if the user has several older versions of an application lying about and does not boot the latest version when the machine starts. Then whichever has been “seen” first could become the registered location.

True, any central registry can have issues.

However, in that specific use-case, it would be messy anyway with an app that tries to autoupdate: for instance, the user runs the outdated app and it will try to update itself.

So, if the goal is to keep older versions around, then one should make sure to comment out that line in the !Boot file, in which case this becomes easier to handle than having all the older version constantly trying to update (and not way to stop this).

Given that it’s not a must have, it’s a choice of the App developer and still allow the user to disable it, I think it works in both contexts.

No?

So, another way could be:

In !Boot

| First registration (if already registered, do nothing)
*PkgSystem_RegisterIfNew -app AppName -info <Obey$Dir>.!Info -path <Obey$Dir>

| If AppName (and all info) match the registry, it can update the path if it's different (given the! Info has an md5, you can have multiple apps with the same name registered and maintained separately)

In the !Run

| Check for updates:
*PkgSystem_CheckForUpdate -app AppName -info <Obey$Dir>.!Info -path <Obey$Dir>

| If there is an update it can do it and update the specified entry (aka replace it with the new one)

But again, every single solution will not be absolutely perfect for all possible use cases. Jay Miner always said: Engineering is the art of compromise. So, IMHO, it all boils down to what works on most use cases.

HTH

This thread inspired me to add a simple update mechanism to one of my programs, by means of an update button in the Info window. It checks for a new version and will load it automatically if there is one. The heavy burden however, is that I now shall have to write new versions.

The heavy burden however, is that I now shall have to write new versions.

XD

I don’t have strong opinions on any of the ideas here, but just to note that the core of the packaging system that PackMan uses is a library called LibPkg . PackMan is one example of a frontend for the library (there used to be another frontend called !RiscPkg, now abandoned).

So if somebody wanted to write another way to manage the existing package DB but with a different interface from the PackMan frontend, then that would be feasible by making your own app that uses the LibPkg API. Hence a putative ‘PkgSystem_CheckForUpdate’ could cooperate with the system that already exists, if that was desirable.

Hey Theo,

but just to note that the core of the packaging system that PackMan uses is a library called LibPkg

Thanks, I already started to have a look at LibPkg. I am perfectly happy with !PacMan, however, IMHO, we do need also CLI based version of it (for many reasons). So, I started to work on building my own “CLI !PackMan”.

So far, my most limiting factor has been the very little free time I have for RO, so not sure if and when it’ll be available, but I do have perfectly building module written in GCC C++ that is using LibPkg and that is awesome :) – My “ideal” plan is to add it as a component of my DME, so, as soon as “testable” it will just appear on !PackMan and in the DME core.

If the PkgSystem would be finished, indeed could be used as you mention and from any App.

I also wish to be able to build it for old systems (like the Archimedes as well, I know this may sound heresy on here, but many have asked if it was possible to have !PackMan like on the old HW, so if it would be possible it would be a great plus).

Hence a putative ‘PkgSystem_CheckForUpdate’ could cooperate with the system that already exists, if that was desirable.

IMHO, it’s very desireable and it would be awesome if we could build a package manager that runs on all systems and RO releases and, again, I am totally happy with the RiscPkg format.

Obviously others can have different opinions on the matter.

This is where I am at (so far):

- The module is already fully linked to !LibPkg
- I am now ensuring it sees the same DB as !PackMan and can do very basic functionalities.
- It will also be used by an Installer program that whoever is interested can use to distribute their own software with (the installer is easy scriptable somewhat remind of FrontEnd) and have a semi-automated local installation (old school, Windows/macOS style),
- Once software is installed (either manually, via !PackMan or via the embedded/local installer) the user (provided a source url has been added) can either upade it via !PackMan or manually or the developer could offer the facility to have an auto-update (I guess this would make everyone happy).
- The hope is to make it possible to use it also to upgrade the OS at some point.
- Don’t mind the naming conventions right now, they are temporary anyway.

That looks promising. I agree having a command line tool makes a number of operations a lot easier, especially ‘just install this long list of packages please’. It’s also handy when building software, because you can just tell somebody ‘pkg install libx liby libz’ to get the dependencies and they all arrive in one go, rather than a lot of clicking.

(One thing that annoys me about setting up a new Windows machine is the amount of clicking you need to do to work installers. I just want to say ’here’s a list of software, go ahead and install it for me please, I’ll come back in half an hour’. Yes I know there’s now WinGet and various enterprisey things, which rather proves the point).

Having it as plumbing so that apps can update themselves, without having to go through the PackMan UI but still keeping them in sync with it, sounds like an interesting idea. One nice thing is that the repository format is easy to create, so if you want to make a personal repository, eg for testing, or to distribute updates on DVD or flash media if you wanted, then that’s possible too.

I think it might be challenging to fit into a 4MB machine given it’s C++, but maybe… And I agree that provisioning such machines (and emulators) with modern software is annoying, and being able to just pull down a list of software would be handy. I suspect some of it won’t run due to memory limitations, but maybe you just need a curated feed of packages for that.

One thing to bear in mind is that LibPkg and PackMan have co-evolved so, when you get to the point of having something stable, it’s worth thinking about how to keep them built so they don’t get out of sync with different LibPkg versions (eg if the DB format changes or similar).

is the amount of clicking you need to do to work installers

Well, you have to accept the terms.
Then you have to pick where you want to install it.
Then choose whether it’s to be added to the quick launch thing.
And maybe whether you want a full or custom install.
And then click to not install some shovelware crap that will mess with your browser.
Then there’s a button to click to finally begin the installation.

And when that’s done, a button to click to dismiss the installer. You know, once you’ve given it a pat on the back for not choking on (some random completely irrelevant thing).

But don’t forget before you do that, there’s a tick box to autorun the program when you quit the installer, plus maybe another to open the release notes in Notepad. And maybe yet another to pop up the manual.

Some installers have a silent mode where a single command will perform a complete default installation with no clicking, but it will often do so by answering Yes to everything, so if there’s crapware included…

I think it might be challenging to fit into a 4MB machine

I don’t think progress ought to risk being held back due to the whimsy of supporting a thirty two year old machine that is quite restricted compared to modern machines (in both hardware and firmware).

Don’t forget the installers that say “Click ‘Finish’ to continue starting the install”! Are we starting, continuing or finishing? Make up your mind!

@Rick, you’ve forgotten clicking the authorisation to install it with admin packages. And if it’s not very common you might have to convince your anti-virus program that it’s OK — Norton regards the current Ovation Pro for Windows installer as suffering from "WS.Reputation1”. Not that Macs are any easier nowadays, unless you’re installing from the AppStore.

Norton regards the current Ovation Pro for Windows installer as suffering from "WS.Reputation1”.

That’s not a real virus or infection.
That’s Norton saying “not many people have asked for this to be scanned so we feel the need to scare you by saying it has a low reputation”.

Crowd sourcing is what gave us Boaty McBoatface, so maybe Norton should be using heuristics (does anything in here do stuff a virus might do) rather than trying to crowdsource it, because that absolutely doesn’t mean it is safe, it just means lots of people have (or in this case have not) tried to run this thing.

A quick Google suggests that Norton, rather than prompting the user for what to do, will treat a low reputation the same as an actual virus detection – namely tossing the file into quarantine and deleting the original.

Is this your experience? Is Norton really that broken?

I have Avast on my PC, but since it’s an old XP box, the “get an extra year” thing goes into an endless loop; but I’m not that worried as the machine is router firewall blocked from external sites; I don’t trust Norton, not after seeing how much pain a friend went through trying to uninstall it.
I did have the other one once (AVG?) but it gave too many false positives in rather bulls**t ways, like choking on a tiny non-executable data file that was from a Psion organiser, that sort of nonsense.

Norton regards the current Ovation Pro for Windows installer as suffering from "WS.Reputation1”

Is Norton really that broken?

It also does the same with Arculator and promptly bins a lot of the install files.

Since installing Norton on a number of PC’s here they have noticably slowed down particularly on getting to a usable desktop.

Norton also rather helpfully keeps nudging you on a regular basis, despite me saying do things in the background, to take action to “fix” your PC rather than do it quietly as no doubt they want you to be impressed by it working to find all these issues.

So yes it is broken but at least I get it “free” from my ISP.

I tried copying Win Ovation pro over from a dead win machine to a fresh machine – didn’t work :-(

I tried copying Win Ovation pro

Windows doesn’t work like that.

Did you also copy dependencies (DLLs)? I don’t recall if/what OPWin uses as I’ve not looked.

Did you copy the registry entries? Not just the ones for the program, but also the ones to associate the program with Windows (like what to do with .dpd files).

This sort of guff is why they use installers over there…

I copied OAK Draw over that way – and that seemed to work with Win 10.

Someone here said that the oak installer didn’t work on Win 10.

As Win Ovation Pro is now free , as is the RISC OS version. It probably easier to just download the latest version and use the installer for the Windows version. Note you might need to install as an Admin. I don’t think OPro for Windows uses the registry.
But I think it does provide fonts etc.

Norton has been banned on all machines i have built, sold or used since the 90s – terrible software. Have been using Emsisoft Anti-Malware for a number of years on the Windows workstations – not free but not intrusive, resource hungry, etc. Can also remote manage seats for other machines which is good for family etc.

@Fabio – PkgMod sounds good – would be great to use it group update selected packages. Would also be cool if could list select a series of packages for initial installation (could be extended to custom roll one’s own distribution)

In honour of bikesheds everywhere, I’d suggest renaming the commands to PkgInstall, PkgList, Pkg… as it’s nicer to type ‘PkgInstall fooapp’ rather than ‘PkgModInstall fooapp’. In the way Powershell likes to talk about it, ‘PkgInstall’ is a verb (plus an object), while PkgModInstall is a verb combined with the name of the thing providing the service, making a mixed metaphor.

You could look at the kind of ‘verbs’ that different package managers use, as I think there is quite a bit of commonality between them. If everyone calls their verb ‘install’, there’s not much advantage in having yours be ‘fetch’ or ‘provision’. It would just be RISC OS being different purely to make it more work for everyone, which is something we surely don’t need any more of.

There should be the verbs:

• “want” where you may get the package installed if the installer is happy
• “gimme” where you may get the package installed even if the installer is unhappy
• and “shutit” where it’ll be installed even if parts are missing and/or it is incompatible with your machine.

Oh, and:

• “nope” for when you want the software uninstalled
• and “dunno” for when you want something to be uninstalled but not necessarily what you asked for.

@ DownUnderROUser

PkgMod sounds good

Thanks

would be great to use it group update selected packages.

Yes, the update command, in theory can be used for group updates.

Would also be cool if could list select a series of packages for initial installation (could be extended to custom roll one’s own distribution)

In theory that should be possible, given RiscPkg supports groups. But I haven’t seen anything yet that allows a Package to belong to 2 different groups (for example), which would be needed to create an “Initial Installation” group.

However, for everything “Initial Installation/Configuration”, I have been modelling a tool I called “!FirstBoot”, which is a combination of the Installer mentioned on my original post + PkgMod. And yes, it can install groups of Apps for a user that has no clue about RISC OS, but the groups in !FirstBoot are not related to the RiscPkg groups JFYI. The whole !FirstBoot process is a mix between the old Apple macOS 9 initial configuration and soemthing like CentOS installation (where, after configured a bunch of things, a user can select what groups of Apps and tools to install on a vanilla RISC OS). Also !FirstBoot is in slow progresses, so don’t expect anything anytime soon and, beside, both the Installer and !FirstBoot need PkgMod done and working first, so…

There should be the verbs:

Do they have assigned gender?