Move HD pre-installed apps to PackMan to allow updating

The Pi HD comes with various apps, some part of RISCOS, others 3rd party.

To allow users to easily keep these up to date, it would be great if they were all packaged for distribution via PackMan and then installed to the HD image via PackMan so they can be updated quickly and easily.

The current process of hunting around the internet for the latest version, then having to find pre-reqs such as SharedUnixLib, Tinct is not user friendly.

3rd party apps may require approval from the authors and obviously someone has to maintain them and keep them up to date, but this should hopefully already be happening with the HD image pre-installs, so it’s not that big a leap to move to packaged distros.

The other advantage to distributing software this route is the user can easily remove (or add) software they do or don’t want in the OS install.

then having to find pre-reqs such as SharedUnixLib, Tinct is not user friendly.

I didn’t think Tinct was ARMv7 friendly :) but yes I do understand your point in general.

The other advantage to distributing software this route is the user can easily remove (or add) software they do or don’t want in the OS install.

Some people are a bit vocal about where package managers put the software vs where they want to put that software. Take a deep breath and wait for them to say something negative.

The Pi HD comes with various apps, some part of RISCOS, others 3rd party.

May I just point out that the correct “ectriture”, or perhaps “rendition” is “RISC OS”.

The approved verbal rendition is “RISC” “O” “S”, though I myself have trouble with that!

However, we do need to maintain standards to some level!

May I recommend Pluto’s auto-correction facility!

What a pity it doesn’t work on fora (or, as I would prefer, forums, not being a pedant grade 1).

John

Look at the title bar!

:-)

Better still ROOL can supply a vanilla PI/harddisc4 image and someone else can offer/support an ‘extras’ package.

Or as an absolute maximum just supply packman and people can install what they want – or delete it if they want. That way you can encourage people to use packman

I didn’t think Tinct was ARMv7 friendly :) but yes I do understand your point in general.

This was one I had to manually update when I updated NetSurf to an the latest version (which seems to fix the Page Zero access I was seeing with the pre-installed version on the HD image)

Better still ROOL can supply a vanilla PI/harddisc4 image and someone else can offer/support an ‘extras’ package.

This could of course be an option for developers or folk running servers that don’t need all the bloatware or just want a small install.

Some people are a bit vocal about where package managers put the software vs where they want to put that software.

And due to this PackMan was changed. For many years it allowed you to move things after they were installed and more recently gives you a suggested location which you can override before installing.

There are a few exceptions of course. Some packages haven’t been updated yet and some things do have to go in a specific place.

To allow users to easily keep these up to date, it would be great if they were all packaged for distribution via PackMan and then installed to the HD image via PackMan so they can be updated quickly and easily.

This should be possible. I believe ROOL have even talked about this before. I have had various email conversations, but nothing got to a stage where I had enough feedback to pursue it.

It’s relatively easy to set up. You just need a zip program, write the control and copyright files and look into dependencies. The hard part is to pull the correct version number from the source tree.

To get this to happen, I have a few questions:
1. Should I pull the programs from a pre-built hard-disc image or should they come out of the source build process?
2. Where should I store the Control and Copyright file for each package? The copyright file can be shared between a lot of the applications.
3. To create the packages themselves should it use a make file or an Obey file or something else?
4. How do I get the version for each of the components?
5. Where do the packages need to then be put?
6. The program from the GCCSDK can then be used to build the package indices, so there would need to be some way of automatically running that.

The main thing is how to integrate this into the ROOL build system.

We could start by just creating a few packages to get the system going and then add more and more as time goes on until most of the HardDisc is covered.

As the author of PackMan I’m more than happy to make any adjustments to how it works, if any are required, to it to make this happen.

Should I pull the programs from a pre-built hard-disc image or should they come out of the source build process?

I’d say that they should come from the source build process. The way I’d do it would probably be as follows:

• Add a new build phase to !Builder / srcbuild.
• Modify the shared makefiles (which most components make use of) to implement that build phase (each build phase corresponds to a makefile rule, for each build phase srcbuild basically just goes through all the components and runs amu on that components makefile, specifying the current build phase as the target to build). Aim to find a way to leverage the existing ‘install’ rule – it’s the install rule of each of the makefiles which is responsible for copying applications from their build folder into the disc image (at a location specified in the components file). For the packman rule you could probably have srcbuild specify the filename of the package to generate. The makefile can then manage a temp folder for storing the uncompressed package, copying all the application files in (as per the standard install rule), copying in the extra metadata packman needs, and then zipping it up. Temporary files should be kept so that if the build is run again the makefile should be able to spot which steps do/don’t need repeating (nobody wants to want to sit through watching a bunch of packages get redundantly zipped up!)
• Have a script which uploads the new/updated packages to the package server (see below). This script would run on ROOL’s build machine after the build has finished (and can easily be written in something Unixy, ROOL’s build machine is just a unix/linux box which launches a tweaked version of RPCEmu in order to do each build. CVS interaction and file upload is handled on the Unix side, RPCEmu is just there to do the build steps)

How do I get the version for each of the components?

Each component has a VersionNum file which contains the version number and date, e..g Paint. This is what’s used to generate the version number & date strings for both modules and applications. Apart from being a valid C header, there’s also an awk script which parses the file and injects the version/date into other files – that’s probably your best bet if the data needs to be injected into any package control files.

Where do the packages need to then be put?

During the build, you’d probably want a new, top-level ‘Packages’ directory in the build tree. After the build, you’d want them to be uploaded here:

http://packages.riscosopen.org/

I believe Theo Markettos is the mastermind behind that area of the site. Presumably he or someone at ROOL would be responsible for writing the script which scans the output folder and uploads new packages

It may also be worth taking a look at the ABRelease component which handles packaging of ROMs and the disc image ready for upload (this component implements a special release_autobuild build phase that isn’t currently exposed through the !Builder UI).

The main thing is how to integrate this into the ROOL build system.

Indeed!

Also note that it’s possible that in previous discussions people may have come to different conclusions than me on how to do this kind of stuff (but I can’t remember which threads those discussions were in, and it’s probable there were some email discussions which I haven’t been a part of either). Theo or a ROOL staffer might be your best bet for finding out what the current plan of action is.

As the author of PackMan I’m more than happy to make any adjustments to how it works, if any are required, to it to make this happen.

I have one suggestion: to be able to make packages based on third party zip files. for example a ‘take the file x and directory y from the zip file downloadable at this place, then take the patch x from this other zip file’.

I could for example make a package for SPatience based on original SPatience archive and the RISC OS FR’s patch. No need to ask author and RISC OS FR authorization, since their work will not be inside the archive.

Of course, a way to alert the maintainer when the third party zip is changed (or not available any more) would be great.

I have one suggestion: to be able to make packages based on third party zip files. for example a ‘take the file x and directory y from the zip file downloadable at this place, then take the patch x from this other zip file’.

I could for example make a package for SPatience based on original SPatience archive and the RISC OS FR’s patch. No need to ask author and RISC OS FR authorization, since their work will not be inside the archive.

Of course, a way to alert the maintainer when the third party zip is changed (or not available any more) would be great.

I can see why this would be useful, but it unfortunately doesn’t fit well. All the files for a package need to be in the package file and the copyright and distribution rights need to be clear.

In most cases I try to contact the author(s) for their permission (even when the copyright and distribution rights don’t require it). Most I’ve asked are OK to let their software be packaged even if they are not interested in the packaging themselves.

Wouldn’t it be possible to package a fetcher program/script as though it is the actual program.

When it is executed, it would fetch the files and populate the package folder as though it were normally packaged, and run the program normally the next time.

To make this easier, packman would need an option for run a custom installer on the initial setup, and options to patch files, rather than replace them on update.

However back on to the main subject, I would like packman to be able to manage the whole !Boot too, including softloads.

Meaning if the ROM had it, hdform, and working DHCP, it would be possible to do an over the air install with only the ROM image.

Wouldn’t it be possible to package a fetcher program/script as though it is the actual program.
bq. When it is executed, it would fetch the files and populate the package folder as though it were normally packaged, and run the program normally the next time.

I don’t think this would be possible, or if it was it would probably over complicate things. However, package folder for a downloaded hard disc image could be set up with the information on what is already installed.

However back on to the main subject, I would like packman to be able to manage the whole !Boot too, including softloads.

As far a I can see there will always need to be a minimal boot system, to start with. Most of the components within it can probably be packaged, but I’m expecting there will always need to be something that needs to be there first.

I’ve downloaded and had a look at the build system now and can see how packaging may be added in the individual make files at least. I really need to shell out £50 for the DDE next, so I can look at it running. Christmas is coming soon though!

Can AMU do any text transformation in the makefile? Or would it need an external program to set a variable for it to use?

I really need to get the version in dotted form and transform it so the dots are replaced with slashes for file names so I can create the package file name.
e.g. for Maestro version 1.00 I would want to end up with a variable with the value Maestro_1/00 that an be used for the file name.
I can do this with gcc make, but my old documentation from my very old version of the DDE doesn’t mention anything that could achieve this.

Quick suggestion: date of release in the software list (or detail).

Can AMU do any text transformation in the makefile? Or would it need an external program to set a variable for it to use?

I suspect you’ll have to use an external program.

As far a I can see there will always need to be a minimal boot system, to start with. Most of the components within it can probably be packaged, but I’m expecting there will always need to be something that needs to be there first.

I would envisage a minimal !Boot along with !HForm !Packman and a configure option for the boot drive being in resourcefs in the ROM.

On the Pi, (with the current state of partition support), this should allow RISC OS to be downloaded with NOOBS, with minimal data used on the SD card, and the the rest of the system be installed on a USB drive.
(With proper partition support, it would allow a nice flexible system)

I’ve started to try a few things out for this now and have managed to add a package target to the Maestro make file that will create a package for it.
amu has improved since I last used it (many years ago) and the build tree comes with some useful scripts for getting versions and a copy of sed to replace them in files.
I haven’t yet managed to figure out srcbuild/!Builder to add a package step to that.
My current plan is to create the packages in an InstallDir.Packages directory and then on linux copy them to the location for the website and run a program to index them.
To create a hard disc image without the packaged item on it, my current thoughts are to scan the zips for the packages and create a list of file names to exclude that could be used on the current HardDisc4 image.
Does anyone know if the HardDisc4 images is zipped up under RISC OS or Unix as where this program would run depends on this?

Does anyone know if the HardDisc4 images is zipped up under RISC OS or Unix as where this program would run depends on this?

It’s zipped up under RISC OS, as part of the release_autobuild phase (which isn’t accessible from !Builder, but can be invoked by running srcbuild manually). Specifically it’s the ABRelease component which handles the zipping up of the disc image, ROMs, etc. – see Resources.DiscDev.release_autobuild.

Also note that if you do try invoking release_autobuild manually, there are a couple of system variables you need to set. I’m not 100% sure what they are, but looking at the ABRelease makefile it looks like they’re AutoBuild$Root (root output directory) and AutoBuild$Build (subdirectory for the current build, e.g. DIscDev)