More slick installation of major updates
Andrew Hodgkinson (6) 465 posts |
No – odd, I should get those. However, it appears that my upstream delivery mechanism uses sender domain verification, so sometimes I don’t get messages if the “From” domain is scrambled. You could try sending to “webmaster@riscosopen.org” instead, just in case it was a temporary hiccup rather than a domain issue. |
Alan Buckley (167) 232 posts |
Martin – I’m not really the person to ask about changes to the underlying packaging system, you need to contact Graham Shaw. My thoughts were that the skeleton !Boot and !System would probably be prerequisits before you could do anything with the machine. The packaging system can then be used to fill in everything else. I think this is a reasonable way to go as other systems always have a minimum distribution that needs to be put on the machine before any package management can take place. It sounds like you are making progress creating your own scripts etc so don’t really need to look at what I’ve done. I have a couple of c/c++ programs that may or may not be of some use. If they are email me and I’ll send them to you. pkgindex – This is a c program that creates the index file for the packages from the package files. A version of it is used to generate the indices for the autobuilder site. roolpkg – This is a C program to help build the control files for the package manager. It uses command line parameters to determine what it can do. It can scan !boot to determine the sysvars It can scan !Sprites for the sprites If can get the version number from the messages file If can generate a default copyright text file |
Steve Revill (20) 1361 posts |
I got the email to ‘info’ so at least it went to some of us. |
Martin Bazley (331) 379 posts |
Both me and ROOL have tried this. He seems to have dropped off the Earth. Of course, the RiscPkg policy manual is nothing more than just that – a policy. We could always declare !RiscPkg obsolete and advise users to use your program instead, since it’s being actively developed. This has the advantage that we can pretty much add whatever features we feel like. The two I’ve suggested so far (!Boot installation and a control field giving a location relative to the root of the hard disc for the files to be installed) shouldn’t break anything or be very difficult to implement. This was why I was addressing you, as the author of the only maintained package manager – which is, after all, what’s important to the end user. I agree with your point about the minimum distribution. You sent me your C files the last time we discussed this. Right, off to finish my program now. |
Martin Bazley (331) 379 posts |
The first version has now left my outbox! I also sent a copy of the manual to the address Andrew provided. I hope I made it suffficiently flexible to take whatever filenames the autobuilder throws at it. |
Alan Buckley (167) 232 posts |
The thing I’m developing at the moment is really just a front end to the underlying packaging system and used a library created by Graham called libpkg. The library has versioning in it, so if Graham really has disappeared we could enhance it on our own and just update the version. I’d like to concentrate on getting my front-end to a useable state using the existing library first, but once I’ve achieved that if Graham still hasn’t surfaced I’ll start looking into the internals of the library. Can you work around the current problems or are they show-stoppers? If they are show stoppers, can you work on other parts until they are resolved? |
Martin Bazley (331) 379 posts |
To answer the last paragraph: yes, and yes (in fact, that’s what I’m doing by concentrating on the desktop applications). The trouble is that most of these components need to be installed in highly specific places on the user’s hard disc, which is functionality which simply isn’t currently catered for by the RiscPkg format. Everything on the boot sequence-related downloads page, for example, is impossible to package. It would also be nice if the desktop applications could have their current locations preserved, as it would save everyone a lot of effort when updating applications on their existing hard disc layouts. Unfortunately, there’s no way to do this either. |
James Peacock (318) 129 posts |
I don’t think you can do this without user intervention. As mentioned on the other thread people may even have multiple versions of an application, old versions or backups perhaps. Regardless, given a choice I wouldn’t use an installer on RISC OS unless I could see exactly what was going to happen first and veto it. In my case, the layout is non-standard but nearly everything is Filer_Booted at start up, so the system variables would provide a reasonable guess, though in general being able to override this would be necessary. I don’t mean this post to be negative, I certainly appreciate why such a utility is important: although I find things easy to manage by hand on RISC OS, I don’t on Linux and just leave it to apt to track the files it has scattered all over the system. |
Peter Naulls (143) 147 posts |
I know this is going to sound awfully picky, but we’re absolutely not talking about an “installer” that you see in Windows or has been used in a few RO apps. Those are very narrow in what they do, single purpose, and often don’t even track files at all. I say this so we can yet again avoid going down the road of comparison with Windows (which has been done many times when this topic has come up). The difference between an installer and a package manager is huge. One of the reasons is that a package manager tries to keep track of everything, and understand what works with what, etc.
You have that choice on Linux if you want, too. But in practice, it’s often not even remotely interesting most of the time. But you have the possibility of coming back later and seeing what files belong to what package etc. I know that RISC OS users like to think they need the paranoid control of knowing exactly where every file goes, but in practice I’d argue that they don’t, almost all the time. |
James Peacock (318) 129 posts |
Yes, I meant ‘package manager’ rather than ‘installer’, though given the rest of the message that should have being clear. My paranoia, as you put it, is from experience of using Gentoo’s portage and Debian’s apt on Linux and in both cases finding that the do occasionally get it wrong: removing things I’ve explicitly installed or installing a later version of autoconf which is incompatible with an older version, that sort of thing. |
Peter Naulls (143) 147 posts |
Well, that’s actually an extreme example. autoconf is notoriously fickle with its different versions. and in any case Debian gives you control over which one to use – in cases like this, it often happens that two major versions have different packages. Besides, there’s not likely to be an autoconf on RISC OS. Yes, package managers do get it wrong sometimes if you pull regularly from Debian development repositories, but that does have more than 20000 packages, so you can’t get everything perfect when it’s in flux. If we’re lucky on RISC OS we’ll have 500 odd packages. That’s few enough that one person might be able to retain consistency themselves, perhaps with a little help. |
Alan Buckley (167) 232 posts |
I’ve been in contact with Graham via email today and sent an email asking him about the install locations. I’ve never had any problem contacting him. He says he is involved in other things and has had some problems with his internet infrastructure, but he hasn’t had any recent messages from you or ROOL. I’ll follow the locations up with Graham and get back to you. Am I correct in thinking we need to new paths to enable the ROOL packageing: Root (or some better name) which gives the location of the root of the hard disk that the applications and similar things need to be installed. Boot for installing components into the boot structure. I guess the Root is needed so we can create the whole disk image, it just seems a pity we can’t put things under the Apps folder as the package manager does now. |
Martin Bazley (331) 379 posts |
It looks as if the most recent activity was back in December, on the ‘Packaging ROOL downloads’ thread – however, that ended when we received Graham’s ‘actual’ email address and it looks as if nothing happened since then. One thing which is certain is that the RiscPkg website is out of date with regards to contact details. Anyway, since you seem to be in contact:
The theory is that RiscPkg maintains its own local list (in Choices) of places to install certain package names. The system I had in mind goes something like:
The only tricky thing would be if the user moved the location of the application in the meantime, which would cause problems if RiscPkg inserts anything into the boot sequence which refers to the expected location.
Fortunately, we already have a Boot$Path. Thinking again, I can see there are going to be potential compatability issues with ROL OSes and their multiple-user Choices structure, when we want to install things into PreDesk or Tasks (USB drivers, for example). As the assumption is that these must be run early in the !Boot sequence, it wouldn’t be much good if they ended up in a user’s personal directory. I see that RO4 defines a system variable ChoicesDefault$Dir, so in theory we could check to see if that exists and use Boot:Choices if it doesn’t. So this is how the second feature would work:
That’s the best I can do for now – I’m open to suggestions about potentially cleaner implementations. |
Alan Buckley (167) 232 posts |
Martin, I’ve been using the name “actual” at riscpkg dot org. It usually takes a week or so for a reply, but I usually get a reply. Do you want to try again? It’s probably better if you can talk to him directly. If you have trouble contacting him still email me and I’ll try to contact him cc’ing your address. When I asked about locations his reply was that the original set of Paths provided were just a starting point, so others could be added as required. He thought a Boot path would be OK, as long as it was only used when a better path was not available (i.e. it should only be needed rarely). He thought that rather than my suggestion of a ROOT path we should have paths for each directory required below root. There aren’t that many so this seems sensible to me. Although your idea for how the install works above give most flexibility for installing an application to any location, I’m a little unsure of what the real advantage of doing so is. Working on other systems it is very rare that I bother to change the default path anything is installed into. The other thing is that you would then need to popup a save dialog box for dependencies as well. RiscPkg does allow you to create stub shortcut applications, so you can have a link in any folder to the real application, which should cover the occassion you want to launch the program from somewhere else. When I was saying add a “Boot” path, it was to add it to packaging paths and it would be mapped to Boot$Path as you say. When emailing Graham I asked a little about ROL’s multi-user boot structure. Unfortunately I have no experience of it so I’m open to all suggestion on what can be done for it. Again, please try Graham again he may have more informed comments on your suggestion. Ideally an application wouldn’t use packaging to install application settings, but that doesn’t help with things that need to go into PreDesk and places like that. If you think it would at least be a step forward to add the Paths you need for packaging, can you identify what we would actually need (in addition to what already exists)? |
Martin Bazley (331) 379 posts |
OK, I’ve just sent Graham an email detailing the list above with a few refinements. I can see that a simple ‘Boot’ directory would be prohibitively difficult and messy to implement, so I decided, like you said, that it would probably be better to have paths for each location required in !Boot. In the end this boiled down to Configure, PreDesk and Tasks – plus we already have a Resources and a System catered for. As you quite rightly say, installing application settings is not the job of a package manager, and to be frank I can’t imagine what Acorn thought they were doing with the Choices:Boot directory in the first place – surely a more appropriate place would have been in Utils? However, what’s done is done and the best way to get around the multi-user problem is to use the variable ChoicesDefault$Dir, provided explicitly for that purpose, and to simply use Boot:Choices when the variable does not exist (i.e. on RO5). I’d completely forgotten RiscPkg could create stub applications – I hadn’t actually used it that much – so you’re right, that would seem like the best solution to the installation locations problem. The only reason we have a problem in the first place is that the structure of the HardDisc4 image and the RiscPkg hierarchy are mutually incompatible, so we need some kind of system to emulate the old structure to avoid causing endless grief to every user who installs a package under the new system that they last downloaded under the old. Hopefully once we’ve got these new features up and running it will simply be a matter of drudgery producing the ‘Desc’ files (the system I came up with for my package builder – designed to be created once and then left alone) for every other component on the website. That’ll probably be something I end up doing (after all, I’ve got to be useful for something, right?). |
Alan Buckley (167) 232 posts |
When you mention the paths in Boot I was actually thinking of paths for the files installed in directories below HardDisc4 (e.g. Utilities etc). But your suggestion for doing the same for the Boot directories makes sense to me for the few paths within Boot. |
Martin Bazley (331) 379 posts |
The best solution I could come up with for files installed in, say, Utilities, was a new field in the Control file (detailed above) which gave the suggested installation location. It would be up to the package maintainer to write that properly – for the ROOL components, I suggest we prefix them all with Boot:^. To make it more RISC OS-y and allow users to keep their applications in places other than the defaults, I proposed not to enforce this beyond, for a package which had not been previously downloaded, filling out the writable field in the save box with it. For previously downloaded packages, the idea is that RiscPkg remembers the location the last copy was saved to. |
Alan Buckley (167) 232 posts |
You don’t really need a new field in the Control file for the install location. This is what the Paths file is for. The Path name in the paths file is where you put the files in the zip file. e.g. You could have a new path called “RootUtilities” and that would be set in the Paths file to “Boot:^” and the zip file would then contain a directory RootUtilities where the files go. I haven’t chosen a goor name here, but it should give you an idea of how it works. If it was later decided to add the save dialog box, it would still be possible to make it work using the existing system. |
Martin Bazley (331) 379 posts |
Can you please provide a reference for this claim? I’ve just reread the policy manual and couldn’t find any mention of the feature you describe. The closest is the ‘CrossPaths’ file, which seems nothing to do with it. Of course, if this feature was already implemented it would help the progress of the project enormously. |
Alan Buckley (167) 232 posts |
It does seem hard to spot. The Paths file is mentioned in the RiscPkg help. In the Policy menu at the end of the “Binary Package Format” section, just after the table it talks about the logical paths. As far as I can see the modification we would want for the package system would be to just include some more logical paths for the locations for the ROOL components. I haven’t tried it, but you could probably test it just by adding the new logical paths to the Paths file described in the RiscPkg help and build packages using them. I believe the final “proper” solution would be to ensure RiscPkg was shipped with the full set of paths required and use a new Standards-Version for packages that use the new paths so user know to upgrade RiscPkg to use them. |
Martin Bazley (331) 379 posts |
OK, I’ve found it now. One thing which confused me is that it explicitly stated the Paths files could be found in Choices:RiscPkg – no such directory exists! I eventually found it inside the !RiscPkg application itself. Presumably the idea is that if you want to define extra ones you create the directory yourself and copy it there – not good practice, I’m afraid, not allowing for the fact that the help file may be totally wrong. Having had a look at this, it looks as if the most compatible solution would be to define a path, as you had it, called ‘Root’ and build the directory structure below that according to the paths in the Binaries components file (linked to by Jeffrey on page one). Then we could have a line in ‘Paths’ called:Root = <Boot$Dir>.^ Unfortunately we’re still going to have problems with multiple-user systems, since we can’t guarantee that the default user’s Choices:Boot will be in the same place on RO4/6 and on RO5. This will need some coding on Graham’s part. And I’m still not quite sure how this is going to fit in with the aforementioned confusion over the Paths file – I mean, is our only option to make a ‘Root’ path official? |
Alan Buckley (167) 232 posts |
I don’t create extra paths. There are just the official ones. The Choices:RiscPkg ones are if you override the default paths that are stored in !RiscPkg. I think the functionality is correct here. It just may be lacking a UI to it. When I mentioned the idea of a Root path to Graham he pointed out that there are very few paths we need under root so it would probably be better to have a path for each. e.g.RootUtils=<Boot$Dir>.^.Utilities RootPrint=<Boot$Dir>.^.Printingetc. (probably with better names though) As far as I can see we do need to make all these paths official, because that would ensure that sensible defaults are shipped with !RiscPkg. As you say, I’m sure coding will be needed for multiple-user systems. Have you heard anything from Graham yet? |
Martin Bazley (331) 379 posts |
Still no word from Graham – the email I sent him is rather outdated now. :-( OK, from the Binaries file, it appears that the complete list of paths referred to is thus:
It seems pretty disappointing that these can only be implemented as part of a set of new standards. Wouldn’t it be simpler to just specify it in the package file? |
Martin Bazley (331) 379 posts |
Graham has replied! Unfortunately he replied to a two-week old email, dating from when I basically had no idea what I was doing. So I think I’ve managed to isolate the requirements for packaging to proceed to two new features: One, a list of new paths corresponding to the locations components are built in. I personally think it would be easier if we could just combine the ‘Library’, ‘RO500Hook’ and ‘Utils’ into just the one path called ‘Boot’. I’m also mystified as to why SoftSCSI is built inside RO500Hook and not in Choices – surely it needs to be moved inside Choices for it to work? Two, a change to the way RiscPkg installs new files. Graham has quite rightly pointed out to me that, technically, one package != one application, and so storing installation locations inside a list of packages wouldn’t work. The alternative I proposed consisted of a list of logical pathnames of applications (e.g. Apps.!Chars) and the locations said applications were eventually saved into. The list I sent Graham was: EDIT: had to put it in pre tags due to problems with the up symbol. Either: Boot:^.Apps.!Chars pathname exists. New version of !Chars overwrites Boot:^.Apps.!Chars. Or: Boot:^.Apps.!Chars pathname does not exist. Either: A logical pathname of Apps.!Chars has not been previously downloaded. Either: Boot:^.Apps (parent directory of application directory) exists. !Chars installed to Boot:^.Apps.!Chars. New entry created in list of logical pathnames for Apps.!Chars, with a save location of Boot:^.Apps.!Chars. Or: Boot:^.Apps does not exist. User is offered a save-as box. User saves application. RiscPkg notes the location saved to and updates the entry for Apps.!Chars with it, for the next update downloaded. Or: A logical pathname of Apps.!Chars has been previously downloaded. Either: The !Chars directory the last version was saved to exists. New version of !Chars installed in previous location. Or: The !Chars directory the last version was saved to does not exist. User is offered a save-as box, etc. (as above) |
Jeffrey Lee (213) 6048 posts |
Some magic is used to copy the correct RO5XXHook.Boot folder into Choices the first time the boot sequence runs. I say “magic”, because I’m not quite sure where the code is that does it! Also, even after the default choices directory/directories are copied, the boot sequence still relies on some path variables (Choices$Path, BootResources$Path) to allow for transparent use of the hook folders. |