Splitting up !Boot

I’ve been pondering some of the difficulties people (including myself) have been having with updating their OS to make use of new features like EDID, etc. Particularly on the Pi, it’s a bit fiddly to update the disc image safely, and getting it wrong can result in losing CMOS settings, or a usable display, or even data. I might be getting things wildly wrong, but it seems that most difficulties come from getting stuff mixed-up in the labyrinthine !Boot structure, and so I wondered if there was a way of making this somewhat less all-encompassing?

Specifically, it seems to me the problem comes about because !Boot is really a hybrid that:
(1) contains system-critical stuff that the computer needs in order to startup properly, shared libraries, modules, etc.
(2) holds data on OS-wide choices such as monitor types and network settings that won’t fit in CMOS
(3) is a storage area for application preferences (!Boot.Choices) and user resources (!Boot.Resources)

Ideally, users shouldn’t ever need to delve into the directory tree, but in practice it can often be necessary to get things working. If you ever make a mess of things and need to install a fresh !Boot, you might lose all your cherished app preferences and resources.

So I wondered if splitting (3) out of the directory might be beneficial. After all, if my app choices and resources such as !AWViewer etc. are safely parked away from the main !Boot, then updating the latter, or even replacing it entirely, becomes a simpler proposition. The proposal would be that !Boot retains elements (1) and (2) above, whereas a new top-level directory (say, !Choices) holds (3). It would need some tidying to make this distinction a clean one, as the current !Boot.Choices is a mix of (2) and (3), but would it be worth looking into?

A couple of years ago, the topic of packaging the boot sequence/disc image came up again, and I shared some of my thoughts about it. Possibly those thoughts are useful here as well: https://www.riscosopen.org/forum/forums/5/topics/3178#posts-41094

Possibly those thoughts are useful here as well

Yes, though I don’t fully understand the issues in your proposal, since I don’t fully understand what the Hook directories do :)

Is the initial copy from these directories done to ensure that the !Boot structure used matches up with the OS features in ROM? If so, would it be difficult to modify the current system to avoid !Boot copying bits of itself?

Is the initial copy from these directories done to ensure that the !Boot structure used matches up with the OS features in ROM?

Pretty much, yeah.

If so, would it be difficult to modify the current system to avoid !Boot copying bits of itself?

I’m not sure. Based on what I saw when trying out the EDID stuff, it should be pretty straightforward. But I don’t fully understand the boot sequence either, so some more investigation will be required.

I never fully understood the !Boot and booting process since it appeared. My 2 cents are: as !Boot is needed but now is a source of problems and misunderstandings, maybe it’s time for a rewriting from scratch. To see what’s really needed and move some stuff away or delete/merge.

AIUI the reason why the hook systems is there dates back to when schools often had networked computers that didn’t have a hard disc (e.g. RISC OS 3.1 A310s, A3000s etc). They could be set to boot from a computer that did have a hard disc and would be given the appropriate resources.
There are probably few people now doing that. I hesitate to suggest removing it as I’m sure a few people do rely on it. Anyone in this neck of the woods doing that?

I’m surprised no ones mention Rick Murray’s Harinezumi

Building logging as standard into Booting would save a lot of people a lot of time!

We aren’t talking about removing the hook folders – just the logic that creates the initial !Boot.Choices.Boot folder by copying some of the content from the hook folder (and any other places where bits of the boot sequence get copied around on first boot)

Particularly on the Pi, it’s a bit fiddly to update the disc image safely,

I think the problem with the Pi is the need to leave the Loader object intact (no moving it, deleting it…). So perhaps what might help is to hook into some relevant FS vectors to block potentially dangerous activities on that file?

So perhaps what might help is to hook into some relevant FS vectors to block potentially dangerous activities on that file?

Sounds very complicated. How about adding proper partitioning instead and getting rid of that “Loader” construct? Maybe even adding “link files” that provide access to partitions via things that look like files, which would provide the “Loader” concept without all the associated problems?

Sounds very complicated. How about adding proper partitioning instead […]

Yes, because that’s so much simpler… 🤦

So perhaps what might help is to hook into some relevant FS vectors to block potentially dangerous activities on that file?

There’s a little module that accompanied the PC card called GuardDD which protects against write/delete actions on image files like the DOS “partition” used by the PC card.
Probably the simplest 26 —> 32 bit conversion you’ll ever come across

;   Claims FileV, traps OS_File 6 (delete) and watches for
;   DOS partitions being deleted

Of course you could wait for proper partitioning which is a permanent fix and we wouldn’t want to waste a few minutes 32-bitting something if we can wait months (many) for a real fix.

I never fully understood the !Boot and booting process since it appeared. My 2 cents are: as !Boot is needed but now is a source of problems and misunderstandings, maybe it’s time for a rewriting from scratch. To see what’s really needed and move some stuff away or delete/merge.

I already spent many hours doing just that back in 2012, unpicking 15 years of bit rot where all kinds of odd additions had been made without really understanding why (my favourite being the icon sprites being loaded 3 times). I made lots of notes in addition to some of the original design docs which are useful background.

I suspect there are some aspects that were obvious at the time that didn’t get written down, for example there are some magic locations that FSLock knows about (!Scrap and Public) when everything else is read only.

Some of the work for the never-finished OMAP3Live did make it into !Boot, for example it will now create a default Choices structure on a RAM disc if it finds itself running from a read only medium (eg. ResourceFS). It also needs to cope with network booting, so the order takes account of the network stack not being configured until reasonably late on.

AIUI the reason why the hook […] There are probably few people now doing that. I hesitate to suggest removing it as I’m sure a few people do rely on it. Anyone in this neck of the woods doing that?

You definitely need the hook directories, otherwise it’s not a universal boot sequence, as the startup (and the places where it looks for things like !Configure) differ from OS version to version. This is handled by building path variables which look back only as far as they should do (eg. RISC OS 5 doesn’t look back to RISC OS 4 stuff). In removing it someone clinging on to RISC OS 5.16 would suddenly find their USB memory sticks don’t work because it was softloaded at that time.

The “decade” numbering of the hook directories give an opportunity to make breaking changes by rounding up the OS version number. But still inevitably, if you remove something there’ll be commenters foaming at the mouth that it’s not like a Risc PC did it back in 1994.

Ideally, users shouldn’t ever need to delve into the directory tree, but in practice it can often be necessary to get things working. If you ever make a mess of things and need to install a fresh !Boot, you might lose all your cherished app preferences and resources.

Back to the original question: the two things that seem most out of place are !Scrap and Choices.

You should already be able to move !Scrap somewhere else (FSLock notwithstanding), it being in BootResources is just a convenience that it gets Filer_Run during booting. I’ve not tried recently, but I don’t remember anything in !Boot actually needing it so it should be movable and just add to the “Run at startup” list in Configure.

Choices is a bit of an overlap of boot choices and all the other applications. !ResetBoot wipes both of course. In principle a container !Choices app could be created (like Acorn used to ship on a floppy), though we’d pretty quickly find out who’s abusing Choices: versus Choices$Write or just writing direct into !Boot.Choices.Blah without using either. Plenty of the bundled apps used to get it wrong so I bet all kinds of 3rd party apps do too.

While it’s probably a good idea to tidy things up a bit (MacOS has both System and Library directories, with Library being similar to Resources under RISC OS), can someone just confirm whether I understand things correctly (specifically on the Pi)?

• If an updated !Boot does not contain a Loader directory then installing it via Configure/Boot/Install will not affect the Pi firmware or ROM
• If an updated !Boot does contain Loader (as a normal directory) then Configure/Boot/Install will update the FAT32 “partition” correctly

If both of those are true, then isn’t the easiest “quick fix” simply to distribute all updates as !Boot directories with directions to update them through Configure? Basically, avoid people from having up open up !Boot and Loader at all. Obviously there will be cases where manual intervention will be required, but it seems to me that leveraging the existing update installation system will reduce the likelihood of failure.

But still inevitably, if you remove something there’ll be commenters foaming at the mouth that it’s not like a Risc PC did it back in 1994.

Hmm… I’ve not seen Steve Revill around here recently. Has he given up reading? ;-)

Probably the simplest 26 —> 32 bit conversion you’ll ever come across

Hehe… Had to add in a trap on OS_FSControl,26 to stop it being renamed (moved) as well.

Tested on a 4MiB DOS file (&FC8) on RAMdisc. No way in hell I’m trying it out on my live Loader file…

REM GuardLoader
REM by Rick, Friday, 24th March 2017
REM
REM Basically an extended version of GuardDD v0.05
REM by Matthew, April 1998 / 09 Dec 1998
REM
REM It looks as if the Aleph1 code is now GPL; so this
REM may be as well...
REM
REM Claims FileV, traps OS_File 6 (delete) and watches for
REM DOS partitions being deleted
REM
REM Also claims FSControlV 25 (rename) to stop 'em being moved.
REM
REM Builds with plain BASIC

ON ERROR PRINT REPORT$+" at "+STR$(ERL):END

DIM module 2048
FOR pass=12 TO 14 STEP 2
P%=0:O%=module:L%=O%+2048
[OPT pass
:
    EQUD    0 ; start
    EQUD    init
    EQUD    final
    EQUD    0 ; svc
    EQUD    title
    EQUD    help
    EQUD    0 ; command
    EQUD    0 ; SWI number
    EQUD    0 ; SWI handler
    EQUD    0 ; SWI table
    EQUD    0 ; SWI decode
    EQUD    0 ; MsgTrans
    EQUD    flags
    :
.title
    EQUS    "GuardLoader"+CHR$(0)
.help
    EQUS    "GuardLoader"+CHR$9+CHR$9+"0.01 (24 Mar 2017)"+CHR$(0)
    ALIGN

.flags
    EQUD    1 ; 32 bit OK

.err_caught
    EQUD &108C3
    EQUS "You can't move or delete DOS partitions (> 2MB) while GuardLoader "
    EQUS " is loaded! Quit it, or change the filetype."+CHR$(0)
    ALIGN

.init
    STMFD   R13!,{R14}

    ; FileV for delete
    MOV     R0,#8
    ADR     R1,filev
    MOV     R2,R12
    SWI     "XOS_Claim"

    ; FSControlV for rename (*Move)
    MOV     R0, #15
    ADR     R1, fsctrlv
    MOV     R2, R12
    SWI     "XOS_Claim"

    LDMFD   R13!,{PC}


.final
    STMFD   R13!,{R14}

    MOV     R0,#8
    ADR     R1,filev
    MOV     R2,R12
    SWI     "XOS_Release"

    MOV     R0, #15
    ADR     R1, fsctrlv
    MOV     R2, R12
    SWI     "XOS_Release"

    LDMFD   R13!,{PC}
;
; Not quite as simple as it seems because image files' types are
; always reported as &1000 and we have to use OS_File 17 to read
; the load address and interpret that instead.
;

.fsctrlv
    CMP     R0, #25
    MOVNE   PC, R14
    ; original filename is in R1
    B       do_check

.filev
    CMP     R0,#6
    MOVNE   PC,R14
    ; filename is in R1

.do_check
    ; expects R1 to point to the filename
    STMFD   R13!,{R0-R6,R14}
    MOV     R0,#17
    SWI     "XOS_File"
    LDR     R3,load_match
    BIC     R2,R2,#&FF
    CMP     R4,#(2<<20)         ;if >=2MB
    CMPGE   R2,R3               ;and filetype = FC8
    LDMFD   R13!,{R0-R6,R14}
    MOVNE   PC,R14
    LDMFD   R13!,{R14}          ;generate an error
    ADR     R0,err_caught
    SWI     "OS_GenerateError"
    MSR     CPSR_f, #(1<<27) ; disable interrupts
    MOV     PC,R14
.load_match
    EQUD    &FFFFC800
:
]:NEXT
SYS "OS_File",10,"GuardLoader",&FFA,0,module,O%

Hehe… Had to add in a trap on OS_FSControl,26 to stop it being renamed (moved) as well.

Hmmm, never tried a move. I had this feeling you’d look and modify :)

Tested on a 4MiB DOS file (&FC8) on RAMdisc. No way in hell I’m trying it out on my live Loader file…

I’m sure someone with a few spare cards will test just that.
Should keep people safe until proper partitioning surfaces.

I had this feeling you’d look and modify :)

The hardest part was grabbing the original sources, given riscos.info is down…again. [http://downforeveryoneorjustme.com/riscos.info]

So, having made this module, like wot I just have, would you recommend putting it in PreDesk or in Tasks?

Rick – what does line below do?

ie I=7 in cpsr_c (RO5)

Rick – what does line below do?

Nothing, the preceding OS_GenerateError doesn’t return.

Yup, you’re right. :-)

I was looking at the old 26 bit PSR to see what that bit was as I was expecting to set V or somesuch, but didn’t change it to be the correct I but in 32 bit PSR. Oops.

That said, ORRing on I bit (as the original code does)… What re-enables interrupts afterwards?

Choices → !Choices = nice idea.

Perhaps that – yes – !Boot could be separated completely from !Resources.
!Boot would be without ANY user resources or preferences.
The other things will go in $.!Resources or – better – $.!System

!Boot → small static !Boot on disc (could be in ROM under ROS5)
!Boot.Resources.* → $.!System.*
Nota: RISC OS could boot in Wimp even if !System is not present at all.
But it’ll need !Boot

Other changes:
!Boot.Choices → Choices
System → Modules (better name, IMHO)
!Boot.Resources.Wimp and !Boot.Resources.Configure.Textures → new tool: Themes. Some people could not install/use it, if not needed.
!Boot.Resources.Configure.Monitors → new tool: Monitor with MDF and tool to choose non automatic/EDID setup. Some people could not install/use it, if not needed.

!Configure → Bye bye. Replace with Resources:$.Configure. Can be accessed with a right click on Apps. Populated with some default static applets (present in !Boot), and some other coming from !System.* things. For example the !Fonts applet will simply call Fonts.!Run, and use the Fonts default sprite. Simple and similar to Resources:$.Apps.

So an upgrade is just ‘delete !Boot, put the new one’. Or nothing if in ROM.
In rare cases, some $.!System applications will need to be updated.
The same way some $.Apps applications needs sometimes to be updated too.

Nota for $.!System things: version should be easy to check. Today, if I choose Help on !Fonts, I have no idea of the version used. !Fonts should be an independent solution, mixing current !Font and !FontSetup in one product. Click on it, and you configure Fonts. Of course, !Boot will register it automatically (as all other $.!System components) in Resources:$.Configure. And then it will replace the minimal $.Fonts applet integrated in !Boot (needed if Fonts is not installed).

I’m not sure I’m clear :)

Hehe… Had to add in a trap on OS_FSControl,26 to stop it being renamed (moved) as well.

Nice tool. With a better !Configure applet for Lock, we could have Full lock, Boot Lock and FS defined lock options. Would be great. And – at least – useful. Of course, for security reasons, management tool should be in ROM and password stored in CMOS. Full lock / Boot lock / FS lock / Unlock states could be also fixed in CMOS. FS lock would rely on Protected bits applied to files. So you could protect what you want and be sure that no one will Unprotect it.

Just my 2 cents.

Of course you could wait for proper partitioning which is a permanent fix and we wouldn’t want to waste a few minutes 32-bitting something if we can wait months (many) for a real fix.

The problem with GuardDD and similar ideas as well as Rick’s quick hack is that it does not solve the problem at all. I can still rename !Boot, install a new !Boot and copy Loader to the new !Boot. Which breaks the Loader concept.

I don’t think there is a “quick fix solution” to the problem apart from the real solution, which is partitioning support. Which is hopefully just around the corner, judging by CVS activities.

Maybe a simple !Boot upgrader would be a viable solution (just like SysMerge). It would need some intelligence to identify the !Boot version it upgrades, it needs to know which files needs to be added and removed, and it needs to understand which changes the user made and which are the places that should not be touched (like Choices).

OK, not so simple anymore…

We need at least a separation of !Boot into the “system part” and the “user-changeable part”. At the moment, !Boot includes too many things of completely different character.

HD working partitioning was talked about at the 2017 SW show.

The SD card that I setup for my Pi is partitioned.
I did it like this:

1) Use Hform to format the card for Riscos, but reduce the disc size by (say) 64MB

This leaves 64MB unused at the end of the card.
Hform destroys the existing partition table.

2) Make a new partition table in sector 0 of the card.

Partion 1 → the 64MB area at the top of the drive – set it for FAT use.
Partion 2 → the existing riscos area

3) Use a FAT formatter to format the FAT partition
4) Copy the Pi boot stuff to the FAT area, and the RiscOS !Boot etc to the Riscos area.