Extended CMOS file

A while ago it was suggested that it would be useful to have a bigger “CMOS” file on newer machines that boot from SD. I’ve brought the topic up again in connection with reading EDID.

There is no particular size limit for a “CMOS” file – it’s limited only by the size of the boot partition.

It was suggested that key=value pairs would be a good way to store extended information.

Would it be useful?

Would it be useful to have a module that allowed writing and reading key=value pairs to and from a file?

Some of the information might need to be read start boot-up. It’s not too difficult to read key=value pairs without a module, i.e. before the module has been invoked. It’s rather harder to write them (think of re-writing a value of a changed length, or of deleting a value), and I think you’d normally only want to do so when the OS is up.

A while ago it was suggested that it would be useful to have a bigger “CMOS” file on newer machines that boot from SD.

And it would seem an appropriate time to rename it, say, SETTINGS, as there’s no longer any CMOS involved at all!

The purpose of the CMOS memory was to store settings that were required before the !Boot contents were readable. Things like screen size, ROM modules to be unplugged, location of the !Boot (if any) plus any settings required for a machine (like the Pico build) that doesn’t have a !Boot directory. Iyonix still has CMOS RAM as does Risc PC.

Why not just offer the option of a file !Boot (as an alternative to the !Boot directory) with the necessary settings?

There seems no point in putting extra things into ‘CMOS’ that could be more easily understood as a configuration plug-in inside !Boot.

Dave ever since you originally suggested it I’ve thought it a good idea and I think others or I have mentioned on more than one occasion since that the problem being commented on would be fixed/alleviated if your idea was used.
I suspect the problem is who is going to implement it. I think it will require someone with quite a good low level knowledge and they are not many about, and those that are I doubt have any spare time.
I wonder if a bounty might help?

I have this naive idea that it would be quite simple to implement, and it might be something that I try in the very near future.

I thing SETTINGS/TXT would be a good name.

It has to be in the boot partition to be of most use, as it can contain settings that are useful prior to boot. Being a text file, it would be easy for any code to read key=value pairs, and would not require the module to be present. Once the OS is up, though, it would be easier to use a module.

The whole scheme would benefit from an Allocation system to ensure that key names don’t clash. Prefixing key names with an app name and a slash would help, of course.

ChrisH – I think that some of the settings that Dave is talking about are ones that are needed before the boot disc is defined and accessed. Stuff that works off !Boot.Choices settings don’t need this, but there’s a lot that needs to be defined earlier in the startup process. This is more for extending *configure options.

One problem is how to define where ‘settings/txt’ resides. You need a ‘cmos’ settings to define the boot filesystem. On a machine with firmware and rom in a defined place it can boot either from SDFS, SATA or SCSIFS (for example). I think the *configure options need a bit of pruning. Many things would be better as a configure plug-in in !Boot.

One problem is how to define where ‘settings/txt’ resides.

Dave has already said it needs to go in the Boot Loader partition, and capitalised it appropriately as SETTINGS/TXT. Presumably the first so many bytes will contain the existing information which is currently read, IIRC, from the end of the rom image, but the file after that can contain additional settings required before RISC OS booting occurs, possibly the EDID info, and anything else people want to add using the scheme Dave outlined.

I withdraw the suggestion that any application settings should be in there. That’s the province of the Boot Choices system.

As to how the contents could be accessible during the booting process: I discovered a long time ago that it is possible to issue more than one fatload command. (I assume that there are still fatload commands – I’m not near a machine to look.) So we could assign a RAM area for the SETTINGS/TXT file to be fatloaded into, and the results could be read from there. Since we don’t seem to be short of RAM, I’d suggest 16 kiB (“no-one will ever need more than…”!)

The RAM area wouldn’t need to exist after boot-up, AFAICS; from then on you’d access the information using my proposed RM.

One thing to be aware of: the file moves with the SD card. Some have argued that this is a Bad Thing. The trouble is that there is no perfect solution for more than 256 bytes of non-volatile storage accessible before boot-up. This one has the benefits of being cheap, easy, and pretty versatile.

If, as it really has to be, it’s in the boot partition, people will inevitably be editing it on different machines sometimes. Which means you’ll probably want to be tolerant of different line endings in text files, as some editors change the line endings to suit – yes, that old chestnut! You’ll also need to consider the encoding you’re going to mandate. All in all, I think it would be a good move.

Would it be useful to have a module that allowed writing and reading key=value pairs to and from a file?

Doesn’t the NC NVRAM module do something like this from a type of NVRAM; maybe RISC OS could benefit from a simple bit of code to read the FAT partition to load in the settings file from the DOS bit…
[the ephemeral problem of how to load settings before the filesystem is active]

Presumably the first so many bytes will contain the existing information

I originally thought of doing it that way, but there are a couple of problems:

• it would no longer be a text file;

• I see no good reason to make it replace the real CMOS that some people have bought.

To have a mixed binary/text file isn’t an insuperable difficulty, but it certainly makes it difficult to edit on any other platform.

Doesn’t the NC NVRAM module do something like this from a type of NVRAM

Do you have any more information? Do you have some code?

If, as it really has to be, it’s in the boot partition, people will inevitably be editing it on different machines sometimes. Which means you’ll probably want to be tolerant of different line endings in text files

Good point.

I’m not sure how best to denote EOF, as the data content will be of very variable length, and we have to give any easy mechanism to prevent really simple readers from reading past the end of the real data. I’m thinking of a file format that permits no comments (there is limited space), no empty lines, and EOF is denoted either by an empty line or by a line containing “EndOfFile”.

Also, I’m thinking of a module that writes without altering the case, performs lookups case INsensitively, and returns values without altering the case.

Should there be a HAL function to read from this file?

Doesn’t the NC NVRAM module do something like this from a type of NVRAM

Do you have any more information? Do you have some code?

https://www.riscosopen.org/viewer/view/castle/RiscOS/Sources/HWSupport/NVRAM/

I almost added that module to the Iyonix ROM when I was adding the configuration options for the NVidia driver, until I realised that the PCI module already assigns CMOS bytes for PCI cards (reusing the CMOS locations for the unused Podule slots).

There are a couple of issues with the module in its present form which will make it tricky to integrate with existing machines (even those which already have >256 bytes of CMOS). I think I fixed those issues in my local copy of the module, but since I didn’t actually need to use it the code is still sitting on my machine, completely untested. I also have a copy of the functional spec that’s alluded to in the docs – I’ll try and get that submitted tonight, at the least.

Presumably the first so many bytes will contain the existing information

I originally thought of doing it that way, but there are a couple of problems:

• it would no longer be a text file;

• I see no good reason to make it replace the real CMOS that some people have bought.

OK, then SETTINGS1/DAT and SETTINGS2/TXT – or even omit the numbers! That way we keep the original and add an extra file in text format. This means the code to load the original already exists and needs little alteration, and you get your new editable section for the EDID stuff and whatever!

I’m thinking of a file format that permits no comments (there is limited space), no empty lines, and EOF is denoted either by an empty line or by a line containing “EndOfFile”.

What is the space-limiting factor? Surely we have plenty of space in a boot partition for a file of this kind of size. And we have plenty of RAM to read it into if you need to read the whole thing in at once? Unless there isn’t much scratch RAM available early in the boot process because there’s no way to claim blocks of memory? Is that the problem?

All in all, I think it would be a shame to have a human-readable file, but preclude the use of comments.

It’s not really a space limitation. I’m proposing a file format that is machine editable – think of Configure plug-ins that might want to edit it. How do you associate comments with active lines? If a key=value pair is deleted, how do you ensure that any related comments have been deleted? You as a human being reading the files will understand it (probably!) but it would require serious AI for a machine to do it.

The uses for the file I’m talking about are rather limited. I think you’re thinking of choices files that relate to one application. But even those choices files that are machine edited may have no or very limited comments in them.

I discovered a long time ago that it is possible to issue more than one fatload command. (I assume that there are still fatload commands – I’m not near a machine to look.)

Beagle, yes.
And on a Pi?

To have a mixed binary/text file isn’t an insuperable difficulty

Why? Surely that would defeat the purpose of having a text version?

I’m not sure how best to denote EOF, as the data content will be of very variable length, and we have to give any easy mechanism to prevent really simple readers from reading past the end of the real data.

Won’t the FAT directory entry provide the file length?

If a key=value pair is deleted, how do you ensure that any related comments have been deleted?

Maybe better that data is not deleted but rather just commented out. Benefits? The config can parse the previous value and use that as a suggested value, plus humans tweaking the file can look at the options that are commented out and enable them (which is much nicer than thinking “what option do I need to do meh?”.

Everything should be a keyword=parameter line, which should match up with the existent configuration. It should, preferably, be case INsensitive.

As for how to handle the use of existing CMOS RAM dongles, therein lies the big question. The ultimate outcome really ought to be that RISC OS reads from a settings file on the boot partition (and reverts to other methods in the absence of a boot partition, etc). “Going forward” it may be prudent to lose the dependency on old restricted settings in CMOS.

And on a Pi?

Ah. Yes. Huge fly (more pterodactyl-size) in the ointment. It has a completely different way of booting.

So my method would work on BeagleBoards but not on RPi.

I wonder where I can find the list of acceptable commands for the RPi CONFIG/TXT?

Hmm. Does anyone know if initramfs (or ramfsaddr plus ramfsfile) do anything more than just load the given file at the given address?

Similarly device_tree_address and device_tree?

Having searched a little, I’m surprised that no address is specified in CONFIG/TXT to load the kernel at. Does that mean it defaults to load at 0?

I would be quite happy to try initramfs and device_tree, but what address would be safe and sensible to give them, and where would I expect them to be visible via RISC OS? Or does memory management prevent access by relocating it once the OS is up and running?

I wonder where I can find the list of acceptable commands for the RPi CONFIG/TXT?

http://elinux.org/RPiconfig

Having searched a little, I’m surprised that no address is specified in CONFIG/TXT to load the kernel at. Does that mean it defaults to load at 0?

According to http://elinux.org/RPiconfig there’s a kernel_address parameter, although it doesn’t list the default value.

Edit: The other Chris snuck in while I was still reading :)