Making ROM writeable?

Is this possible?

While I have had things I’d like to have done in the past; at the moment I’d like to poke around the Filer_Boot code. For example, to remove the WimpTask OSCLI SWI call in the Filer to see how that alters behaviour.

I could softload a Filer, and spend forever trying to get everything else to talk to it.
I could make the change and build a new ROM image. A lot of work for a minor tweak.

It would be much simpler if, for advanced hacking, it was possible to directly tweak the RISC OS code in situ. How would this be possible?

I have tried OS_Memory,14 on what I believe is the ROM base address (&FC000000) and it returns without an error, but all reads (using *Memory on the logical address returned) are zero. I’m guessing the ROM address is logical, so who knows what actually got mapped in. ;-)
OS_FindMemMapEntries returns -1 for the addresses &FC000000 (ROM base) and &FC225000 (page containing start of RMA).
I’m having trouble working out how to call RISCOS_LogToPhys. There doesn’t seem to be a SWI for this (the traditional OS memory SWIs specify that the address must be in RAM…).

Any ideas?

I don’t think there’s any sane way of getting the physical address of the ROM, short of directly peeking at the page tables. OS_FindMemMapEntries/OS_Memory 0 won’t really help you since they only deal with things which are part of the standard RAM pool, which the soft ROM is not. The way I usually deal with this is to do a custom ROM build with the kernel tweaked to make the ROM pages writable, but you don’t seem keen on that idea :-)

Why not just *RMFaster the Filer and poke at it that way? I think everything will continue working correctly, you’ll just have to re-open any filer windows you were using. Although to really work out how removing the WimpTask affects things you should be building a custom ROM, or softloading a new version somewhere near the start of the boot sequence.

Not an expert in this, but here’s a curve-ball thought. What about doing this via an emulator? RPCEmu could be altered so you could manipulate the ROM via the emulated image itself. That way it doesn’t matter what the SWIs let you do or not do.

short of directly peeking at the page tables

Mmm… 02020001 02020401 02020801 02020C01 … I think I’m going to have to dig up my ARM ARM to work that out.

The way I usually deal with this is to do a custom ROM build with the kernel tweaked to make the ROM pages writable,

You mean this bit?
https://www.riscosopen.org/viewer/view/castle/RiscOS/Sources/Kernel/s/Attic/HAL?rev=1.1.2.40#l725
[edit: perhaps line 1212 is better?]

BTW, if the HAL code is used in the Pi image (etc), how come it’s in the Attic?

Why not just *RMFaster the Filer and poke at it that way? I think everything will continue working correctly, you’ll just have to re-open any filer windows you were using.

Filer icons disappear. Doing RMFaster SDFSFiler (etc) doesn’t bring them back. I remember this from the days of RISC OS 2/3, upset the Filer and you’re kind of stuck. Oh, and trying Filer SWIs says “Use *Desktop to start Filer”. Poking read-only memory is going to be simpler than sorting that out. ;-)

Although to really work out how removing the WimpTask affects things

Was just to enable/disable to see what affect it had. I’m guessing the “second run works” is because the Filer knows about the app, so no WimpTask is issued. Just wanted to confirm that this was the case.

By the way, small-tiny thing I noticed along the way:
https://www.riscosopen.org/viewer/view/castle/RiscOS/Sources/Kernel/s/MemInfo?rev=4.4.2.24#l685
Why the B %FT97 when label 97 is the following line? Wouldn’t a comment like “; and fall through to...” be better?

Ooooh boy.

Level 1 page table is read by:
SYS "OS_Memory", 16+(7<<8) TO ,base%,size%,used%

ROM start address is &FC000000.
This is split up as bits 31-20 are table index, bits 19-0 are section (for section) or bits 19-12 are second table index (with bits 11-0 are a page index).

So, &FC000000 >>> 20 is &FC0. Add that to the base, &FB400000.

*Memory FB400FC0 is &0DAC2001, which in binary is %00001101101011000010000000000001.

Tail end bits are %01 so it is a coarse table descriptor. Bits 31-10 are the upper bits of the address (&0DAC2000) with bits 19-12 of the original address being 9-2 of the coarse address. Since these bits are zero, we can ignore this part for now and just look at address &0DAC2000.

And here I grind to a halt. Address &0DAC2000 throws an abort on data transfer. This is a physical address, right? Using OS_Memory to try to map in that as a physical address just gives a lot of null bytes. Hmmm… Something’s not right – there ought to be a table there…

This stuff makes my head hurt. I’m going to go listen to Kalafina, come back to it another time.

Filer icons disappear. Doing RMFaster SDFSFiler (etc) doesn’t bring them back.

You’ll need to do a *WimpTask Desktop after the RMFaster to restart the Filer’s Wimp task.

Is it worth mentioning the ROMPatch module? I do not think it applies to RO5, but I played around with it once with RO4. It is part of the !Patch application, which dates back to RO3, but the version I have is 0.15 13 Apr 2004.

You mean this bit?
https://www.riscosopen.org/viewer/view/castle/RiscOS/Sources/Kernel/s/Attic/HAL?rev=1.1.2.40#l725

Yep.

[edit: perhaps line 1212 is better?]

1212 isn’t really relevant since I don’t bother compressing ROMs during development.

BTW, if the HAL code is used in the Pi image (etc), how come it’s in the Attic?

It’s a quirk of how CVS handles branches. Any files which don’t exist in the trunk will be shown as being in the attic, even if they still exist in other branches.

By the way, small-tiny thing I noticed along the way:
https://www.riscosopen.org/viewer/view/castle/RiscOS/Sources/Kernel/s/MemInfo?rev=4.4.2.24#l685
Why the B %FT97 when label 97 is the following line? Wouldn’t a comment like “; and fall through to…” be better?

It’s just using a branch for consistency with the other cases, I think.

So, &FC000000 >>> 20 is &FC0. Add that to the base, &FB400000.

Bzzt! Wrong.

It’s true that you want the top 12 bits of &FC000000, but you want to end up with a word-aligned address, i.e. you want &FB400000 + ((&FC000000>>20) << 2) = &FB403F00

Tail end bits are %01 so it is a coarse table descriptor.

Coarse table descriptor? How old is the ARM ARM that you’re using? For the Pi you’ll need to be looking at one which describes VMSAv6 (or above), as that’s the page table format we use on ARMv6+.

Is it worth mentioning the ROMPatch module?

Works by peeking & poking the page tables and CAM in order to find a spare RAM page and map it in over the relevant bits of the ROM. But doesn’t understand VMSAv6 so might not be very useful here.

The softload tool also peeks at the page tables to locate the ROM, but that doesn’t support VMSAv6 yet either.

1212 isn’t really relevant since I don’t bother compressing ROMs during development.

Okay, good to know that I can forget about that bit.

Just as an aside – does ROM compression give worthwhile gains? The SD access isn’t slow, so just loading a regular image may be slower than decompressing in memory, but is this something the user will actually notice?

It’s true that you want the top 12 bits of &FC000000, but you want to end up with a word-aligned address, i.e. you want &FB400000 + ((&FC000000>>20) << 2) = &FB403F00

Ah, yes. << 2 to word align. Duh, how did I forget that? And, ba-ding! There’s five words of data there. Time for a rummage. :-)

Coarse table descriptor? How old is the ARM ARM that you’re using?

It’s the Big ARM ARM, second printed edition. So, yeah, it is eons old. I have a PDF version dated 2005 (describes VMSAv6) that I downloaded from some random place (ARM’s site wanted a complicated password with numbers and letters and stuff and I don’t remember it, so it was easier just to look elsewhere – somebody ought to point ARM to this: http://xkcd.com/936/ ).

At a brief glance, not much has changed as far as we are concerned. There’s a nice feature that the ARM can use a fixed Translation Table for I/O and OS stuff, and (an)other one(s?) that change(s) on application paging – could give some gains there, but I didn’t notice any reference to this sort of behaviour when looking at the code yesterday.
[note to self: start at B4.7.2 p724]

Um… “If bits[0:1] == 0b01, the entry gives the physical address of a coarse second-level table,” and section B4.7.7 heading is “Second-level descriptor – Coarse page table format”. What version of ARM ARM are you reading that it says something different?

Okay, so CP15:C1#0 is &C51A7F (determined by MRC CP15,0,R0,C1,C0) which is:

%110001010001101001111111
                        1 - MMU on
                       1  - Strict alignment
                      1   - L1 unified data cache (or always on)
                     1    - Write buffer enabled
                  111     - SBO (?), reads as %111
                 0        - Little-endian
                0         - System protection (ARMv6 deprecated)
               1          - ROM protection (ARMv6 deprecated)
              0           - F (implementation defined)
             1            - Branch prediction enabled
            1             - L1 instruction cache enabled (or always on)
           0              - Normal vectors (&00 - &1C)
          0               - Normal cache replacement strategy
         0                - Deprecated in ARMv6
        1                 - SBO?
       0                  - Always zero
      1                   - SBO?
     0                    - Always zero
    0                     - Should be zero
   0                      - All performance features enabled
  1                       - Unaligned loads/stores permitted and act "correctly"
 1                        - Subpage AP bits disabled
                          - [some other stuff, it's all zeroes, so unimportant ;-)]

The important one for us is the XP bit, as this changes the descriptor layout.

The word at &FB403F00 is &0DB0880E which is %00001101101100001000100000001110. As the XP bit is 1, the tail is %1X so it is an Extended Small Page descriptor (4K page). [edit: NO IT ISN’T – I’m looking at the L1 table, duh!]

The descriptor itself breaks down as:

%00001101101100001000100000001110
                                0 - XN bit (means this memory is executable)
                               1  - Extended Small Page
                              1   - B
                             1    - C
                           00     - AP
                        000       - TEX
                       0          - APX
                      0           - S (memory not shared)
                     1            - nG (marked as process specific (ROM not global?!?)
 00001101101100001000             - Base address (&0DB08)

Using *MemoryA to paste in AP=<anything non-zero> hangs the machine. Ditto if APX is set. So… I dunno. Maybe you simply can’t use the Debugger to mess with the page tables on a live system? ;-)

Well, if I can’t tweak the memory access permissions… unlike Alex Salmond, I actually had a Plan B.

I am inclined to believe the address specified, as the startup says:

HalStartup
HalQueryPl
0E100000 007F8000 HalQueryPldone
0E100000 HalStartup2
00008000 00000000 0E000000 ROM start, RAM start, RAM size
ROM relocated
HalStartup3 .. rst
rend
00000000 0DB00000 HalStart from OS
0E000000 02000000 F7000000  HAL Init completed
HAL initialised

BINGO!

The start of the RISC OS code image can be seen at &FC010000 where the first word will say “OSIm”.

Calling SYS "OS_Memory", 14, &0DB08000 TO ,,laddr%,key% will map in a megabyte of physical memory and give it a logical address. In my case, the address is &FAE00000. Now that we know the offset to the “OSIm” header is +&10000 (64K further on), we can try dumping &FAE10000… and there it is.

Now, as old hands with the Wimp, we might have the value &4B534154 burned into our minds. Unless you are a C coder where a library hides the mechanics from you. ;-)

Well… Um…

*MemoryA FAE10000
+ FAE10000 : OSIm : 6D49534F : STCVSL  CP3,C5,[R9,#-316]
  Enter new value : 4B534154
+ FAE10000 : TASK : 4B534154 : BLMI    &FC2E0558
  Enter new value :
Escape
*Memory FAE10000 +4
Address  :     3 2 1 0   [...snipped...]  :    ASCII Data
FAE10000 :    4B534154                    : TASK

And, finally, the moment of truth…

*Memory FC010000 +4
Address  :     3 2 1 0   [...snipped...]  :    ASCII Data
FC010000 :    4B534154                    : TASK

Howzat?, as they might say in cricket.

I think we can all agree that this is scarily ass-backward, but I have successfully altered a word in “ROM” on a live system using nothing more than a TaskWindow, BASIC, SciCalc (for the binary conversions) and the Debugger (because it does it in SVC mode).

I’m pleased with this. I can go watch some animé now. And maybe try to remember why I wanted to do it in the first place. [ought to write a blog entry on ARM/RISC OS memory hacking for fun and freakery].

Many MANY thanks to Jeffrey for pointing out the word aligned bit. So obvious, now I know. ;-)

Just as an aside – does ROM compression give worthwhile gains? The SD access isn’t slow, so just loading a regular image may be slower than decompressing in memory, but is this something the user will actually notice?

Not all SD interfaces or bootloaders are created equal. The BeagleBoard (at least with the version of u-boot I was using at the time I developed the code) only averages about 900KB/s in u-boot when reading from SD. So cutting a 5MB ROM down to ~2.2MB can make an appreciable difference to boot time.

For the Pi, I think the emphasis was more on using the compression to cut down on the disc space required (to allow RISC OS ROMs to be squeezed in alongside Linux kernels in multi-boot scenarios).

There’s a nice feature that the ARM can use a fixed Translation Table for I/O and OS stuff, and (an)other one(s?) that change(s) on application paging – could give some gains there, but I didn’t notice any reference to this sort of behaviour when looking at the code yesterday.

Yeah, that’s one of the things I’d like to have a play around with at some point. I’m not sure how much performance it would give us (there’s also lots of stuff hidden in there suggesting that we can skip cache/TLB maintenance for some ops), but it would actually fit quite well with RISC OS’s memory map.

Um… “If bits[0:1] == 0b01, the entry gives the physical address of a coarse second-level table,” and section B4.7.7 heading is “Second-level descriptor – Coarse page table format”. What version of ARM ARM are you reading that it says something different?

The ARMv7 ARM refers to them as “0b01, Page table. The descriptor gives the address of a second-level translation table, that specifies the mapping of the associated 1MByte VA range.”

The word at &FB403F00 is &0DB0880E which is %00001101101100001000100000001110. As the XP bit is 1, the tail is %1X so it is an Extended Small Page descriptor (4K page).

Bzzt! You’ve wandered into the docs for the L2 descriptors. For the L1 page table, bits 0-1 of %10 and bit 18 of %0 corresponds to a section (1MB) mapping, with a (drumroll) base address of &0DB00000.

Bzzt! You’ve wandered into the docs for the L2 descriptors.

Yup. I see that now.

I’ll tell you what. I’m going to print this chapter out instead of scrolling up and down twenty or so pages on a PDF on an iPad. Prolly shoulda done that in the first place… ;-)

Okay, my bad. Read through the thread while on break and, yup, &FB403F00 is the first lookup so is a Level1 table. I thought I’d already looked at that and this was the second lookup.
Thanks for the clarification.

We have reached our destination.

Here, for your enjoyment, is Plan A.
It is incomplete (very little sanity checking, assumes L1 pointer, only deals with the 1st megabyte). Fixing it is an exercise for the reader. The code is commented profusely.

DIM code% 64

FOR l% = 0 TO 2 STEP 2
  P% = code%
  [ OPT   l%
    SWI   "OS_EnterOS"    ; enter SVC mode
    LDR   R2, rom_check   ; load check value
    LDR   R1, rom_l1_ptr  ; load pointer to table
    LDR   R0, [R1]        ; load value from table
    CMP   R0, R2          ; check is expected value
    EOREQ R0, R0, #&8000  ; OK = disable APX bit (15)
    ORREQ R0, R0, #&0C00  ; OK = set both AP bits (10,11)
    STREQ R0, [R1]        ; OK = write back updated value
    SWI   "OS_LeaveOS"    ; back to USR mode
    MOV   PC, R14         ; done.

  .rom_l1_ptr
    EQUD  &FB403F00       ; hardwired to RO5.12/Pi
                          ; should use OS_Memory and calc it
  .rom_check
    EQUD  &0DB0880E       ; what SHOULD be there...
  ]
NEXT

CALL code%

And to test…

*Memory FC010000 +4
Address  :     3 2 1 0   [...snipped...]  :    ASCII Data
FC010000 :    6D49534F                    : OSIm
*MemoryA FC01000
+ FC010000 : OSIm : 6D49534F : STCVSL  CP3,C5,[R9,#-316]
  Enter new value : DEADBABE
+ FC010000 : ¾°–Þ : DEADBABE : MCRLE   CP10,5,R11,C13,C4,5
  Enter new value :
Escape
*Memory FC010000 +4
Address  :     3 2 1 0   [...snipped...]  :    ASCII Data
FC010000 :    DEADBABE                    : ¾°–Þ

At &FB403F00 are five words (*Memory will show you). You can do this quickly to test using *MemoryA and changing the “880E” at the end to be “0C0E”. Repeat for each of the entries.

To test? Enter PRINT ~!&FC013AD0 and if the reply is “43534952” then enter !&FC013AD0=&4B434952 and then type in *Help Commands and when it is done, go up to the top of the output and read what the OS is called. (^_^)

If you would prefer to have the ROM writeable only in SVC mode, set the AP bits to %10 instead of %11. This means, instead of “C00” at the end, it should be “800” [in other words, just disable the APX bit].

Have fun!

Hi Rick,
I’ve just tried the above and it didn’t work for me straight away.
On further investigation, the rom_check value is different for me.
I was using 256Mb RPiB and read a value of &0BB0880E at &FB403F00.
Change the value of rom_check to suit and it works.
This led me try different hardware and firmware.
512B 21JUN14 &1BB0880E
256B 21JUN14 &0BB0880E
256B 20AUG14 &0BB0880E

512B+ 04JUN14 &1BB0880E
512B+ 04JUN14 &1DB0880E

The last two are same hardware, but different SD cards, probably not vanilla.

The table addresses and ROM size can be read using OS_Memory. That is your homework… ;-)

It’s not the table address or ROM size that’s the issue, it’s your hard-coding of the expected physical address, as that will differ depending on the machine type & amount of RAM (and for the Pi, how much has been reserved for the GPU).

If you mask out the top 12 bits of the L1PT entry when validating it (so that you’re only checking the entry type & flags) then you’ll have a solution that works with all ARMv6+ machines – at least until we decide to change the ROM or L1PT logical addresses, or make the HAL + ROM disjoint in physical space (forcing page mapping instead of section mapping), etc.

Also, technically you need to flush the TLB after altering the page table entry.

it’s your hard-coding of the expected physical address

I said it was very unfinished. ;-) The use of the address check was to abort if anything unexpected was found in lieu of having code that actually calculated these addresses in preference to just saying “it’s here”.

I think we need to (note – all reads/writes in SVC mode!):

• Read CP15 to get the CPU type and configuration. This is so we can determine what sort of address mapping is in use (ie pre-ARM6 hardware), and also so we can reject stuff that we don’t understand before we break things. ;-)
• Work out where the ROM is mapped in. This appears fixed at &FC000000 (Pi, playing with different GPU allocations) but it doesn’t hurt to get the address of UtilityModule and mask out the lesser three bytes (&FC01F508 → &FC000000).
• Work out where the L1 table is. OS_Memory 16+(7<<8) returns this in R1 (with R2=16K as a sanity check). This is &FB400000 on a 256MB Pi regardless of whether the GPU is using 32, 64 or 128 MiB. As it is a virtual address, it seems that this may be the address of it (it’s the same on RPCEmu on RPC-era hardware). However, there’s a call to read the address, safer to use it and not assume anything.
• Work out the length of the ROM. OS_Memory 8+(5<<8) will return the count in R1 and the size in R2. It lies and reports as if it was 4KiB pages instead of 1MiB sections. Not to matter, R1*R2 gives the total size. ((R1*R2)/(1024*1024)) replies 5. Note – we’re asking for the SoftROM so this will be a fail point if this is some sort of ‘real’ ROM – for instance RPCEmu returns a count of zero.
• Work out an effective address for the ROM in the table. That’s this part: &FB400000 + ((&FC000000>>20) << 2).
• Read what’s there in the table.
• If the latter two bits are %10, it is a section. If anything else, ABORT. We can deal with twelve hundred and eighty 4K pages some other day.
• The ROM addresses are contiguous, so no need to look them up one by one, just verify that there are five entries at this place in the table and each ends with %10.
• Read them and alter the flags as appropriate, then write them back. Might be a candidate for disabling interrupts here?
• Poke CP15 once more to flush the TLB. This probably “worked for me” because I was running the code in a TaskWindow, I don’t doubt that the Wimp messes with the table and TLB a lot behind the scenes.
• Use OS_Module to look up a module that should be present, but is not likely to be being used. Perhaps “BootCommands”? Get a pointer to the module name and change it to “RootCommands”, verify this worked, then restore the “B”.
• For extra Brownie Points, verify the cases when it does NOT work – for instance when setting the ROM back to read-only, or when setting it to be writeable only from SVC mode. This means dealing with abort on data transfer errors, environment handlers, etc. Loads of fun with BASIC. ;-)
• Done.

I might put something together this weekend if I have some time free.

What size is the RO5.21 ROMs decompressed?

Can a *save rom xxx +??? – be done from a working setup?

Version RO5.21 doesn’t load from the ROMs directory – but softloads with the loader.

Sorry, just noticed this when reviewing the thread.

What size is the RO5.21 ROMs decompressed?

I think there is about 300KiB free from 5MiB. So just say “about five megabytes”.

Can a *save rom xxx +??? – be done from a working setup?

Interesting question. Does any part of the HAL write into itself? In other words, would a saved dump match the image? [and if not! what is different?]

Version RO5.21 doesn’t load from the ROMs directory – but softloads with the loader.

? Are you referring to the emulator?

Modern systems are loaded by a bootloader, either custom (Pi) or UBoot (OMAP, though a smaller custom loader gets that started). As such, the boot process is more complicated than “ROM initially mapped at &0, begin there…”. I’ve not investigated the boot, but I don’t think it just starts executing at the beginning any more, plus the boot loader may or may not poke in some sort of environment data, plus there may be certain assumptions made regarding the system state (for instance, if the image is loaded into RAM, this means the memory has been brought up and is active), plus there may be restrictions on what system resources can and cannot be used (shared RAM with GPU, ditto timers, etc).

T’was simpler in the old days, but then again we didn’t have the luxury of changing the entire OS in a couple of seconds by swapping a little memory card. ;-)

Trying to run RedSq RO 5.21 from the ROM directory – seems that the memory checking – for a real RPC was reinstated to later version of 5.19 – but fails with RedSq – I was wondering if the code for this could be bi-passed.
If anyone has any idea where in the ROM file this would be – it would be nice to know.

Thanks