valgrind 'memcheck', for ROM

On Sunday I spent a fair bit of time investigating an uninitialised variable problem, all the while wishing that we had a tool available that was similar to valgrind’s memcheck feature. But instead of checking just applications, we’d want tool(s) that are capable of checking the following:

• Memory accessed by the kernel during ROM initialisation – to track down any reads that target uninitialised memory
• CMOS accesses prior to the keyboard scan being completed (a bit unrelated to RAM checking, but still important for making sure that the system can always be recovered)
• Access to uninitialised pages (an extension of the ROM init checking – essentially we just need a hook in the kernel’s memory manager so that when a page is added to a DA, it gets flagged as uninitialised)
• Access to uninitialised heap blocks (as above, add a hook to OS_Heap so that memory regions will be marked as uninitialised on alloc / free), and any other similar things (e.g. the kernel’s “chocolate” blocks)
• And of course valgrind checking for apps would be nice too (just hook into CLib’s heap manager)

The first point could most easily be done by modifying an emulator (most likely RPCEmu, although that would limit us to testing IOMD ROMs). The second could just be done by adding debug output to the kernel (and could safely be left enabled for all dev builds). The remaining items could potentially be done as a runtime thing on any RISC OS machine, if we had an abort handler that’s smart enough to be able to trap & emulate all memory write accesses – but that would be a lot of work, so it might make sense to start off by building it into an emulator again (could easily use MCR instructions to communicate to the emulator that memory region(s) should be flagged as uninitialised)

Also:

• A tool to check for misuse of OS_SynchroniseCodeAreas (or more specifically, the underlying MCR ops). I.e. it will complain if an attempt is made to execute an instruction which hasn’t been flushed from the cache via a suitable op. (another job for a modified emulator, most likely)

If we had hypervisors which could run on the modern boards then it might be possible to use those to implement low-level error checking logic which would have otherwise been implemented via emulation. That way we wouldn’t be limited to just testing the platforms that RPCEmu/QEMU/etc. support. Essentially the hypervisor will be running the emulation layer, but because the hypervisor is separate to the OS it would avoid us needing to make large changes to the OS itself (changes which might affect the validity of the testing).

But instead of checking just applications, we’d want tool(s) that are capable of checking the following:

Sounds like an extension of a system log utility in that the log lists success and failure as well as the detail of what actually happened.

Making sure the test results are visible to the user/developer is certainly important – but the hard part here is the bit which actually does the testing :-)

I was working on the basis that the result logging element was the shell and the test elements were subsidiary. With the right sort of log facility you just chuck the test info at the log facility and let it format and store things. Back to the old chestnut of system wide logging really.

What you’re after isn’t easy to implement in a generic form, due to the differing feature set in every ARM core.

Outside of running the OS under full emulation you’d want to run the OS at USER level and emulate or Hypervise all LDR, STR, LDM, STM, MCR, MRC etc, essentially a full type 1 Hypervisor that’s generic enough to run on ARM600 and newer if you want to check all OS build versions.

As a type 1 Hypervisor is a lot of work for all the ARM versions the OS supports, the next step down would be to again run the OS at USER level with an Abort handler that emulates all relevant instructions. The key difference from a Hypervisor being its not aware of hardware devices, so has a lot less to implement.

If speed isn’t an issue then the simplest option is to breakpoint every instruction, examing what it’s about to do before letting it execute…having breakpointed the next instruction that’s going to execute.

Where things get interesting is around handling IRQ/FIQ, as you need to be handling them quicker than they can generate to avoid IRQ races.

What you’re after isn’t easy to implement in a generic form, due to the differing feature set in every ARM core.

Hence it being a wishlist item :-)