Anyone used both cores on OMAP4?

Hi,

Just wondering whether anyone had yet managed to write a piece of code which used both cores of the Cortex-A9 under RISC OS? I don’t mean ‘multi-threaded RISC OS’, or ‘finished product application’; I mean a quick and dirty demo of code running simultaneously in both cores and giving a result (compared to a single core doing the same).

Spent a bit of time today looking at the OMAP4 document, the Cortex-A9, MPCore documents and was trying to work out what the needs were to achieve this. It looks like the OMAP4 boot prior to loading the RISC OS kernel should have initialised both cores – but is there a way to read/write to the SCU registers, or is memory mapped in such a way that code can be passed to the second core?

I’m not sure of the specifics of how to breathe life into the second core, but I do know the basics:

1. You need to tell the CPU the physical address of the code to execute (via the SCU? via secure monitor calls to the OMAP4 boot ROM? via CP15 ops?)
2. The second core will start up with its MMU disabled, and so will theoretically have access to all memory. Enough for you to do some basic “hello world!” type stuff, but will be inadequate for any real work (you’ll be stuck with the MMU off, and to preserve your sanity you’ll have to make sure any work buffers received from the host core are physically contiguous)
3. So you’ll want to get some code in there to enable the MMU and caches. Copying the relevant code from the kernel sources might make sense – Kernel.s.HAL has the basic startup and MMU enable code, while Kernel.s.ARMops has all the cache maintenance stuff.
4. Once you’ve got the MMU on and some way of passing messages between the two cores (hardware FIFO? special memory page?) you’ll want to add some routines to allow you to map and unmap memory so that the host core can pass arbitrary workloads to the slave core. If the memory is to be mapped on both cores at once then that might involve some tweaks to the page table flags to mark it as shareable (not sure offhand if that applies to all memory or just cacheable). Plus you’ll also need to deal with Service_PagesUnsafe/Service_PagesSafe which may fire from time to time if the OS decides to swap the physical page allocation around.
5. Since your startup code will be running with the MMU + caches disabled you’ll probably want to use PCI_RAMAlloc to allocate the required memory – that way it’ll be physically contiguous for the slave core and uncacheable for the host core (so no need to worry about flushing it from the cache before starting the slave)
6. For any private memory needed by the slave core it’s probably best to get the host to allocate it using PCI_RAMAlloc. This will obviously result in a redundant mapping in the hosts address space, but is the only real option until I find the time to work on proper physical memory allocation (or until we make the kernel multi-core aware!). Or you could start modifying the OMAP4 HAL so that it only tells RISC OS about 50% or 75% or whatever of the available RAM and reserves the rest for the second core.

Just out of interest – how does one get the processors to communicate with each other? Is it possible for one core to IRQ the other?

Just out of interest – how does one get the processors to communicate with each other? Is it possible for one core to IRQ the other?

There are three basic methods of detecting communication:

1. IRQs (if the hardware supports it)
2. Events (SEV to broadcast an event to all cores, WFE to wait for an event)
3. Polling shared memory/resources

But those are just ways of detecting that another core is trying to communicate with you – they don’t convey any data. The data itself will have to be sent via special hardware (inter-core FIFOs) or via a memory buffer whose address is known to both cores.

Polling shared memory/resources
But those are just ways of detecting that another core is trying to communicate with you – they don’t convey any data. The data itself will have to be sent via special hardware (inter-core FIFOs) or via a memory buffer whose address is known to both cores.

Would a simple method be to allow an overlap of memory and have an area that processor 1 saw as write only and processor 2 saw as read only? Write area A with processor 1 read area A with processor 2 and have processor 2 write a simple ack to area B – processor 1 can now clear the area A content or write new. Data passing the other way would use assigned memory areas C and D.
Poll loops at each end check for the existence of data. No doubt slicker with IRQs or events

Yes, that approach would work quite well for AMP systems. For example it’s basically the same approach that VCHIQ uses to talk to the Pi’s GPU.

For SMP systems, although I’m by no means an expert, I think a centralised message queue would only be needed by the kernel, to notify the other cores of important events which need immediate attention (e.g. Service_PagesUnsafe). And since that queue is important you’d want to make sure you use something IRQ-based to get the attention of the other cores. Everything else can be done using mutexes, semaphores, etc. dotted all over the place in memory, which is probably where events come into play. E.g. when core 1 tries to lock a mutex which is held by core 2 it enters a loop where it calls WFE and then polls the mutex status. As part of the mutex unlock function, core 2 will make a call to SEV to speculatively wake up any other cores which might be waiting on it. In a real implementation things would be a bit more complex (core 1 would want to check if any other threads are runnable before it goes to sleep, and you might want to use a flag on the mutex to avoid spamming SEV unless it’s known that another core is waiting), but I think that’s the basic idea behind how SEV/WFE are intended to be used.