Let's get started with a Pandora port

Great news on getting the Kernel up and running!! I look forward to seeing the desktop up and running with a game of Meteors on it in no time… :)

Sounds like you’re making fantastic progress, Jeffrey – congratulations on getting the kernel running on real hardware, that’s a major milestone.

Your post does raise a fair number of issues. I’m interested in your analysis of the changes to the VMSA in ARM v6; although I’m not particularly familiar with the RISC OS kernel’s memory handling at this level, I have skim-read old and new ARM ARMs and I can’t see all that much that’s radically different, with the notable exceptions of:

• support for 1K pages (fine page tables) has been withdrawn – but these were new in ARM v5 and RISC OS was never modified to take advantage of them as far as I know, so this is no great loss
• various extensions are made for managing multi-core chips and clusters of cores – but running RISC OS on a multi-core chip would be a major undertaking with wide ramifications, and not something I’d worry about at this stage

You mentioned task switching support, and I’m afraid I’m not sure what you’re getting at. Please tell me if I missed something, but the only things I can see that might fall into this category are:

• Domains. These have been around since ARM v3 (ARM6), and are of limited use to us since they appear to only handle access permissions and not address remapping.
• The Fast Context Switch Extension (FCSE). This has been around since ARM v4T (any ARM7 after the 7500FE, and ARM9 onwards but not StrongARM). If we were to use this, it looks like it would limit the size of the application slot to 32MB and the number of tasks to 127 (in fact less than this because we’ll still need much of the upper part of the memory map for shared memory areas, IO mapping, the OS itself etc). The shared memory between &0000-&7FFF would also really mess this up because it would have to be mapped in at multiple “modified virtual addresses” (MVA) which would cause a big cache coherency headache. I’m really not sure it’s a good idea anyway: one of the main advantages of multiply mapping applications into the same memory space is that you effectively increase the size of the virtual memory – you can allocate address space for each application as big as the physical RAM of the machine just in case it needs it later and not worry about the fact you’ve done so for multiple applications because adding a new application doesn’t chew up another lump of the virtual address space. By enforcing the mapping of all applications to a 32-bit MVA, you lose this benefit.

I’ve been doing some thinking on the subject of how many features the kernel should have to determine at runtime rather than build time…

Historically, people are used to a single ROM image working on all IOMD-class machines, despite the fact that these spanned two different ARM architectures as well as a number of peripheral chipset variations. Technically, you could produce tuned ROM images for each hardware combination that wouldn’t work on the others, but I don’t think users would come to grips with such a move after all this time. However, I would contend that we’ve seen the last socketed ARM chip: at (approx) GHz speeds it’s just not practical to design the sockets unless you have the mass-market of the x86 family behind you. As such, with the exception of the IOMD HAL, I very much doubt that any other HAL will have to operate with multiple CPU architectures, so any given HAL could be mated with an OS image that’s tuned for a particular CPU architecture.

The HAL was designed with the intention that, in the ideal case, a single OS image could serve for all platforms, and every hardware variant could be handled merely by merely concatenating different HAL binaries with the same OS image. This is a lofty goal, and has much to recommend it, not least from the point of view of ease of testing and speed of building multiple ROMs. However, it was conceived at a time when every current ARM core (StrongARM, ARM7 and ARM9) seemed to be converging on ARM v4 or v4T. I feel the use of a single setting of the Machine environment variable was partly a reaction against the over-use of different settings for every STB/NC variant with slightly different GPIO configurations, and I wonder if it’s maybe too simplistic: today you can buy ARM chips based on v7, v6, v5 and probably still v4 (though I believe v3 has finally faded into obscurity). The OS image for the Iyonix has in reality already been tweaked for its CPU architecture via build options to FPEmulator. An OS image tuned for a Cortex chip has potentially far more to benefit from being tuned to the architecture – imagine the effect on SpriteExtend’s JPEG renderer of being coded in vector assembler! I guess what I’m suggesting is that Machine could usefully take values of “v3”, “v4”, “v4T”, “v5TE”, “v7-A” etc.

I suppose the acid test is whether having to do a run-time check of the current architecture will have a significant impact on the time it takes to do a task switch, which will be the most critical MMU operation in daily use. If it does, it should probably be made a build-time switch irrespective of any maintenance or testing headaches it causes.

Another point you raised is the XScale ECC support, which is currently switched off in the kernel. I had to look this up too as I wasn’t aware of it: it seems to correspond to the P bit in the level 1 page table, and it looks like ARM have adopted this XScale extension in v6/v7 (although they call it the IMP bit). However, it seems that this bit is basically a signal fed to the memory controller and is therefore outside the scope of the core: in particular I can’t see that it’s used at all by the IOP321’s memory controller (as used in the Iyonix). It is available in the 80200 XScale chip used in the development boards on which Pace originally ported RISC OS to the XScale, where it’s used to indicate that the page is ECC-protected, so that’s probably when this code dates back to. It also seems to be unimplemented in the PXA range of XScale chips; I think we can safely ignore it.

Another XScale innovation, the mini data cache, does not appear to have been embraced by ARM, so it’s maybe just as well that RISC OS never bent its APIs to accommodate it. If any applications feel the need to use it, they’re obviously already very architecture-specific, so they can probably be left to poke the page tables safe in the knowledge that the format is known to them.

On the other hand, being able to section-map memory might be of use to some developers. If the page table format is going to start varying on different CPUs, I think it would be wise to expose an official API to say “I want this memory to be section mapped”. This would naturally manifest itself as a flag to OS_DynamicArea 0.

Sorry, that was a longer post than I expected it was going to be.

Regarding the fast context switching I believe what Jeffrey is referring to is the introduction of a second translation table base register (in ARM v6). The logical memory map is divided into a lower and an upper portion with translations for these portions is done using one or other of the translation tables. It is then possible to switch one translation table without affecting the other.

An overview of this can be found in the following document:

http://www.arm.com/pdfs/ARMv6_Architecture.pdf

Regarding the fast context switching I believe what Jeffrey is referring to is the introduction of a second translation table base register (in ARM v6).

Yes, that’s the one. Have a look at pages B4-41 and B4-25 in the v6 ARM. With the current HAL kernel the application area ends at 512MB, so you’d program the translation table base control register with N=3, so addresses 0-512MB go via TTBR0 and 512+ go via TTBR1. Then you’d just make sure that all the page table entries above 0×8000 in TTBR0 have the ‘not Global’ (nG) flag set, to ensure that they get tagged with the ASID when they’re loaded into the TLB. The ASID is 8 bits, so the Wimp could support 256 tasks under this method before it starts to run into problems.

I guess what I’m suggesting is that Machine could usefully take values of “v3”, “v4”, “v4T”, “v5TE”, “v7-A” etc.

Yes, that sounds like a sensible approach to me.

I suppose the acid test is whether having to do a run-time check of the current architecture will have a significant impact on the time it takes to do a task switch, which will be the most critical MMU operation in daily use. If it does, it should probably be made a build-time switch irrespective of any maintenance or testing headaches it causes.

I’m more worried about the difficulty in writing kernel code that operates correctly, rather than making sure task switching is fast. If the kernel has to support multiple ARM architectures at runtime then the code becomes a lot more complicated, and there’s a danger it could try to do something that it shouldn’t do for a certain architecture (e.g. relying on the presence of certain flags in the CP15 control register). Being able to limit the kernel at compile time to support only one or two architecture versions would allow rigid compile-time checks to be performed to make sure there’s no chance of it doing anything bad. Although admittedly I think most of my concerns only apply to the startup sequence.

Another point you raised is the XScale ECC support, which is currently switched off in the kernel. I had to look this up too as I wasn’t aware of it: it seems to correspond to the P bit in the level 1 page table, and it looks like ARM have adopted this XScale extension in v6/v7 (although they call it the IMP bit)

‘IMP’ meaning ‘implementation defined’. Hence my worries about it being controlled by a compile-time switch – on one CPU the flag could do one thing, on another CPU it could do something completely different.

On the other hand, being able to section-map memory might be of use to some developers. If the page table format is going to start varying on different CPUs, I think it would be wise to expose an official API to say “I want this memory to be section mapped”. This would naturally manifest itself as a flag to OS_DynamicArea 0.

Also there’s the ‘eXecute Never’ bit, which could be useful for improving RISC OS’s memory protection by preventing code from being executed from certain areas of memory.

Hi,

I am quite impressed by the results i have read in this forum. Does anyone have a short how-to to get up to speed with this port? I have qemu 0.10 on my Debian 5.0 (64bit) machine and sufficient knowledge to do both debugging and programming for this project. So i would like to know the requirements to setup a build of the project and test it in qemu. Also is there an overview of what is done and what needs to be addressed?

Regards,

Jan Rinze.

Excellent!

There isn’t really a complete guide to putting everything together at the moment. So far I’ve been using ZhouYaJin’s qemu-omap3, with some local patches applied. But the maemo version of QEMU (Assuming that’s the qemu 0.10 you’re talking about!) looks like it would be a much better choice since it’s being actively developed. I’ll download a copy today and see if I can get it running. I was running into the limits of what qemu-omap3 emulates, so wasn’t going to recommend using it for further development, but hopefully the maemo version will allow us to develop a few more features before we all get our hands on real hardware.

Regarding compiling the code, I’ve just updated the wiki page with some (mostly) complete steps. Unfortunately after a brief search I couldn’t find any guides on the site for anonymous CVS access, and I’m not enough of a CVS export to guess the right settings, so you’re on you’re own with that. If you get CVS working or find anything else to add then please update the page! I suspect most of the setup steps should/could be split off into a general setup guide on a different page, but again I couldn’t find one after a brief look around (only old guides referring to setup with the source batch archives)

The wiki page also lists the current state of the project. I’m currently working on video support, which should be nearly complete (apart from the fact it doesn’t yet work on real hardware). Next I was going to look at USB support, since it’ll be the most useful feature that’s currently missing. But it looks like that could be a mammoth task since AFAIK the RISC OS EHCI driver is incomplete (which may cause problems for the revision C beagleboard’s extra USB port if it can’t be switched into OHCI moe), and there’s no support at all for OTG ports (which is the only USB interface available on rev B beagleboards). If ROOL have any words of wisdom or master plans on how they were going to overhaul the USB code, then they would be greatly appreciated!

Meanwhile, Uwe Kall is having a go at writing the SD/MMC driver. However he’s held up a bit by the fact that his RiscPC is dead, so the only way he has of independently working on the project is to use a ROM image I’ve supplied to run BASIC on the beagleboard and attempt to write the SD/MMC driver as a module. I’m not sure if he has any plans beyond that, but I suspect the knowledge he’ll gain from writing the SD/MMC driver will allow him to knock out a NAND driver relatively quickly. So unless you have previous experience with writing filesystems (presuambly via FileCore) I don’t think there’s much point in you looking at making a NAND driver. But depending on how reliable QEMU’s emulation is it may be possible for one of us to team up with him and help get a working storage driver, which would make his life a lot easier if he’s unable to get his hands on a replacement RISC OS machine.

So basically I’m not really sure what to suggest that you should start looking at. I think most of the outstanding features require more accurate emulation that QEMU can provide, so unless you’re planning on getting a beagleboard any time soon (hopefully the revision C boards will be released on time at the end of this month) you’ll have to be very careful in your choices.

Actually, there is one (hopefully) simple thing you could look at to get you started – for a while I’ve been thinking that it could be useful if there was a way to wrap a RISC OS ROM image so that it looks like a linux kernel – i.e. so you can use u-boot or any other kernel bootloader to load it. This may not necessarily be useful for running RISC OS on a beagleboard, but it may be useful for dual-boot situations, or for devices where we’re unable to (safely) replace the kernel bootloader (but on which we can replace the linux kernel image). But since this is a fairly low priority thing at the moment, feel free to work on something else instead :)

Looks like the maemo QEMU also needs some patches to work with RISC OS. Also either something’s broken or it just doesn’t play nice with my linux PC, because for the first minute or two after starting the emulator the SDL window will be almost completely unresponsive, and I’m yet to get it to succesfully boot the beagleboard test kernel (since it’s so unresponsive after hitting ‘c’ I can’t even change to different emulator screens or pause it to find out what’s going on).

I’ll have another go with it tomorrow and see if I can get it working properly.

Hi Jan, it is always good to have additional help. So far, I’ve mostly been the ‘Beaker’ of the project, so feel free to send test input for real hardware – i’ll provide you with the error log. In addition to Jeffrey’s reply, the DMA handling that should later be used for USB (mass storage e.g.) as well as audio might be a valuable field to gain knowledge. What do you think, Jeffrey?

In addition to Jeffrey’s reply, the DMA handling that should later be used for USB (mass storage e.g.) as well as audio might be a valuable field to gain knowledge. What do you think, Jeffrey?

Yes, DMA would be a good thing to look at – assuming QEMU emulates enough of it!

I haven’t had any luck getting the Maemo version of QEMU working on my Linux PC yet, but I’ll have a go at producing a RISC OS compatability patch for it and writing up some instructions for how to use it to run builds. There should only be a couple of bits that need patching, so even if I can’t run it reliably I can still be fairly certain the patch is correct.

There’s some good news however – I was expecting an official announcement somewhere for this, but I’ve spotted that the pic of the beagleboard in the page header at beagleboard.org is now of a revision C board, not a revision B. Plus the reference manual has been updated for revision C, and Digi-Key are now listing the revision C boards for sale! (Although the product page linked to from the beagleboard site still lists the revision B specs) So in a couple of weeks I’ll hopefully have a board of my own to work with.

http://search.digikey.com/scripts/DkSearch/dksus.dll?Detail&name=296-23428-ND

I’ve produced a patch for the Maemo version of QEMU, and it almost allows it to boot RISC OS. On my machine it reaches the ‘No keyboard present – autobooting’ message before stopping. This is with a ROM image that boots on real hardware, so either it’s just QEMU being flakey on my machine or it’s a bug. In either case I can’t stop the emulation and find out what’s happening because the emulator is so unresponsive.

I’ve updated the wiki page with information on how to get RISC OS working with the two QEMU versions.

Also, a note about QEMU’s debugging facilities – I’m not sure if the Maemo version is any better, but in the version I used, ‘info registers’ and ‘memsave’ seemed to be the only two debugging commands that did anything useful (or anything at all) on ARM targets (beyond ‘stop’ and ‘c’, of course).

What were you concerned about the state of the EHCI driver? It could definitely benefit from a refresher from more recent NetBSD sources.

Are you sure that this OMAP uses EHCI? Other ones use a core from Mentor – there’s a linux driver for it (obviously!) but I don’t think there’s a BSD one. I’m not sure what that means in terms of licensing. I originally chose the BSD code over the linux code when I did the RISC OS USB modules to avoid licensing issues.

It probably has built in transaction translators anyway, most embedded USB cores do, rather than distinct EHCI and OHCI cores.

I’ll have a look at the datasheet…

Ok it looks like they’ve really gone to town, with a Mentor MUSB OTG core and a Synopsys EHCI/OHCI core. RISC OS will have a much better time with the latter as long as the Pandora platform exposes it properly. There seems to be stuff on the Pandora Wiki about the A socket only doing high speed (as well as some guff about USB 2.0 not being backwards compatible – complete nonsense). I can’t imagine how that can have been designed if it’s true.

It shouldn’t be too hard to get the RISC OS USB stack running on this platform, you’ll just need to add a new method for finding the OHCI/EHCI register set base addresses instead of PCI, which is what it currently does, and route the interrupt a bit differently.

What were you concerned about the state of the EHCI driver?

Specifically, the todo list in NetBSD.dev.usb.c.ehci. Although on re-reading it now I see that I misread item 2 (lack of interrupt isochronous support) as meaning that no EHCI devices would work at all, instead of merely just the interrupt isochronous devices not working. So I guess things aren’t as bad as I first thought (although the lack of USB 2.0 hub support does worry me a bit, if the todo list is correct)

There seems to be stuff on the Pandora Wiki about the A socket only doing high speed (as well as some guff about USB 2.0 not being backwards compatible – complete nonsense). I can’t imagine how that can have been designed if it’s true.

The beagleboard uses an SMSC USB3326C as the transceiver for the host port. I’m certainly not an expert on USB, but everything I’ve read so far suggests that it’s only capable of supporting EHCI mode.

It shouldn’t be too hard to get the RISC OS USB stack running on this platform…

I may hold you to that!

The OHCIModule is already capable of polling the HAL (and PCI bus) for USB hardware, but at the moment the EHCIModule only polls the PCI bus. So I’m going to work under the assumption that all I need to do is add similar HAL support to EHCIModule and make sure any low level hardware bits are initialised correctly. But anything beyond that and I suspect I’ll have to spend some long hours reading through the USB specs (which one of us may end up having to do anyway to get the OTG port working).

I fixed a few bugs in the EHCI code from NetBSD which I fed back to them. Interrupts transfers were definitely working and we never really had working isochronous transfers in RISC OS :( That means that high speed hubs should be working fine – at least they were last time I looked. Maybe the TODO list never got updated…

Adding the HAL support sounds like the right way to go.

The OTG port uses a totally different host controller i.e. not OHCI or EHCI. It’s much simpler being a device than a host (although you have to choose a class spec to adhere to to be of much use).

Uwe, if you need a Risc PC for your development work, let me know. I might have a spare one I could give to you. Do you know which parts from your old one survived? I’m sure I could come up with a reasonably sized HD, a StrongARM, a reasonable amount of RAM and a networking card, but I’m not sure if I have a RISC OS 4 ROM set spare.

I’ve got my own rev. C2 beagleboard now, although the DVI output seems to be broken so I’ll probably have to RMA it. But until that happens I’ve managed to fix the HAL to set up the extra 128MB of RAM on the C boards, and checked it into CVS along with the latest broken video code (which is disabled, you’ll be pleased to hear).

Uwe – Echoing Steffen, I also have my old work Risc PC from Acorn sitting (literally) under the bed gathering dust. It’s pretty beaten up but last time I checked it worked. Even has an early revision ViewFinder in it, if I remember correctly. Whether it still powers up is another issue but it could be used for spare parts; I also have the odd spare PSU or two and a healthy collection of sheets of the stick-on Risc PC square rubber feet :-)

Steffen & Andrew – Thank you both for the offer. I checked what component is dead over the last days (i had no testing work from Jeffrey ;-) and after ripping everything into pieces and replacing and cleaning it part by part there was no remaining error so currently it works again – let’s see how long it keeps running, i’ll backup everything! That brings me to another point – My C/C++ Package is not up-to-date so i sent a few questions about the current and future compatibility of the dev package to Castle and got no answer up to now…so i’m waiting for that too. I wonder if the dev suite will be updated continuously?

A number of people have been having trouble obtaining tools from Castle/Iyonix Ltd recently – we at ROOL are talking to them to see if we can sort this out.

The tools haven’t been updated in a while, but I have just recently been having a look at what would be involved in updating them to make use of v6 and v7 instruction sets. No promises on when anything will come from this though! The toolchain is built from source code including some from third parties such as ARM who have indicated that they are unlikely to grant permission for it to be open-sourced, so if anyone fancies helping out they’ll need to sign an NDA before working on them.

Hmmm… not having a compiler could really develop into a stopper for ongoing development. How much work would it be to change the build environment to generate the RISCOS kernel with gcc? Maybe someone could start a project for an alternative build environment? This would probably add to the number of potential developers. Anybody? (ok, this is off topic somehow for this thread)

Can I point out a bug in the RAM clear code (HAL/s/RAM)? Register v4 is missing from the STMIA instructions so you’re only clearing 28 bytes not 32 as advertised. Cut/paste as appropriate.

Fixed now – cheers!

Also, a brief update on the video code, for anyone that is wondering – I had a look at it today and it looks like the DSI PLL that’s used to generate the pixel clock is failing to lock, even though I’m following what I believe to be the correct programming sequence. Apart from a couple of bugs I’ve already fixed I can’t see anything that I’m doing wrong, so I’m going to try building a small testbed by combining the HALtester with the DSS2 video driver from Angstrom Linux. Then I should be able to strip out all the unnecessary stuff and compare it against the code in the HAL to see where it’s going wrong.

Uwe, I’ll start a new thread on build environments in a second.

Since I’m posting here, I might as well take the opportunity to thank everyone on this thread for the hard work they’ve been putting in so far. Some very exciting things happening here!

Good news, everyone…

First of all, I’ve managed to fix my beagleboard without RMA’ing it. After a bit of examination I spotted what appeared to be a sliver of metal between two pins of the DVI chip, which seemed to be causing them to short each other. After poking it with a suitably small object I managed to clear it out the way and now the DVI output appears to work fine.

Secondly, and probably more importantly, I’ve managed to get the RISC OS video driver working. There’s still a fair amount of testing and cleanup to do, but it should be checked in to CVS within the next day or two.

Let me be the first to congratulate you (on this forum at least)! That is a fantastic achievement, and very good news for all who are hoping this port succeeds!