Orange Pi

If nothing else works, start again.
I killed the tree, grabbed an OMAP3 tarball and messed with the “right” files again.
Although it’s taking forever I’m making changes incrementally, and building between changes to try to catch the source of the breakage.

There’s still something pointing to the OMAP3 HAL that I have to find. Back to the logs I guess.

If there’s a bright side, I noticed OMAP3 uses EHCI. So does the H3.
Changing the machine type to CortexA7 has yielded a few unsupported instruction errors, but it looks pretty easy to deal with.

No progress with the proper build system HAL however I did find one hell of a pitfall to watch out for.

I started fresh as stated before. Went through and edited the files.
Then I changed the name of the root directory of the dev tree from OMAP3Dev to AWH3Dev, went through the motions and went to do a test build with the build tool.
AWH3 was gone! I went through the tree. All the config files I’d made had vanished!
My theories are either quantum interference from near parallel universes or something to do with fat32fs write behind caching.

So I’ve re-made or re-edited the files, and have a build running right now hoping to see it fail because the HAL is kind of empty rather than not being able to find whatever.OMAP3 which has been stopping me dead for weeks.

e: I suspect some of the frustrations with processor type not changing and other quirks like complaining about the modules file pointing to the wrong place are all due to the same core issue. It may be time that I set up an ADFS USB stick for building on, because something just isn’t right.

Then I changed the name of the root directory of the dev tree from OMAP3Dev to AWH3Dev, went through the motions and went to do a test build with the build tool.

I think some of the generated dependency information that’s added to makefiles uses the full paths of files. So if you rename a source tree after doing a (partial) build you might end up with a bunch of “file not found” errors (this won’t explain why your files got deleted, but it’s something to be wary of in future).

Also another thing to be wary of is that the build system does make copies of some of the files – e.g. castle.BuildSys, mixed.Library, etc. all get copied to the root of the tree (and used from there). So if you’re editing files you need to make sure you edit the right ones, and if you’re diffing against a clean copy of the source tree (e.g. straight from CVS or a source archive) you need to make sure to have your changes in both locations otherwise you’ll miss some of them.

e: I suspect some of the frustrations with processor type not changing and other quirks like complaining about the modules file pointing to the wrong place are all due to the same core issue. It may be time that I set up an ADFS USB stick for building on, because something just isn’t right.

Yeah, I’d say using an ADFS/FileCore formatted drive is the safest course of action. I don’t think anyone really tests the build system on anything else, and for something like DOSFS/Fat32FS you may find that stuff breaks in subtle ways due to e.g. some file attributes not being fully supported. Historically there was also a bug in the build system (probably in amu?) where it failed to work correctly if files built too quickly (timestamp granularity issue); the fix that was applied works for filesystems with centisecond accuracy like FileCore but I’m not sure if it’s guaranteed to work with FAT which only stores modification times with 2s granularity.

I didn’t rename the tree , just added a new target.

After renaming the tree from OMAP3 to AWH3 I re-ran the prepare script. I also remove the old entry from the builder and add a new one. I also do the cleaning steps every time with the build because of the potential for things to get broken.

I should add that I’ve had more iterations of doing various combinations of trees than I can remember. Definitely not saying that I have some kind of knowledge. Just empirically trying things to see what does and doesn’t work. The files disappearing was a new one for me… perhaps.

.bq Historically there was also a bug in the build system (probably in amu?) where it failed to work correctly if files built too quickly (timestamp granularity issue)

What an odd bug.

It just occurred to me that I’ve been building ROMs fine for the Pi for months on fat32fs.

Also another thing to be wary of is that the build system does make copies of some of the files – e.g. castle.BuildSys, mixed.Library, etc.

That one’s already caught me out. Took me long enough to work out that the lesson stuck.

I didn’t rename the tree , just added a new target.

It doesn’t seem to matter much. Thankfully the build tree is set up with either relative structure or absolute with the root as an environment variable, or whatever it’s called in this context.

I am thinking I’ll just go through with my earlier plan of just butchering OMAP3 for now, and trying to rename later.

e: I’d forgotten how horribly slow it is to copy USB to USB on an RPi. Got a USB stick formatted, OMAP3 tree uncompressed, populated with tools and scripts run. Glad I didn’t need to use it for anything. Just checked in on it every 1/2 hour or so.

e2: DDE’s !Find helped me find some stragglers. Finally got a new build target! One of the issues was I’d missed a line in castle.blahblahblah.ModuleDB

Now there’s a working target I can start cobbling together something using the preexisting framework.
I maybe wrong but after spending so long digging through the trees of a few different targets,just about everything in the HAL has already been implemented elsewhere on various platforms except the display.

I’m going to aim to see something spat out from the UART by the end of the week. Simple goal.

Is FileCore slower than FAT? It feels it. Doesn’t matter. It doesn’t mess with the cases of the characters in the filenames unlike FAT.

I’ll be honest. Step 4 in the not quite 10 step plan is my downfall. Or rather this part:

Add the test program into the RISC OS build system by creating a new product file, components file, and env file.

I know how to edit these files. It’s this bit “Add the test program into the RISC OS build system” that I don’t get. It’s a little nebulous. Is there an example anywhere of this part of the step and the building / execution?

Tickling he hardware the right way to do this is dead easy. I’ve done it a few different ways in fact. But how do I do it in relation to the framework? I’m not sure what’s required of me here.

As I’ve stated I’m also writing a standalone “fantasy” HAL using GCC with header files containing what I believe to be correctly laid out functions for the HAL (Thanks OMAP4 devs). Unless there’s a way of using it directly, it’s more of a template of how to get bits of hardware up and running.

Last night I had a quick hunt around and found the H5 user manual. After having a look through it, it would seem that if an H3 (A7) version could be made to work, an H5 (A8/A53) would probably work by mostly just having it’s memory map tweaked, which would be nice.

Why don’t you copy a target and change it to your need. You can definitely create something in s.Top that you can test with. Just leave everything else as is.

Oh you are using GCC. Well I still don’t understand why you reinvent the wheel, you can do that later.
It is much more fun to start coding , than writing Makefiles. :)

I know how to edit these files. It’s this bit “Add the test program into the RISC OS build system” that I don’t get. It’s a little nebulous. Is there an example anywhere of this part of the step and the building / execution?

If I was to start a port to a new platform called Wibble, I’d add support for Wibble by performing the following steps:

1. Copy a suitable-looking product folder (e.g. the OMAP3Dev one) and give it the name WibbleDev
2. Likewise, make a copy of a suitable-looking components file and call it Wibble
3. Likewise, make a copy of a suitable-looking env file and call it Wibble.
4. Update the Build setting in the env file to be ROOL.Wibble (IIRC this tells Builder what components file to use)
5. Using the test program as the base for the Wibble HAL, I’ll want to decide which of the different license folders it fits in (e.g. castle, bsd, mixed) and plonk it in xxx.RiscOS.Sources.HAL.Wibble
6. Edit Products.WibbleDev.modules so that instead of referencing castle/RiscOS/Sources/HAL/OMAP3 it references xxx/RiscOS/Sources/HAL/Wibble
7. Edit ModuleDB to add a new entry for the Wibble HAL (i.e. base it around one of the existing HAL entries, and give it a name like HAL_Wibble)
8. Edit BuildSys.Components.ROOL.Wibble so that instead of using HAL_OMAP3, it uses HAL_Wibble
9. For the components file, env file, and moduledb, make sure to update both locations if you’re working on “prepared” sources. Otherwise it either won’t work or you’ll have a hard time finding your changes when diffing against a clean source tree.
10. If Builder is already loaded, opening up its window and re-selecting the build folder should cause it to re-scan for the available environments, causing your Wibble environment to appear in the environments menu.

The stuff relating the the Products file/folder is only relevant for CVS, so there’s no need to do that if all you’re interested in is getting code building.

There’s a pre-requisite for this that isn’t explicitly mentioned: your test program must have a makefile that provides the different rules/targets that the build system uses. For a HAL the two most important ones would be ‘rom’ (build the HAL) and ‘install_rom’ (copy the binary as ‘${INSTDIR}.${TARGET}’). Since nobody’s written a HAL using GCC before, you may have to work out the makefile rules yourself (or you can try building the code using the DDE + CModule makefile, like the other HALs do)

But because the Linux code is typically GPL we can’t use it to develop drivers for RISC OS.

GPL is a terrible license and I wont go near GPL code. (Not that Castle’s license is much better)

I love open source, but always favoured a BSD-style license and all code I’ve published is BSD.

Why would it not be possible to look at the GPL to work out what the hardware is doing, and then use the spec to write something else?

Absolutely. Reverse engineering is fair use under copyright law. Problem is manufacturers make it difficult with binary blobs and other just ignore GPL license and fail to comply in providing the full source.

I hesitate to suggest it, in case I’m misunderstanding things or it is obvious but I’d add step 11
11. Build Wibble and check that the binary is the same as what the platform wibble is based on produces.

I expect builds have the build name imbedded in the binary so initially make the name wibble the same length!

That’s step 5 of the not-quite-ten-step plan :-)

Jeffrey. The implicit step which you wrote at the end is what I was overlooking. As we say “If it were a dog it would’ve bit me!”.
I was so concerned about breaking the big, mysterious framework that I didn’t think to mess with the Makefile.

That reminds me… which Hdr files etc in the HAL part of the tree that HAL uses are used by other parts of the build system?

If, and that’s a big IF I were to try to get a gcc HAL working with the actual ROM instead of a testbed, I’d probably need to port the format of some of those files and keep them synchronised

The other major issue I could foresee is getting the HAL to talk to the rest of the ROM. I wouldn’t have a clue how to do that beyond hand stitching jump tables after build.
Not sure if I mentioned, but my relative strong areas are bare metal / embedded programming, designing embedded systems, interfaces and smart adapters, and reverse engineering hardware and software. Not professionally. My skills have atrophied somewhat too but working with RISC OS is helping me to remember and learn new skills.
I’m saying these things about me because it may be hard to get why I keep falling over.
The summarized version could be I can get the hardware to do what I want, however I don’t know how to get the software to do what the build tree wants.

Besides holes in my knowledge and that irritating issue on needing to assign a big wimpslot to gcc somehow, it’s very tempting to chain in the gcc makefile to the DDE makefile to generate the ROM

Thankfully I still have the most recent revision of all the files you mentioned, including their duplicates in other parts of the tree (which was one of my stumbling points). Looks like I just need to gut the makefile and make a top.s or whatever to feed it.

Chris re your step 11. If I were to try going the gcc route I’m not too worried. The binary is just a stripped arm-unknown-riscos file built without any supporting libraries, and it runs without issue on the H3 which is a very plain Cortex A7. It’s also why I recoiled in horror when I dug deeper into the OMAP code.

By the way I also have no idea how to build things in Norcroft C without the runtime, hardwire the start address, program and data space, specify the stack, add stack overflow and interrupt handler functions, or use the _start entry point instead of main().
That’s pretty much why I’m messing wigh gcc instead of Norcroft C.

James, I have to agree. However unless someone feels like getting the Mali up and running there’s really no need to look at blobs. The LIMA project already has an at least partially working open Mali core anyway if anyone wanted that.

I think it’s CedarX or something which is where access can be gained for all the H3’s nice video handling ability, so again not the Mali. I wouldn’t be messing with that either tbh.

e: Forgot to add that last night I realised how to cut out most of the messing around for testing the binary.
The GPIO module and serial block driver is a part of the RO ROM now, right?
I just need to go back to where I was a year ago, messing around with the UART GPIO pins.
Ie unplug the USB UART adapter from the OPi PC’s debug UART, and plug RX, TX and Gnd into the RPi with a few DuPont wires. Put an elastic band around the lot, Pop open !Connector (or was it !HearSay?) and copy the binary straight over. Easy!

That reminds me… which Hdr files etc in the HAL part of the tree that HAL uses are used by other parts of the build system?

Headers that are shared with other components will get copied to the Export folder in the root of the build tree, by the “export headers” build phase (which is the export_hdrs makefile target). I don’t think there are any HALs that export their headers, but they do certainly rely on headers exported by the rest of the system (HALEntries, OSEntries, etc.)

If, and that’s a big IF I were to try to get a gcc HAL working with the actual ROM instead of a testbed, I’d probably need to port the format of some of those files and keep them synchronised

Remember that the RISC OS version of GCC has ‘asasm’, which is compatible with objasm sources. So if you’re happy with ARM’s assembler syntax then there’s no particular need to use the GNU syntax.

The only gotcha is that recent versions of asasm don’t allow you to specify a hard-float ABI in the ELF header, which causes problems linking with hard-float FPA code produced by GCC’s C compiler. But since the HAL is a standalone binary that gets concatenated to the rest of the ROM image, you should be able to sidestep this issue by using C compiler flags that are compatible with whatever objasm is hard-wired to.

The other major issue I could foresee is getting the HAL to talk to the rest of the ROM. I wouldn’t have a clue how to do that beyond hand stitching jump tables after build.

The key to this is that:

1. The build system pads the HAL out to a fixed size
2. The HAL will know this size at build time (the header Hdr:HALSize.<HALSize> defines the OSROM_HALSize symbol)
3. Immediately after the HAL comes the kernel
4. The kernel starts with an OS image header which contains the offsets of all of the OS routines that the HAL is allowed to call

reverse engineering hardware and software

Well consider this a reverse engineering task, but with a really good disassembler that magically provides you with the original source code of the binary you’re trying to disassemble ;-)

The poor level of documentation for the OS and build system means that the people who are able to work things out for themselves are usually the ones who succeed.

Besides holes in my knowledge and that irritating issue on needing to assign a big wimpslot to gcc somehow, it’s very tempting to chain in the gcc makefile to the DDE makefile to generate the ROM

It might be worth looking at the makefiles for some of the apps that are built using GCC. Using GCC with AMU is possible, the only tricky part is that the dependency files generated by GCC aren’t in a format which AMU is happy with, and so a perl script is used to re-format them (part of the GCCRules makefile fragment)

By the way I also have no idea how to build things in Norcroft C without the runtime

Examine the makefile for one of the HALs. Or just look at the commands that are executed in the build output.

hardwire the start address, program and data space, specify the stack, add stack overflow and interrupt handler functions, or use the _start entry point instead of main().

There’s a reason why the HALs are mostly written in assembler – getting C to work within the constraints of the HAL can be pretty tricky. See Using C in assembler components for some more detail on this. Possibly GCC will handle this better than Norcroft, but it’s not something I’ve looked at in any detail.

If you can’t get GCC to produce read-only, position-independent code, I’d be tempted to split the HAL into two distinct parts: the pre-MMU part (everything up to the OS_Start call) and the post-MMU part (which will contain HAL_Init and all the other HAL entry points called by the kernel). That way you should be able to give both parts different static base addresses (physical address for the pre-MMU part, logical address for the post-MMU part) and things should “just work”.

So the size of this doesn’t blow out I’ll just respond to necessary bits. Believe me, I have taken it all in. You have answered pretty much everything I’ve needed to know.

I’ve had literally no time this week to have a proper look at anything. I stole a few minutes just now to go through a disassembly of the GCC code that I’ve generated for my test.
While I can’t say with 100% certaintly it looks to be position independent code. When I get a chance I’ll upload it to the OPi PC and try running it from a couple of different (sane) addresses.
I’ve stated before that it’s more of a test program to make sure hardware does what I think it should, but decided to make it “HAL Compatible” to make writing the HAL easier. However It is really starting to look like GCC could be used to generate at least most of the HAL part of the ROM.

The poor level of documentation for the OS and build system means that the people who are able to work things out for themselves are usually the ones who succeed.

That goes without saying. I think you have written most of the useful documentation on the build system. May I suggest that you take what you have written for me and add a version of it to the Wiki? It goes a very long way towards making it all a lot less mysterious.

…the only tricky part is that the dependency files generated by GCC aren’t in a format which AMU is happy with…

This part was where things seemed a bit hopeless. Thanks for that useful information.

e: The code generated looks position independent but something isn’t. Interestingly when executed from another address instead of spitting out “Hello World” it prints out a chunk of 0xff to the UART. So perhaps the print code is working but looking in the wrong place. I don’t know. Doesn’t really matter anyway. It’s just a test program.

That goes without saying. I think you have written most of the useful documentation on the build system. May I suggest that you take what you have written for me and add a version of it to the Wiki? It goes a very long way towards making it all a lot less mysterious.

Done, I think.

https://www.riscosopen.org/wiki/documentation/show/How%20to%20port%20RISC%20OS%20to%20new%20hardware
https://www.riscosopen.org/wiki/documentation/show/HAL%20OS%20layout%20and%20headers

That’s excellent! It makes working on porting a lot more accessible.

No idea if I said but I did sort out a build target and got some code up and running the normal way. Using gcc is far more interesting though so that’s what I’ve been playing with.
Last night I got the DDE make to cooperate enough to build my gcc code. Make collapses in a heap at the end but it’s no surprise. The file took many attempts to work out some of the quirks of both DDE make and gcc to the point where it would do something.

I think if I give ld a better command file the GCC code should be ROMmable. After quite a bit of playing around the disassembly looks relocatable.

When the makefile is at least fixed enough not to die at the end I’ll try importing headers via that script.

e: worked out some of the quirks involving the DDE make utility and the makefile functions properly now. Only real irritation is the gcc toolchain has inconsistent file name handling so I have to sort that again.

I think the Linux port has some things to teach me so I’m going to dig around in that for a bit.

Makefile is going well, but I’ve hit a small wrinkle with asasm. I redid a copy of the only assembly file from gas to asasm, however for reasons I can’t quite pin down the code from it goes into the linker (ld) and doesn’t come out the other end. It just silently disappears, and a binary is generated missing a means to start.

The reason I decided to try was simply so I could use IMPORTed .hdr files in an easy way that has them near the beginning of the linked binary.

Please correct me if I’m wrong. It’ll sure save me a lot of time. I think it should work fine, however I am less than totally confident.

If it works, it should be relatively easy to construct a proper HAL in C, which could potentially make porting a little more accessible.

I just want to clarify to people readin this in passing that the DDE suite is still needed. It’s just I can’t find a way to make Norcroft C work the way I want.

Makefile is going well, but I’ve hit a small wrinkle with asasm. I redid a copy of the only assembly file from gas to asasm, however for reasons I can’t quite pin down the code from it goes into the linker (ld) and doesn’t come out the other end. It just silently disappears, and a binary is generated missing a means to start.

Thoughts that come to mind:

• Something wrong with the ldscript which is causing it to not include the code
• Dead code stripping has decided that none of your code is actually being used

I’ve never used ld directly before (only via the gcc frontend), and have never really messed with ldscripts, and don’t really know how asasm would fit into things (I guess the AREA would determine which ELF section the code gets placed into, but maybe it’s not a direct mapping?)

Got it to build again!
I forced the toolchain to fall over by adding an asasm asm file with a dummy routine that an extern function referenced in gcc.
It was that asasm was using hard float, and gas / gcc was using soft float.
Fun thing is there is a known bug where -soft-float is missing from asasm and there are no alternate soft float options so I had to downgrade asasm to the one from gcc 4.1
After sorting all that it all links fine.

So I have:
*a small gas assembly stub with the entry point, and a couple of other things gcc needs. Some C source in some files, a linker command file, an asasm assembly file with the GETs and IMPORTs in it, and the makefile.

The only reason I have “Top.s” in gas format still is I’m terrible with asasm and I also haven’t tested the asasm version of “Top.s” post downgrade yet.

In terms of progress towards booting RO I haven’t made it far. Seeing if I can get the asm / C framework running is the time consuming thing.

s.deps: Warning: Symbol rom_checkedout_ok is imported but not used, or exported but not defined
ld -fpic o.top o.deps o.begin o.roio o.uart -T memmap -o elf/hal
o.begin: In function `notmain':
begin.c:(.text+0x214): undefined reference to `rom_checkedout_ok'
AMU: *** exit (1) ***

Does anyone ever wish it was possible to grab a program by the proverbial scruff of the neck and show it something? I know I do.

I pretty much picked

rom_checkedout_ok

With objasm/asasm all symbols are private by default.

• If you want a symbol to be visible to other object files, you need to EXPORT it
• If you want to reference a symbol exported by another object file, you need to IMPORT it

Jeffrey, Unfortunately I already had it set up like that.

This is probably the first real chance I’ve had to look at the code since then. I’ve got a multi pronged approach going with multiple makefiles and a couple of targets. I’m trying to find approaches which work, and which method to use.
With the version I’m stuck on, which we are talking about I spotted a glaring issue after fiddling with the asm target for the AWH3 when the HAL wouldn’t build. For some reason the “Export Headers” phase wasn’t done in !Builder. I don’t remember cleaning the build recently. It was a problem with both the asm and C based targets. Live and learn I guess.

I removed -soft-float from the asasm flags to trip up ld, because it’s not providing me with any useful hints on what’s happening to the asasm objects.
The result was telling. ld complained about every object after Top.s’ object being soft float. So it is acknowledging Top.o exists. The only thing I can think of is that ld is overwriting the code from Top.o with the contents of the first C object.
So what I need to do is work out what’s functionally different between the original gas version of Top.s and the asasm version, as the gas version works as intended. Unfortunately as far as I know gas can’t support hdr files. Can it?

I know ld works with asasm. One of the GCC examples does exactly that. Not sure what’s going on yet.
Sorry about all the posting about these issues. It’s just a really curly problem, which I feel may be well worth solving. Being able to use C in the HAL, even pre-MMU may help make RO a lot more portable.

Unfortunately as far as I know gas can’t support hdr files. Can it?

No.

You might be able to use some of the C header files (a lot of the ones that are generated from assembler hdr files are just a long list of numeric #defines), but that’s about it.

It would probably be worth using something like objdump to check what the differences are between the gas and asasm object files.

Whatever the difference is, it has some powerful magic. Nothing seemed particularly enlightening.

A friend had a look at the HAL code and dumps and had some useful suggestions. In the end not much progress was made at the time but I had some things to think about.

This morning I roughly tacked ld into the OMAP3 based Makefile so it’d spit out a link and ld version of the final linked file.

Armed with a new linker script for ld and a new as yet unknown to me gcc flag I finally got ld to put Top.o at the beginning of the file!

The Norcroft linked file still has the weird problem with failing to resolve two symbols properly in on epart of the file yet does it fine in another.
I know it’s slapping me in the face with the answer but I don’t know how to fix it.

0×41000000 Is pretty arbitrary to be honest. I think it’s the default base address for loading the linux image with U-Boot.

The ‘/’ used for objdump and objcopy is because they don’t seem to support the RO ‘.’. I just hardcoded them in for now. No need to be fancy.

I have no idea how to deal with the area alignment complaint of the ARM linker.

gcc -Wall -Werror -O2 -fPIC -nostdlib -nostartfiles -ffreestanding -fstack-check=no -c -static -ffunction-sections -o o.Begin c.Begin
gcc -Wall -Werror -O2 -fPIC -nostdlib -nostartfiles -ffreestanding -fstack-check=no -c -static -ffunction-sections -o o.uart c.uart
gcc -Wall -Werror -O2 -fPIC -nostdlib -nostartfiles -ffreestanding -fstack-check=no -c -static -ffunction-sections -o o.roio c.roio
asasm -depend !Depend -throwback   -ihdr -i<Hdr$Dir>.Global -i<Hdr$Dir>.Interface -i<Hdr$Dir>.Interface2 -pd "APCS SETS \"APCS-32\"" -pd "Machine SETS \"CortexA7\"" -pd "UserIF SETS \"Sovereign\"" -soft-float -o o.Top s.Top
link -o aof.AWH3C -aof o.Top o.roio o.uart o.Begin  RISCOSLIB:o.romcstubs
ARM Linker: (Warning) Attribute conflict within AREA .bss
    (conflict first found with Begin(.bss)).
ARM Linker: (attribute difference = {area-alignment}).
ARM Linker: finished,  1 informational, 1 warning and 0 error messages.
ld -fpic -static o.Top o.roio o.uart o.Begin -T memmap -o elf/hal
objdump -D elf/hal > list/hal
AWH3C HAL: rom module built
do mkdir -p aif
link -aif -bin -d -o aif._AWH3C o.Top o.roio o.uart o.Begin  RISCOSLIB:o.romcstubs
ARM Linker: (Warning) Attribute conflict within AREA .bss
    (conflict first found with Begin(.bss)).
ARM Linker: (attribute difference = {area-alignment}).
ARM Linker: finished,  1 informational, 1 warning and 0 error messages.
ToGPA -s aif._AWH3C gpa.GPA
AWH3C HAL: debug image built
link   -o linked.AWH3C -bin -base 0x41000000 aof.AWH3C
objcopy elf/hal -O binary linked/AWH3Cld