HALDevice header location

I wonder what people think?. There has been a recent drive to move HALDevice headers out of Kernel.Hdr to various disparate places in the source tree . e.g. the GPIODevice header has now vanished back into the GPIO module from TankStage. This meant HAVING to include that module to build a rom as the iMx6 HAL legitimately exports a GPIOHALDevice. There are many others it exports.

That suggests these headers NEED to be in a common location, not spirited away in one or many client modules that use them.

There is only 1 Kernel. There are many HALs. A Kernel always links to a HAL (now), hence somewhere like Kernel.Hdr is a very good container for these common interface specifications.

If Kernel.Hdr is deemed unacceptable as a HALDevice header store, perhaps we should use some other location, such as Programmer.HdrSrc, or Kernel.HALHdr, to contain these. Either way it needs to be somewhere that those with normal cvs code write access can update, and all HALDevice headers really need to be in the same place.

Any thoughts? Thanks

Having the headers somewhere central sounds good to me – although I’d prefer them to be in Programmer.HdrSrc rather than Kernel.Hdr. Most of the devices are of no interest to the kernel, so it doesn’t really make much sense to store their definitions there (much like we don’t have the big list of SWI or error chunk allocations stored in the kernel headers).

One problem with having the headers stored in the modules that use the devices is that if you have two different modules that want to use the same device then the two definitions could go out of sync and cause issues (I’m currently running into something like that with SoundDMA – if a new feature is added to one of them I need to update the headers for all three just to make sure everything builds OK! In this case it’s just a regular header export causing the problems, but there are two different AudioDevice headers too)

Having a common place for HAL device headers might also help avoid some of the headaches related to code licensing – imagine if there was a driver whose license prevents it from being included in ROM, like the first version of the VCHIQ driver for the Pi, which had to be released under the GPL. Being able to have the HAL device header in a common location under a ROM-friendly license would allow you to build the ROM without having any of that GPL code in the build tree, simplifying the process of proving that you aren’t infringing the license.

On a related note to the device headers, it’s always struck me as a bit odd the way that the HAL device IDs are held in the kernel headers rather than in HdrSrc – so maybe it’s worth considering moving those as well.

So – if we were to place all HALDevice headers in a new folder in HdrSrc, e.g. HAL-Hdr, would that be wise?

If no one has objections to this, perhaps we should just do it?

(consequences .. mainly to ensure that this new folder is correctly exported within the build system .. a small makefile mod for HdrSrc I suspect)

If no one has objections to this, perhaps we should just do it?

Yes. ROOL has a large number of objections. Please see private mail – hopefully we can post the conclusions of our offline discussion once we’ve reached an agreement. I didn’t want to post everything here because the discussion via email pre-dates the start of this forum thread.

Steve. I have replied to you separately, but I believe this is an issue of some relevance, and would appreciate other views on this.

ROOL view, essentially it seems, appears to be that HALDevice export belongs with the module requiring it.

Surely the purpose of a HALDevice being exposed by the HAL is to let one or many potential client modules discover the device. Otherwise if you have need of a device for 2 potentially different purposes you will oblige HALs to expose essentially the same thing twice.

Granted, where a HALDevice is supplied by and registered through a module then it is/may be appropriate for the module to be the owner.

My view then is that HALDevice headers for anything exposed by a HAL should be in a common place for all users, clients and HALs, which rather points the the HdrSrc component for this purpose. Additionally these need to be in an appropriate common licence format. (As Castle I would be rather keen that this be the Castle one).

OK. If you want to discuss this on the forum, here’s my view.

It sounds like you are trying to establish precedent for all HAL device headers having Castle licence headers, which ROOL would strongly oppose – it goes against the ethos of the community development we’ve supported for the past decade. This is a community repo with a choice of licences that contributors might elect to use. All contributions benefit the whole, including Castle, so it’s unacceptable to try to force people to hand over ownership of their work to Castle on the basis of FUD about where these header files should live.

Here are all the technical arguments against your proposal:

It makes no sense to have multiple driver modules talking to the same HAL device. You always need some arbitration between multiple clients, and putting that arbitration inside the HAL device means code duplication between HALs and restricts you to synchronous arbitration (polling) since service calls, wimp messages and other forms of “inter-process” communication are off limits in HAL code. Also remember the HAL is supposed to be a thin layer on top of the hardware. You are still thinking in terms of traditional module APIs – this is not valid.

Components shouldn’t be referring directly to files in other components – this is why we have the Exports directory to isolate components from one another. As far as each HAL is concerned, they shouldn’t have to care which component actually exported any given header.

Also recall that despite the name, a “HAL device” is not necessarily implemented with a HAL. Typical reasons why this might be the case include when it is a driver for an optional expansion card, or if its implementation requires the use of SWIs (which are forbidden from the HAL proper) or if there is a licensing conflict between the implementation and the rest of the HAL. You can quite happily build modules that provide these types of HAL devices without also having to build a kernel.

If a HAL provides a particular type of HAL device, you almost certainly want to include the module that interfaces to that HAL device in the ROM build as well. Even in rare cases where this might not be required (e.g. RISC OS Pico) it’s not a great burden to include the module as a “type EXP” component.

It doesn’t really make much more sense to put these header files in HdrSrc than anywhere else. The aim should be to minimise the number of components that need updating when an API is extended, so the headers belong either with the caller of the API (the relevant driver module) or the callee (the implementation of the HAL device) and since there is one of the former and many of the latter which would otherwise need to be kept in sync, the former is the logical choice.

The extent to which the use of a header file by other source files can be said to form a derivative work is something of a legal grey area (though obviously it would apply to both the caller and callee of the API that it describes). Our usual advice to people creating new header files is to use a permissive licence (such as BSD) to avoid ambiguity.

OK. If you want to discuss this on the forum, here’s my view.

It sounds like you are trying to establish precedent for all HAL device headers having Castle licence headers, which ROOL would strongly oppose – it goes against the ethos of the community development we’ve supported for the past decade. This is a community repo with a choice of licences that contributors might elect to use. All contributions benefit the whole, including Castle, so it’s unacceptable to try to force people to hand over ownership of their work to Castle on the basis of FUD about where these header files should live.

Thanks Steve for misunderstanding. Common header location is NOT the same issue as licence header. Headers that others may uses should simply be licensed in a manner that does not make code reuse awkward. Thou may remember that ROOL took a considerable time to do ‘due diligence’ on the licensing/copyright of what has been released. I (as Castle in this specific case) do not wish to have to redo this for ‘common header files’ in the future. Hence the request to minimise the licence headers in use. the BSD style licensing also holds good I guess.

Here are all the technical arguments against your proposal:

It makes no sense to have multiple driver modules talking to the same HAL device. You always need some arbitration between multiple clients, and putting that arbitration inside the HAL device means code duplication between HALs and restricts you to synchronous arbitration (polling) since service calls, wimp messages and other forms of “inter-process” communication are off limits in HAL code. Also remember the HAL is supposed to be a thin layer on top of the hardware. You are still thinking in terms of traditional module APIs – this is not valid.

As a HAL writer (initial Pi work and iMx6 so far) I tend to make any hardware I can visible via the commonly available interfaces so that at a later date I or others can make use of the info. Your argument above pretty much states that one should not expose hardware until support for it is a fait accompli. Could that be considered shortsighted?

Components shouldn’t be referring directly to files in other components – this is why we have the Exports directory to isolate components from one another. As far as each HAL is concerned, they shouldn’t have to care which component actually exported any given header.

Good as far as it goes, but requires builds to be aware of far more stuff, even if just as an export, that would be required with the common storage location I would want to see. HdeSrc contains ‘things’ that are published to the exports directory. What is wrong with putting the published HALDevice components there?

Also recall that despite the name, a “HAL device” is not necessarily implemented with a HAL. Typical reasons why this might be the case include when it is a driver for an optional expansion card, or if its implementation requires the use of SWIs (which are forbidden from the HAL proper) or if there is a licensing conflict between the implementation and the rest of the HAL. You can quite happily build modules that provide these types of HAL devices without also having to build a kernel.

I have no issues with modules ‘pretending’ to be the HAL, by dint of exporting relevant HALDevices.

If a HAL provides a particular type of HAL device, you almost certainly want to include the module that interfaces to that HAL device in the ROM build as well. Even in rare cases where this might not be required (e.g. RISC OS Pico) it’s not a great burden to include the module as a “type EXP” component.

See above.

It doesn’t really make much more sense to put these header files in HdrSrc than anywhere else. The aim should be to minimise the number of components that need updating when an API is extended, so the headers belong either with the caller of the API (the relevant driver module) or the callee (the implementation of the HAL device) and since there is one of the former and many of the latter which would otherwise need to be kept in sync, the former is the logical choice.

HdrSrc will need to be present is almost any build, to create the exports folder. I’d rather develop without having to load 1/2 dozen or mor pther comonents just to get at 1 exported header file. See above

The extent to which the use of a header file by other source files can be said to form a derivative work is something of a legal grey area (though obviously it would apply to both the caller and callee of the API that it describes). Our usual advice to people creating new header files is to use a permissive licence (such as BSD) to avoid ambiguity.

we are broadly in agreement here. Though perhaps it could be wise to provide stronger guidance in relation to any headerfiles that are of ‘generic’ use.

I thing the best analogy I could give is that a third party wishes to make a produce using a subset of the tree. Due diligence on what is built would need certainty that anything incorporated into the build tree was free from encumberance.

Is it then wise that a prerequesite of building a HAL was inclusion of lots of stuff .. just to export some headers, or do extra work / fork to cut stuff out of the HAL for this?

(I have acknowledged the move out of Kernel.Hdr .. this is NOT the issue at stake)

Thanks Steve for misunderstanding. Common header location is NOT the same issue as licence header. Headers that others may uses should simply be licensed in a manner that does not make code reuse awkward.

To be fair, you were the one who conflated the two issues into the one thread: “Additionally these need to be in an appropriate common licence format. (As Castle I would be rather keen that this be the Castle one).”

Thou may remember that ROOL took a considerable time to do ‘due diligence’ on the licensing/copyright of what has been released. I (as Castle in this specific case) do not wish to have to redo this for ‘common header files’ in the future. Hence the request to minimise the licence headers in use. the BSD style licensing also holds good I guess.

This is one of the reasons why CVS write access is only open to a limited number of people whom we trust to have sufficient understanding of software licences to know what is permitted (or at least know when to refer issues to us if they are uncertain).

As a HAL writer (initial Pi work and iMx6 so far) I tend to make any hardware I can visible via the commonly available interfaces so that at a later date I or others can make use of the info. Your argument above pretty much states that one should not expose hardware until support for it is a fait accompli. Could that be considered shortsighted?

Most silicon blocks don’t follow a standardised layout and function. If no API has yet been defined for access to that type of device and you haven’t surveyed alternative implementations to see what sort of quirks tend to exist, then you won’t be able to do much more than announce the base address of the device. That is not abstracting it, it’s just placing the responsibility on the module to know everything about how to drive all possible platforms – witness the ever-increasing complexity of the GPIO module to see why this is a bad idea.

Good as far as it goes, but requires builds to be aware of far more stuff, even if just as an export, that would be required with the common storage location I would want to see.

I don’t see it as being likely that any given ROM build would require significantly more components. HALs don’t generally expose lots of HAL devices where the ROM doesn’t also contain modules that interface to those HAL devices. HALs don’t normally get built separately from the corresponding ROM.

HdrSrc will need to be present is almost any build, to create the exports folder. I’d rather develop without having to load 1/2 dozen or more other components just to get at 1 exported header file.

You seem to be concerned about being able to build HALs having only checked out a very minimal number of components from CVS. I’m afraid the number of components needed to reasonably populate the Exports directory is considerable and is growing all the time, and this trend is unlikely to reverse. The logical extension of what you’re saying would be for all components to move their public headers into HdrSrc just to make it quicker for a developer to get up and running. I’d be against this on the grounds of maintainability – for example, it would make it impossible to prepare stable releases where individual components (and their associated header files) may come from different times in history or different branches (perhaps containing back-ported bugfixes).

If you want a quick-and-easy populated Export directory to work with, I believe one is included with the DDE.

I feel like I should point out that whilst ROOL has made every effort to comply with its responsibilities under relevant licence conditions, it has never offered indemnification to any other party who chooses to redistribute the software on this website. It is up to each redistributor to do their own due diligence.

the BSD style licensing also holds good I guess.

For ease of (re)use, it should be the norm for all header files, IMHO.

Having worked on a variety of other systems (open and closed) I do find it one of the elegancies of the RISC OS build system to be able to run an export phase to collect up all the exposed APIs between components. Contrast that with Linux’s monolithic include directory with all the world’s headers in, which roughly shadows the layout of the sources, but with the association of who implements what being lost. Keeping the header next to the component that implements (or the component that consumes) it seems much safer, then once the exports phase is done the lazy path specifying

GET Hdr:WhatEver

ROM component lists already contain 10-20 things they don’t build, like MakePSFont and Econet, and adding another one or two seems little hardship – the components are small, computers are fast, dialup is history.

If no API has yet been defined for access to that type of device and you haven’t surveyed alternative implementations to see what sort of quirks tend to exist, then you won’t be able to do much more than announce the base address of the device. That is not abstracting it

There’s a great draft design document which is always worth re-reading whenever seeking to find out where the boundary at the edge of a HAL is. It’s a very carefully worded document and covers things like why APIs should be defined as C function prototypes rather than a single entry with wacky variadic sub reasons (maps well to assembler, HLL not so much). The result invariably means the HAL API represents the lowest common denominator or most generic view of that device type; just sawing a module in half and putting half the code in the HAL and half in a module with a very device specific API between the two halves hasn’t achieved much.

I’m not really sure what this thread’s about now as it’s wandered off into topics probably best discussed direct between ROOL and Castle, than wearing out my eyeballs.

I’m not really sure what this thread’s about now

I think it’s about who has the pink dinosaur

(see penultimate paragraph) :-)

I think the clue is in the name, Hardware Abstraction Layer.

Anything that interfaces directly with the hardware and then presents a common interface to the user is part of the HAL.

If the coder cannot live with whatever the licence, that the HAL for that hardware has, then it should be a stand alone module that they can determine their own licence.

Maybe the main HAL headers in the HAL.xxx.hdr directory, and a sub-directory for the individual parts.