Bug or feature? Minimum git version

My usual way of updating my local clones of ROOL “Product” git repos is as follows:

git pull
git submodule sync --recursive
git submodule update --init --remote --recursive

However the third command has started failing on git versions prior to 2.28, because SDIOLib is using “main” instead of “master” as its default branch name, and the .gitmodules in the superproject doesn’t specify the branch name. Pre-2.28 git will default to using “master” as the remote branch to use when updating the submodule, causing it to fail. 2.28 and newer default to the default branch of the submodule’s repository (i.e. “main”), so they’ll work fine.

The change to git was back in June 2020, but not all Linux distros have caught up yet; in October I retired the x86 Ubuntu Linux box that I was using for NAS/Linux duties and replaced it with a Raspberry Pi OS install on a spare Pi 4. At the time the latest Pi OS was based on Debian Buster, and the default Buster package lists only come with git 2.20. (The solution for this, if you don’t want to update to the newer Bullseye-based release of the OS, is to add the buster-backports package repo (1) (2) and install “git/buster-backports” which will bring things up to git 2.30 )

Although ensuring all developers are using up-to-date git versions is one solution to the problem, another solution would be to just be explicit in .gitmodules about which branch should be used. Hence the question: is requiring people to use a minimum git version of 2.28 a bug or a feature?

In my experience having submodules which track branches (rather than SHAs) is unwise because it breaks the idempotence rule of source control. That is, when you check out the git repository at any given time in the future, its contents should not change. Thus the use of `—remote` to follow a tracking branch is a bad choice because it breaks that – what you check out can be different to that which was checked in. And if it’s different to what was checked in, that means you cannot reliably build a given version of the source.

Whilst it tries to ensure that you’re pulling something that is following a branch and thus pulls what you might think was intended, it guarantees that what you’ve pulled it NOT what it in source control as the definition of the submodule. So in my opinion, you should not be using remote tracking braches in this way if you want to retain the reproducibility of builds.

If, however, you don’t care about your source being reproducible, and what you checked in being what was checked out, the `—remote` option might be useful. But I cannot see a circumstance that you would actually want this in a large product.

Sorry, I’ve not looked at how the RISC OS Open sources are managed, but thinking about it, I guess it’s possible that the RISC OS open source management relies on the CVS-like mechanism of checking out named tags to associate the components that go into a given build, but doing that would be essentially duplicating the work of submodules, and if that were the case you wouldn’t want to use the `git submodule update —init —remote —recursive` option, but instead use whatever tool was actually updating the source tree to use the necessary tags that were configured for a given source. Then you would actually get the source tree you were expecting. That said, I’m making an assumption about how you might manage things when not using submodules which retained their idempotence.

Personally, on my trees with submodules I use `git pull —recurse-submodules=on-demand` to pull down just the submodules that I’ve got present in the tree, at the version they were checked in. With a large git repostory with many submodules (my RISC OS source repository has 736 submodules present – the RISC OS ‘All’ product has 511, which is still substantial), having only the submodules present that you’re working on makes `git status` (and many others) significantly faster, and means that you’re more careful about not pulling in things that you do not need.

The way ROOL manage things is:

• Each component (module, library, etc.) is a git repository
• Each product (ROM image, disc image, etc.) is a git repository which imports all of the relevant components as submodules
• Where reproducible builds are wanted (e.g. stable releases of ROMs), a tagged commit will be created in that product’s repo which has the submodules set to the correct commit hashes for that build. So as long as you check out that tag and don’t do a —remote update on the submodules, you should have the same source as everyone else.
• For normal development builds, most of the time you’d just want to use the latest versions of the relevant components, so when getting latest we just use “git submodule update —remote” to update the submodules locally. We/ROOL don’t create/push new commits within the product repo in order to track those changes (even though that would be useful in some situations, e.g. to find what component versions were in use for a given nightly build)

If ROOL pushed commits to every relevant product every time a component changes then that would solve the problem of —remote not working (since nobody would need to use that command). But since ROOL aren’t doing that (yet), the question about the expected git version is valid.

Yeah, that’s pretty much what I was assuming in my thoughts half way through. It’s using the old CVS tagging mechanism for component tracking, ignoring the sha tracking and only using submodules as a mechanism to associate git repositories with one another.

The simplest solution, if I’ve understood correctly, would be to just change the default branch name on the SDIOLib to `master` then it will be consistent with everything else. That way you can also tell anyone ‘the branch `master` is always the latest version’, rather than relying on specific version of git at the user’s end.

I know that’s not an option you gave, but it has the greatest benefit in providing a consistency and ensuring that no matter what version of git you use you get the same behaviour.