Git

Hi folks. New to the forums, apologies for any mishaps.

There’s a bounty for porting git to RISC OS. I’ve made a reasonable start on it this month in my spare time. It currently does a couple of the basics, notably it’s capable of cloning from an HTTPS repository, and a couple of minor things as I get things fired up. Before I go any further, I just wanted to check if I’m treading on anyone else’s toes at this stage. (I’d hate to dive into this more deeply at this stage only to find out someone else is already tackling this particular task). If no-one else is, I’m actually quite interested in continuing this project.

Any feedback welcomed. Thanks all.

Is it at a stage where it could make a local copy of the RISC OS Pi source from the repository, on my Pi?
If so, I’d be interested in giving it a try. I never did manage to get CVS to work (back when that was the way), and I think making a copy of the repo would be less problematic in the long term than periodically downloading and unpacking tarballs.

Small question – as I’m not familiar with Git – if I made a copy and then wanted to refresh the copy to get it up to date, but I had made changes to a file…is there a way to protect that file so it is left alone?

I just did a clone of one of the source repos on my Pi here (running RISC OS 5.27) successfully using the following command from the * prompt:

git clone https://gitlab.riscosopen.org/RiscOS/Sources/Kernel.git $.Kernel

So yes it’s already achieving that, but there is a lot more work to be done yet before the git tool can become useful just yet.

The answer to your question about protecting the file is simple, you would most likely just use a local branch in your copy of the repo, allowing you to pull changes down from the master branch of the origin whilst protecting the work you’re doing in your branch. That’s beyond the scope of this thread somewhat but if you are interested in learning more about git, there’s an online book that’s free and useful reference material: https://git-scm.com/book/en/v2

Before I go any further, I just wanted to check if I’m treading on anyone else’s toes at this stage.

I haven’t heard of anyone working on it.

OK, there is the SimpleGit port that I worked on, but I haven’t done any work on that in months, and didn’t have any immediate plans to improve it further.

Hopefully you’ve heard of the SimpleGit port (thread), because if you’re following ROOL’s advice in the bounty description and building the client around libgit2, then you can get yourself a headstart by using the libgit2 recipe that’s in the GCCSDK autobuilder. In my experience libgit2 worked great, the main issues that needed resolving were that SimpleGit didn’t expose enough of its features (and had some syntax differences from standard git), and some potentially tricky customisation work was needed (perhaps to libgit2 itself) in order to support per-repository configuration of RISC OS vs. Unix filename/filetype translation.

Good luck!

Thanks Jeffrey! I’ve got it up and running with libgit2 and mbedTLS, with everything built using !GCC. The build process is clean, documented, and simple. The libgit2 API appears to be quite useful.

There’s a lot of work to do yet, obviously, but it’s got to that point where it’s all moving in the right direction, so I’ll happily carry on for us. Hopefully have something a bit more interesting to show in the weeks to come. Don’t touch that dial, etc!

Well done. I did have a little look at git and libgit2 ages ago, but I didn’t get all that far with it.

I’m currently focusing my (somewhat limited) time on Python 3, but I will follow this with interest.

I’m currently evaluating how git might best tackle RISC OS filetypes in repos. There was potentially a way to encode filetypes in git, but after technical tests and discussion with others, it sadly has to be considered a no-go situation (I can bore people with technical details another time).

I currently hold my own views on how git might best work with RISC OS filetypes but – and without wanting to start an overly huge debate! – if anyone would like to throw in their view on how this might work I would be interested to hear them, as I’d rather get a selection of views on this before moving further.

So if you have a view on how RISC OS and git should deal with filetypes, do please pipe up so I can hear.

(If anyone wants an update whilst I’m here, well: there’s a lot still to do at the moment, but progress is satisfactory so far. The following git commands are largely in place: help, version, init, clone, add, rm, status, branch, checkout, commit, fetch, merge, push. I have been able to conduct a workflow that involved cloning a pre-existing repo from GitHub (https, authenticated), creating a branch, working on the branch and pushing the branch up to GitHub, whilst fetching and merging master branch changes that had been committed outside of RISC OS. So there’s lots of promising stuff happening, it’s just a matter of time really).

I currently hold my own views on how git might best work with RISC OS filetypes but – and without wanting to start an overly huge debate! – if anyone would like to throw in their view on how this might work I would be interested to hear them, as I’d rather get a selection of views on this before moving further.

In my opinion the only sensible way of handling RISC OS filetypes in git is to support the de-facto “RISC OS filetypes on foreign filesystems” standard that’s been in place for the last decade or more: A mix of mimemap-based filetype inference based on DOS/Unix filetype extensions, and “,xxx” style extensions for anything else. Perhaps there’ll be a config option or two to control behaviour (e.g. some repos might want to completely ban mimemap based conversion, to avoid different people seeing conflicting filetypes if they have different mimemap files locally). But on the whole we should aim for a system that’s interoperable with the native CVS + SVN ports, and network filesystems. E.g. a person who checks out a repository using native RISC OS git should see exactly the same files as someone who checks it out on Unix and then accesses it via a network share (assuming both users are using sensible settings).

The following git commands are largely in place: help, version, init, clone, add, rm, status, branch, checkout, commit, fetch, merge, push. I have been able to conduct a workflow that involved cloning a pre-existing repo from GitHub (https, authenticated), creating a branch, working on the branch and pushing the branch up to GitHub, whilst fetching and merging master branch changes that had been committed outside of RISC OS. So there’s lots of promising stuff happening, it’s just a matter of time really

Sounds good!

I’ll reinforce what Jeffrey said. While adding an “,xxx” extension to the filename as well as using MimeMap has its own problems, at least it is consistent with emulators and network filing systems, so we know very well what those problems are, and nobody came up with a better generic solution yet. Putting a new solution into gitconfig or wherever will ultimately lead to difficult-to-track-down and need-to-explain behaviour.

Well done.

Ah file types and extensions – I’m currently (trying to) deal with this sort of thing in python. RISC OS does things somewhat differently, both in naming and the with types vs extensions.

Here’s some thoughts. Feel free to ignore the ramblings :)

If it’s just a RISC OS project you can just shove ,xyz on each filename in git to hold the type, but for things that also get worked on on other platforms that’s a bit .. ugly.

Perhaps some sort of extension to type mapping, which can be configured globally or per repo?

There is the issue with C (and asm) files, where “foo.c” becomes “c.foo” (ie. a file called foo in directory c). There probably does need to be some way to handle that – perhaps a list of extensions to do it for.

Another thing to consider – if I checkout a repo with a Photo.jpeg in it and a mapping for .jpeg to the JEPG type – do I get a file checked out called Photo (type JPEG), or Photo/jpeg (type JPEG).

The former might be “cleaner” from a purely RISC OS point of view, but then the code to open it will be looking for “photo.jpeg”. Unixlib will use “photo/jpeg” which work, so keeping the extension would seem to make more sense (and would avoid a clash with say photo.png).

However, what if you have a “Photo” (type JPEG). Do you call it “Photo.jpeg” in the repo, or use the Photo,c85 form? I guess it depends if you want it to work on other platforms, but even looking at it with a web browser onto github probably won’t get far if all the files end in ,xyz

Whatever is chosen I think it needs to be configurable per repo.

My 2p… well, probably 20p as it went on a bit .. sorry!

You need to be able to clone the ROOL repository, compile the sources and push any changes. So it needs the same file conversions that the ROOL repository uses. This should also mean that a RISC OS based repository clone on a Linux machine can be copied to RISC OS with LanManFS and compile – as it does now.

The solution as I see it (given what I currently know) is the need to introduce a RISC OS-specific “git filetype” command that will instruct git explicitly to record the filetype of a file. (This can be done in a way that will still allow the repository to maintain compatibility across other environments.)

For files in a git repository that carry filename extensions (e.g. README,fff or README,txt) git will not attempt to derive a filetype from the filename; only by use of the “git filetype” command will git officially begin recording/applying filename→filetype mappings.

By taking this approach, the RISC OS git client will look to provide mechanism, but not policy: existing repositories can continue as they currently do, whilst new repositories (or existing ones) may choose to introduce filetypes as appropriate.

There is slightly more detail behind the scenes than this, but hopefully this paints the direction being taken.

new repositories (or existing ones) may choose to introduce filetypes as appropriate.

How will that work for existing repositories? Specifically, will it be possible to retroactively apply it to the full history of a repository without changing the commit IDs? If the commit IDs change then that’s going to cause a headache for anyone who’s cloned/forked the repository. Or if it isn’t done retroactively, then the repository is going to be left in the awkward position where the repository won’t be usable on RISC OS if you go back in time too far.

We can’t retroactively apply changes like this without affecting commit ids as you say, so that idea would be problematic.

It may be that the RISC OS git client will have to implement the file extension concept, if that is going to be more useful. I’d need to just work through the detail, but roughly speaking it might be that when checking out a repository’s files, git would first look to see if the filetype has been applied using the newer “git filetype” process: if so, it will be able to apply the filetype based on the information it has stored, otherwise it could fall back to the secondary process of mapping a file’s type based on the filename’s extension. (The precise behaviour could likely be configurable in some way to ensure the user can control this appropriately).

Let me know if this makes for a more workable solution for everyone.

if so, it will be able to apply the filetype based on the information it has stored, otherwise it could fall back to the secondary process of mapping a file’s type based on the filename’s extension

That sounds good to me.

Great stuff, thanks for your input on this Jeffrey (and also Colin, Chris, Steffen and Rick). I’m much happier that I’ve got a clear direction at this point. Target is to see if I can get an alpha release ready for end of March. Will keep you all posted nearer the time.

Hi folks – bit of an update. In these difficult times, I’m still working on git. I’m still hopeful of releasing an alpha version by end of the month. I’ve got RISC OS filetypes covered with a specific RISC OS “git filetype” command and it seems be working. I’ve got a bunch of other core git commands largely in place. Still a couple to go at this early stage; specifically “git remote” and “git config” are the two I want to get SOME functionality in place for in time for an alpha release. Obviously Covid-19 is a huge issue for a lot of us right now, myself in particular because as of tomorrow (Monday 9am) I’m now a teacher for my kids as well. But just wanted you all to know that git continues to move forwards for RISC OS, and I’m anticipating that in the next couple of weeks I’ll be able to release “1.0.0-alpha.1” to the community, so that I can at least start getting some real input from you all as to how it’s starting to shape up. Hope that’s OK. Meantime, stay safe all of you, and between us all we’ll soon start kicking this project into real shape.

Thanks for the update Kevin.

“Todays lesson kids is how to check out files with git” :) We are in a similar situation here, trying to juggle working from home, looking after two little ones and staying sane..

git 1.0.0 – showstoppers not withstanding of course – due for release this week.

What’s the latest on this bounty? Would be great to have a fully functioning git client on RISC OS.

@ Andrew

ROOL is slowly reaching a good shape with the Git bounty, looks very promissing! :)

Meanwhile, if you are seeking for a git solution to use git ON RISC OS or WITH RISC OS, here are plenty of videos with all the available approaches while we wait for the full git client to be completed:

https://www.youtube.com/watch?v=65D1y4RrFA4&list=PLEnraaJ9VQfWDl5T4D0P51pG89KRzj0n1

Each video is for a different approach, hope this helps,

Thanks @Paolo I’ll give those videos a look.