Git

0.1 Truly distributed Git

The way to distribute your software is radicle

And maybe pillaging their code is a good way to bring up a DHT

0.2 Git submodules

Libraries are best dealt with as Git submodules.

Git submodules leak complexity and surprising and inconvenient behaviour all over the place if one is trying to make a change that affects multiple modules simultaneously. But having your libraries separate from your git repository results in non portable surprises and complexity. Makes it hard for anyone else to build your project, because they will have to, by hand, tell your project where the libraries are on their system.

When one is developing code, you normally have a git branch. But the git commit of the master project in which the submodule is contained does not notice its subproject has changed, unless the subproject head has changed. And the subject project head will not change if it points to a name, rather than to a particular commit. For ones changes to a submodule to be reflected in the master project in any consistent or predictable way, the submodule has to be in detached head mode, with the head pointing directly to a commit, rather than pointing to a branch that points to a commit.

You will import a submodule from someone else’s project, but eventually you and your team are going to make minor changes to it to customize it to your project. In which case you will need your own remote team repo, in place of the original other team’s repo.

Construct the copied remote repositories so that their default branch is your tracking branch, not the upstream branch.

git init --bare -b «our_branch»

Then, in your local repository where you created the new branch, reset the remote in your local repository to the new remote by editing .gitconfig and push «our_branch» from your local in which you created your team’s new submodule branch to the remote.

or

git clone --bare «their_remote» -b«their_stable_branch_or_release_tag»
cd «our_new_remote»
git symbolic-ref HEAD refs/heads/«our_new_branch»

If you fail to set your remote to your team’s default branch, then your local repositories will keep getting reset back to their team’s branch, and chaos ensues.

When moving the remote of a submodule in your local repository (usually from their team’s remote to your team’s remote) you update .gitmodules in your superproject, and in each submodule that has submodules of its own, then

git submodule update --init  --recursive --force
git submodule foreach --recursive 'git status && git switch «our_branch» && git status && git remote -v && git switch --detach'

Your submodule remotes propagate from .gitmodules file of your superproject, and their branch propagates from the superproject and from the remote default branch.

And the branch will also propagate from .gitmodules if branch = ... is set.

Because git is a distributed archive, it is perfectly possible, and often necessary, to work with all these set to different values, provided that everyone is mindful that they are set to different values, and the consequences and implications of them being set to different values. Which consequences and implications get complicated, unobvious, difficult to predict, and surprising when you are working with submodules.

Git commands in master project do not look inside the subproject. They just look at the subprojects head.

This means that signing off on changes to a submodule is irrelevant. One signs off on the master project, which includes the hash of that submodule commit.

When one is changing submodules for the use of a particular project, making related changes in the master project and submodules, one should not track the changes by creating and updating branch names in the submodule, but by creating and updating branch names in the containing module, so that the commits in the submodule have no name in the submodule, the submodule is always in detached head state, albeit the head may be tagged. Names in submodules are primarily of value for amendments to the submodule as an independent module, intended to be used by multiple projects, and for this purpose, tags are better than branch names. wxWidgets releases are identified by tag, not by branch, and the names of branches are only used to communicate a particular project on the submodule to other people working on that project as their master project.

Branch names within a submodule, though very useful when you working on a submodule, are not useful to the project as a whole, and except for the primary fork name, should be temporary and local., not pushed to the project repository, But when you are modifying the submodules in a project as a single project, making related changes in the module and submodule, the shared names that are common to all developers belong in the primary project module,and when you have done with a submodule,

git switch --detach

From the point of view of the containing superproject, submodule commits are nameless with detached head, except when you are working on them, the name in primary module naming a group of related commits in several submodules, which commits do not receive independent names of their own, even though the commits have to be made within the submodule, not in the containing module which names the complete set of interrelated commits.

The submodule commits may well belong to different branches and tags in the superproject, but the submodules know nothing of superproject names, and the superproject knows nothing of submodules names.

The primary build should invoke the submodule build, which will check each file in the submodule for changes, only when the submodule detached head has changed. And therefore, you want it to change, you want the submodule head to be nameless and detached, whenever you modify a submodule as part of a larger project where you test your changes by rebuilding the whole project to make sure all your related changes fit together.

When tracking an upstream submodule that has submodules of its own, which have their own upstreams

Update your version with

git pull upstream --recurse-submodules=on-demand «their-latest-release»

Make sure things still work. Get everything working. (You do have unit test, right?)

When you are working a submodule, your branch has to have a name, or when you push it and pull it, strange things will happen. But the superproject pushes and pulls by commit, not by name, so when you are done,

then:

git submodule foreach –recursive ’git push`

git submodule foreach --recursive 'git switch --detach'
git submodule foreach --recursive 'git push`

As its own thing, a submodule has branches with names. As a component of a superproject, it has nameless commits.

If you are in a submodule directory of the superproject, and you push and pull, what you are pushing and pulling had better have a name, or else unpleasant surprises will happen. If you are in the superproject directory and pushing and pulling the whole thing, that commit better be detached.

You pull a named release of the project that is a submodule of your project from upstream, diddling with it to make it work with your project, then you push it to origin under its own name, the you detach it from its name, so the superproject will know that the submodule has been changed.

All of which, of course, presupposes you have already set unit tests, upstream, origin, and your tracking branch appropriately.

Even if your local modifications are nameless in your local submodule repository, on your remote submodule repository they need to have a name to be pushed to, hence you need to have a tracking branch in each of your remote images of each of your submodules, and that tracking branch will need to point to the root of a tree of all the nameless commits that the names and commits in your superproject that contains this submodules point to.

You want .gitmodules in your local image of the repository to reflect the location and fork of your new remote repository, with your remote as its origin and their remote as its upstream.

You need an enormous pile of source code, the work of many people over a very long time, and GitSubmodules allows this to scale, because the local great big pile of source code references many independent and sovereign repositories in the cloud. If you have one enormous pile of source code in one enormous git repository, things get very very slow. If you rely someone else’s compiled code, things break and you get accidental and deliberate backdoors, which is a big concern when you are doing money and cryptography.

When your submodules are simply your copy of someone else code, it gets little bit messy. When you change them, it gets messier.

And visual studio’s handling of submodules is just broken and buggy. A command that works in git-bash will produce unexpected surprising, and unpleasant results in visual studio’s git. I really need to give up on visual studio, it is closed source code, and turning bad.

When one developer makes minor changes in submodule to make it work with the whole project on which several developers are working on, no end of mysterious grief ensues, because strange and curiously difficult to identify differences appear between builds that Git would normally ensure are the same build. Submodules are a halfway house between completely absorbing the other party’s code into your code, and using it as a prebuilt library. Instead, we have walls dividing the project into pieces, which is a lot less grief than on big pile of code, but managing those walls winds up taking a lot of time, and mistakes get made because a git commit in a project with submodules that have changed does not mean quite the same thing, nor have quite the same behaviour, as git commit in a project with unchanging submodules. But then truly integrating a project that is the product of a great deal of time by a great many of people, and managing it thereafter, is likely to take up a great deal more time.

Git Submodules is hierarchical, but source code has strange loops. The Bob module uses the Alice module and the Carol module, but Alice uses Bob and Carol, and Carol uses Alice and Bob. How do you make sure that all your modules are using the same commit of Alice?

Well, if modules have strange loops you make one of them the master, and the rest of them direct submodules of that master, brother subs to each other, and they are all using the same commit of Alice as the master. And you should try to write or modify the source code so that they all call their brother submodules through the one parent module above them in the hierarchy, that they use the source code of their brothers through the source code of their master, rather than directly incorporating the header files of their brothers at compile time, albeit the header file of the master that they include may well include the header of their brother, so that they are indirectly, through the master header file, including the brother header file.

0.3 Git subtrees

Git subtrees are an alternative to submodules, and many people recommend them because they do not break the git model the way submodules do.

But subtrees do not scale. If you have an enormous pile of stuff in your repository, Git has to check every file to see if it has changed every time, which rather rapidly becomes painfully slow if one is incorporating a lot of projects reflecting a lot of work by a lot of people. GitSubmodules means you can incorporate unlimited amounts of stuff, and Git only has to check the particular module that you are actually working on.

Maybe subtrees would work better if one was working on a project where several parts were being developed at once, thus a project small enough that scaling is not an issue. But such projects, if successful, grow into projects where scaling is an issue. And if you are a pure consumer of a library, you don’t care that you are breaking the git model, because you are seldom making synchronized changes in module and submodule, exept to absorb updates written by the upstream party or adjust build parameters that they thave made

The submodule model works fine, provided the divisions between one submodule and the next are such that one is only likely to make changes in one module at at time.