Git

1 Git

To set up Git on the cloud, see and to use git on the cloud see.

On my system, I ssh into the remote system «example.com» as the user git and then in the git home directory:

mkdir MyProject.git
cd MyProject.git
git init --bare

and on my local system I launch the git bash shell, and go to the MyProject directory. I copy a useful .gitinore and useful .gitattributes file into that directory, then launch the bash git shell

git init
git add *
git commit -m"this is a project to so and so"
git remote -v
git remote add origin git@«example.com»:~/MyProject
git remote -v
git push -u origin --all # pushes up the repo and its refs for the first time
git push -u origin --tags

Push,of course, requires that I have the ssh keys in putty format where putty can find them, and another copy in openssh format where git can find them. Git expects the ssh keys in .ssh

If you ssh into the other system instead of puttying into it, only need your keys in one place, which is simpler and safer

Invoke ssh-keygen -t ed25519 -C comment under git bash to automagically set up everything on the client side, then replace their private key with the putty key using putty key gen’s convert key, and their public key with the putty key gen copy and paste public key.

Make sure the config file ~/.ssh/config contains

Host «example.com»
    HostName «example.com»
    Port 22
    IdentityFile ~/.ssh/id_ed25519

Host is the petname, and HostName the globally unique name.
An example of the use of petnames is

Host project3
    User git
    HostName github.com
    IdentityFile ~/.ssh/project3-key
Host publicid
    User git
    HostName github.com
    IdentityFile ~/.ssh/publicid-key
Host github.com
    User git
    IdentityFile ~/.ssh/github-key

Putty likes its keys in a different format to git and open ssh, and created pageant and plink so that git and openssh could handle that format, but pageant and plink are broken. Convert format works, tplink hangs. Just make sure that there is one copy as expected by git and openssh, and one copy as expected by Putty.

Save the private key in ssh format with no three letter extension, and the corresponding public key in putty key gen’s copy and paste format with the three letter extension.pub

1.1 Git Workflow

You need a .gitignore file to stop crap from piling up in the repository, and because not everyone is going to handle eol and locales the same way, you need to have a .gitattributes file, which makes sure that text files that are going to be used in both windows and Linux are eol and utf-8, while text files that will be used only in windows are crlf

At github.com, create a new repository

cd \development\MyProject
git init
git config --global user.name studi-c
git config --global core.editor 'C:/Program Files (x86)/Notepad++/notepad++.exe' -multiInst -notabbar -nosession -noPlugin
git config --global user.email studio@digsig.net
git config --global core.autocrlf false
git remote add origin https://github.com/studi-c/nameofmynewrepository.git
git add --all --dry-run
git add --all
git diff --cache
git commit -m "Initial revision"
git push origin master

After I make a change and test it and it seems to work:

    git pull origin master

Test that the application still works after pulling and merging other developers’ changes

git diff .
git add .
git diff --staged HEAD
git commit -m "My change"
git push origin master

For an more complete list of commands, with useful examples.

To make a git repository world readable, you need git daemon running, but that a half measure, for if you publish your code to the world, you want the world to contribute, and you will need gitlab to manage that.

A simpler way of making it public is to have the post-update hook turn it into a old plain dumb files, and then put a symlink to your directory in the repository in your apache directories, whereupon the clone command takes as its argument the directory url (with no trailing backslash).

1.1.1 Merge Conflicts

Resolving a merge conflict is usually a little bit messy, creating opportunities to make mistakes

The safest way to resolve a merge conflict is to use git mergetool, followed by git gui.

git gui is useless for resolving merge conflicts, but after having resolved them in your mergetool, then when you click on commit in git gui, you can see that the situation is what you think it is before issuing the commit command.

In linux, git gui is not necessarily available unless you have installed not just git, but git-all

The best mergetool by far is Meld. But it is preferable to issue the commit command from within git-gui, since you get a more global view of the situation.

1.2 Sharing git repositories

1.2.1 Git Daemon

Now integrated into Git, needs no separate install.

When correctly set up, the url for repo foo.git is git://host/foo

git-daemon will listen on port 9418. By default, it will allow access to any directory that looks like a git directory and contains the magic file git-daemon-export-ok.

This is by far the simplest and most direct way of allowing the world to get at your git repository.

Git documentation for Git daemon

Git Book for Git daemon

When you have submodules, an issue arises with people accessing the submodules through different protocols. The fix is to make .gitmodules protocol agnostic by using relative urls. As for example

[submodule "libsodium"]
        path = libsodium
        url = ../libsodium.git
        branch = rho-fork
[submodule "mpir"]
        path = mpir
        url = ../mpir.git
        branch = rho-fork
[submodule "wxWidgets"]
        path = wxWidgets
        url = ../wxWidgets.git
        branch = rho-fork

This will enable recursive cloning and all that to work regardless of the root protocol.

After amending the .gitmodules file you need to execute the command git submodule sync, and now that you are using relative urls, when the base url changes, git submodule sync --recursive

1.2.2 Gitweb

Does much the same thing as git-daemon, makes your repository public with a prettier user interface, and somewhat less efficient protocol.

Gitweb provides a great deal of UI for viewing and interacting with your repository, while git-daemon just allows people to clone it, and then they can look at it.

1.2.3 gitolite

It seems that the lightweight way for small group cooperation on public projects is Gitolite, git-daemon, and Gitweb.

Gitolite allows you to easily make people identified by their ssh public key and the filename of the file containing their public key write capability to certain branches and not others.

On Debian host apt-get install gitolite3, though someone complains this version is not up to date and you should install from github.

It then requests your public key, and you subsequently administer it through the cloned repository gitolite-admin on your local machine.

It likes to start with a brand new empty git account, because it is going to manage the authorized-keys file and it is going to construct the git repositories.

Adding existing bare git repositories (and all git repositories it manages have to be bare) is a little bit complex.

So, you give everyone working on the project their set of branches on your repository,and they can do the same on their repositories.

This seems to be a far simpler and more lightweight solution than Phabricator or Gitlab. It also respects Git’s inherently decentralized model. Phabricator and Gitlab provide a great big pile of special purpose collaboration tools, which Gitolite fails to provide, but you have to use those tools and not other tools. Gitolite seems to be overtaking Phabricator. KDE seems to abandoning Phabricator:

The KDE project uses gitolite (in combination with redmine for issue tracking and reviewboard for code review). Apart from the usual access control, the KDE folks are heavy users of the “ad hoc repo creation” features enabled by wildrepos and the accompanying commands. Several of the changes to the “admin defined commands” were also inspired by KDE’s needs. See section 5 and section 6 of the above linked page for details.

So they are using three small tools, gitolite, redmine, and reviewboard, instead of one big monolithic hightly integrated tool. Since we are creating a messaging system where messages can carry money and prove promises and context, the eat-your-own dogfood principle suggests that pull requests and code reviews should come over that messaging system.

Gitolite is designed around giving identified users controlled read and write access, but can provide world read access through gitweb and git-daemon.

1.2.4 Gitea and Gogs

Gitea is the fork, and Gogs is abandonware. Installation seems a little scary, but far less scary than Gitlab or Phabricator. Like Gitolite, it expects an empty git user, and, unlike Gitolite, it expects a minimal ssh setup. If you have several users with several existing keys, pain awaits.

It expects to run on lemp. Install Lemp stack on Debian

Gitea Install instructions:

It’s default identity model is username/password/email, but it supports the username/ssh/gpg user identity model. The user can, and should, enter an ssh and gpg key under profile and settings / ssh gpg keys, and to prevent the use of https/certificate authority as a backdoor, require commits to be gpg signed by people listed as collaborators.

Git now supports signing commits with ssh keys, so probably we should go on the ssh model, rather than the gpg model, but I have not yet deleted the great pile of stuff concerning gpg because I have not yet moved to ssh signing, and do not yet know what awaits if we base Gitea identity on ssh keys.

It can be set to require everything to be ssh signed, thus moving our identity model from username/password to ssh key. Zooko minus names instead of minus keys

If email is enabled, password reset is by default enabled. Unfortunately email password reset makes CA system the root of identity so we have to disable it. We need to make gpg the root of identity, as a temporary measure until our own, better, identity system is working.

It is development model is that anyone can fork a repository, then submit a pull request, which request is then handled by someone with authority to push to that repository.

gpg signatures should, of course, have a completely fake email address, because the email address advertised in the gpg certificate will be spammed and spearphished, rendering it useless. But gpg is completely designed around being used with real email addresses. If you are using it to sign, encrypt, and verify emails through its nice integration with the thunderbird mail client, you have to use real email addresses.

Uploading a repository to Gitea is problematic, because it only accepts repositories on other Gitea, Gog, Gitlab, and Github instances. So you have to create a fresh empty repository on Gitea, set it to accept an ssh key from your .ssh/config, and merge your existing repository into it.

Download the empty gitea repository. Make it even emptier than it already is with git rm. Because the histories are unrelated, almost anything with the same name will cause a merge conflict.

git rm *
git commit -am "preparing to merge into empty repository"
git push

Got to the real repository, and merge the empty gitea repository on top of it:

git pull myemptygitearepository  --allow-unrelated-histories
# The histories are no longer unrelated, so push will work.

We are doing this in the real repository, so that the original history remains unchanged - we want the new empty gitea repository on top, not underneath.

Comes with password based membership utilities and web UI ready to roll, unlike Gitolite which for all its power just declares all the key management stuff that it is so deeply involved in out of scope with the result that everyone rolls their own solution ad-hoc.

These involve fewer hosting headaches than the great weighty offerings of GitLab and Phabricator. They can run on a raspberry pi, and are great ads for the capability of the Go language.

Gitea, like Gitolite, likes to manage people’s ssh keys. You have to upload your ssh and gpg public key as part of profile settings. Everything else, is unfortunately, password based and email based, which is to say based on the domain system and certificate authority system, which is an inherent massive security hole, since the authorities can seize the website and put anything on it they want. Therefore, need to use the Gitea signing feature

This seems to require Gitea to do a lot of signing, which is a gaping security hole, though it can be used with the gpg feature allowing short lived subkeys.

Will sign with subkey, because MasterKey should not be available on the server

1.2.5 Phabricator

Too fat, and not exactly open source.

Server Size : 2GB Ram – 1 CPU Core – 50GB SSD If you have more than five users, this may not suffice, but you can limp along OK with one gigabyte.

Installation and configuration of Phabricator (which is also likely to be useful in installing PhpAdmin, because it covers configuring your “MySQL” (Actually MariaDB) database.

Configuring Phabricator requires a whole lot of configuration of ssh and git, so that ssh and git will work with it.

Phabricator notifications require node.js, will not run with apache. Ugh. But on the other hand, Comen needs node.js regardless. But wordpress requires PHP, not sure that is going to play nice with node.js. Node.js does not play well with apache PHP, and Phabricator seems to use both of them, but likely only uses node.js for notifications, which can wait. Usual gimmick is that to use Apache’s ProxyPass directive to pass stuff to node.js. Running both node.js and apache/PHP is likely to need a bigger server. Apache 2.4.6 has support for proxying websockets mod_proxy_wstunnel

The Phabricator website suggests nginx + php-fpm + “MySQL” (MariaDB) + PHP. Probably this will suffice for Wordpress. nginx is the highest performance web server, but it is not node.js

Apache, node.js, and nginx can all coexist, by routing stuff to each other (usually with the highest performing one, nginx, on top) but you will need a bigger server that way

Blender is a huge open source success, breaking into the big time, being free and open tool for threedee drawing. It’s development platform is of course self hosted, but not on Gitlab, on Phabricator. Maybe they know what they are doing.

Phabricator is a development environment provided by Phacility. It is written in in PHP, in a phabricator development environment, which is designed to support very large development communities and giant corporations. It is written in PHP, which makes me instantly suspicious. But it is free, and, unlike GitLab, genuinely open source.

Being written in PHP, assumes a Lamp stack, apache2, php 5.2 or later, mysql 5.5 or later

Phabricator assumes and enforces a single truth, it throws away a lot of the inherent decentralization of git. You always need to have one copy of the database, which is the King – which people spontaneously do with git, but with git it is spontaneous, and subject to change.

KDE seems to be moving away from Phabricator to Gitolite, but they have an enormously complex system, written in house, for managing access, of which Gitolite is part. Perhaps, Phabricator, being too big, and doing too much was inflexible and got in their way.

Github clients spontaneously choose one git repository as the single truth, but then you have the problem thatGithub itself is likely to be hostile. An easy solution is to have the Github respository a clone of the remote repository, without write privileges.

1.2.6 Gitlab repository

Enemy controlled.

Git, like email, automatically works – provided that all users have ssh login to the git user, but it is rather bare bones, better to fork out the extra cash and support gitlab – but gitlab is far from automagic, and expects one git address for git and one chat address for matterhorn, and I assume expects an MX record also.

Gitlab is a gigantic memory hog, and needs absolute minimum of one core and four gig, and two cores and eight gig strongly recommended for anything except testing it out and toying with it. It will absolutely crash and burn on less than four gig. If you are running gitlab, no cost advantage to running it on debian. But for my own private vpn, huge cost advantage.

Gitlab absolutely requires postgreSQL. They make a half assed effort to stay MySQL compliant, but fall short. Not a big disk space hog – ten gigabytes spare will do, so fine on thirty two gigabyte system.

Gitlab Omnibus edition contains the postfix server, thus can send and receive email to is host address

Setup Gitlab with Protected branch flow

With the protected branch flow everybody works within the same GitLab project. The project maintainers get Maintainer access and the regular developers get Developer access. The maintainers mark the authoritative branches as ’Protected’. The developers push feature branches to the project and create merge requests to have their feature branches reviewed and merged into one of the protected branches. By default, only users with Maintainer access can merge changes into a protected branch. Each new project requires non trivial manual setup.

But, seems likely to be easier just to use the main gitlab site, as least until my project is far enough advanced that the evil of github is likely a threat.

Gitlab is intended to be hosted on your own system, but to learn how to use it in a correctly configured gitlab environment, and to learn what a correctly configured gitlab environment looks like and how it is used, going to need an account on gitlab.

Gitlab requires that the Openssh port 22, the http port 80, and the https port 443 be forwarded. Http should always get automatic redirect to hppts governed by a lets encrypt certificate.

GitLab Mattermost expects to run on its own virtual host. In your DNS you would then have two entries pointing to the same machine, e.g. gitlab.«example.com» and mattermost.«example.com». GitLab Mattermost is disabled by default, to enable it just put the external url in the configuration file.

Github, on the other hand, allows you to point your own domain name to your custom (static) github website and git repository as if on your own system.

I am suspicious of placing your own website on someone else’s system, especially a system owned by social justice warriors, and the restriction to static web pages is likely intended to facilitate political censorship and law enforcement, and physical attacks on dissidents by nominally private but state department supported thugs and Soros supported thugs.

1.2.7 Gitlab on the cloud

Omnibus edition of gitlab, usually available already configured to a cloud provider.

Instructions for installing it yourself.

It is like advisable to setup apache, the apache virtual hosts, and the apache certificates first.

Gitlab markdown, like Github markdown, can mostly succeed in handling html tags.

We are not going to build on the cloud, but we will have source code, chat, and code of conduct on the cloud, probably on git.cpal.cw and chat.cpal.cw

Configuring gitlab is non trival. You want anyone to be able to branch, and anyone to be able to issue a pull request, but you only want authorized users to be able to merge into the master.

Since we want to be open to the world, implement recaptcha for signups, but allow anyone in the world to pull without signing up. To create a branch, they have to sign up. Having created a branch, they can issue a pull request for an authorized user to pull their branch into the master, they have to sign up.

Gitlab workflow is that you have a master branch with protected access, a stable branch with protected access, and an issue tracker.

You create the issue before you push the branch containing fixes for the issue. There must be at most one branch for every issue.

Developers create a branch for any issues they are trying to fix, and their final commit should say something like “fixes #14” or “closes #67.” and then, when the branch is merged into the master, the issue will be marked fixed or closed.

Digital Ocean offers a free entry, and a quite cheap system

But virmach, even cheaper.

Eight gig, two cores, which you will need to run gitlab for everyone, is Debian 9 enterprise edition on virmacht.com $40 per month.

Also, vpn on the cloud.

Currency project should be hosted on digital ocean at git.«example.com», at $20 per month (Four gig, two cores), using Gitlab free omnibus edition. They suggest configuring your own Postfix email server on the machine also, but should this not be automatic? Probably already in the DigitalOcean Gitlab droplet.

How to us the gitlab one click install image to manage git repositories.

You will need to get your SSL certificate from cyberultra and supply it to Gitlab (though gitlab has built in lets-encrypt automation, so maybe not).

Subdomains are a nameserver responsibility, so you really have to point your domain name nameservers to cyberultra, or else move everything to digital ocean.

All digital ocean ips, except floating ips, are static. You will need an A record and an AAAA record, the AAAA record for IP6

Gitlab did not want to support fully browsable public repositories, but they have been supported since 6.2

Gitlab omnibus edition comes integrated with Mattermost, but mattermost is turned off by default.

Before spending money and going public, you might want to install locally and run on your local system. Enable mattermost.

Since gitlab will have the root web page on git.jim.com, you will need another DNS entry pointing at the same host, something like chat.jim.com. So, though both on the same machine, one is the root http page when accessed by one domain name, the other the root entry when accessed by another domain name.