gitcache-ssh

gitcache-ssh is a simple SSH based Git cache.

This tool was conceived and written to dramatically reduce the bandwidth
consumed by developers and CI/CD build and test systems that constantly
communicate with geographically remote Git repositories.

It is designed to be (for the most part, after initial setup), transparrent to
the user or automation systems that are using it.

It provides an asymetric (pull/read-only) cache function to shim between the Git
client and the originating remote upstream Git repository.

Building

$ make
$ make deb
$ make rpm

Prerequisites

• A valid working Golang build environment. Don’t forget to define your
  $GOROOT, $GOPATH and $GOBIN environment variables.
• The FPM packaging tool.
• An OpenSSH server.

Installation

Create a suitable /var/cache/git/ directory to store the local cached copies
of remote Git repositories.

# mkdir -p /var/cache/git/

Add a suitable Match Group or Match User block to your OpenSSH
/etc/ssh/sshd_config configuration file to force the gitcache-ssh wrapper
command to be executed upon login instead of the user’s shell.

For example, to enable the gitcache-ssh command for all users who are a member
of the developers group:

# cat << 'EOF' >> /etc/ssh/sshd_config
AcceptEnv LANG LC_* GIT_CACHESSH_*
Match Group developers
  AllowTCPForwarding no
  X11Forwarding no
  ForceCommand /usr/local/bin/gitcache-ssh
EOF
# systemctl restart ssh

Client Configuration

Git clients need to be configured to pull from the cache for specific
repositories or hosts. This is done using the insteadOf and pushInsteadOf
configuration directives.

The following example will enable the cache for SSH-based access to all
GitHub.com repositories:

$ git config --global --add url."ssh://gitcache.host/git@github.com:".insteadOf "git@github.com:"
$ git config --global --add url."ssh://gitcache.host/git@github.com:".insteadOf "ssh://git@github.com/"
$ git config --global --add url."git@github.com:".pushInsteadOf "git@github.com:"
$ git config --global --add url."ssh://git@github.com/".pushInsteadOf "ssh://git@github.com/"

Enabling for HTTP and HTTPS-based access can be achieved by simply modifying the
URLs that are being re-written:

$ git config --global --add url."ssh://gitcache.host/git@github.com:".insteadOf "https://github.com/"
$ git config --global --add url."ssh://gitcache.host/git@github.com:".insteadOf "http://github.com/"
$ git config --global --add url."https://github.com/".pushInsteadOf "https://github.com/"
$ git config --global --add url."http://github.com/".pushInsteadOf "http://github.com/"

The git config -l | grep insteadof command will display your current URL
re-writing configuration.

Force Cache Refresh from Upstream Origin

The gitcache-ssh wrapper can be forced to refresh the local cache by setting
the environment variable GIT_CACHESSH_SYNC=true. The OpenSSH client will not
forward this environment variable to the remote SSH Git cache host by default,
so a small change is necessary either to the local client’s ~/.ssh/ssh_config
file, or to the SSH command line that Git will invoke.

Ad-hoc command line based solution:

$ GIT_CACHESSH_SYNC=true GIT_SSH_COMMAND="ssh -o SendEnv=GIT_CACHESSH_*" git pull

Permanent configuration based solution:

$ cat >> ~/.ssh/config <<EOF
Host gitcache.host
  SendEnv GIT_CACHESSH_*
EOF
$ GIT_CACHESSH_SYNC=true git pull

To-do

• Replace fpm packaging with native RPM spec file and Debian package
  configuration directory.
• Complete documentation.
• Tidy up logging.
• Address TODO and FIXME mentioned in the source.
• Vendor dependencies for easier reproducable builds.
• Add unit tests.
• Add man page.
• Move as many constants as possible out of the Go binary, into a configuration
  file, or to be read from environment variables.
• Add support to pass-through to the users’ default shell if it is detected
  that the SSH client is not a git command.
• Add installation helper script to configure OpenSSH for you.
• Setup CI pipeline in Travis CI.
• Setup code analysis with CodeClimate.

License

BSD 3-Clause “New” or “Revised” License.

A permissive license similar to the BSD 2-Clause License, but with a 3rd clause
that prohibits others from using the name of the project or its contributors to
promote derived products without written consent.

See LICENSE file for full details.

Author

Nicola Worthington nicola.worthington@nokia.com, nicolaw@tfb.net.