GitPartialCheckout

Introduction

Following the standard git checkout patterns results in cloning and checking out locally the entire Offline repository. When the entire repo is built, it can take a an annoyingly long time. Partial checkout is one powerful way to mitigate this build time. In partial checkout, you rely on a base Offline build for most of the Offline libraries, and you only checkout a little bit of Offline locally and build that subset of code as you work. When you build, if a needed header file is found locally, that will be used, but if it is not in the local working area, then it will be found in the base build. After building, the exe's will pick up your local libraries first, then link the rest from the base release. The same pattern of including your local working area in a path before the base build, is followed for all the paths (fcl, python, data, bin, etc).

The local area is a valid, complete git repo, so you can pull, push, tag, etc. while you are building a subset of the code. Commands and concepts such as pull, tag, hashes, branches, etc, which normally apply to an entire repo, will still apply to the whole repo, not just the check-ed out part.

To implement the partial checkout, we have provided a script called "pgit" which is in your path after "setup mu2e". We have also provided a set of pre-built Offline areas, to serve as the base builds. These can be provided for the head of selected branches.

Two Warnings

The downside to not building the entire repo locally is that your local partial build may get out of sync with the base build. If code in the base release is compiled with a different header file than the local partial build, then then executables will not run correctly, and will probably result in odd memory errors, such as seg faults. It is also possible that the code will fail silently, so these are very serious issues.

There are probably other ways to cause trouble, but all users must be aware of the following two issues.

First is the issue of intermediate commits. For example, the base build is at a certain commit, call it N, and you create a partial checkout based on this base build. After you are setup, someone else commits N+1 to the branch you are working on. In terms of the git repo, you can handle this like a full repo - you can pull or merge the new commit into your local repo, work on that and commit N+2 to the head of the branch. The problem is that when you do pull the intermediate commit N+1, the header files in your working area may become inconsistent with the header file used for the base build, leading to serious errors. Also, you may have checked out a subset of code X, when the intermediate commit concerned disjoint subset Y, so you will not see the effect of the intermediate commit at all.

If you see an intermediate commit, then you can "git diff" and decide if it is harmless with respect to dependencies. For example, if the commit was only to a cc file, then you can checkout this part locally if you want to get its effect, or ignore it in your local checkout, if you are sure it doesn't matter to your work.

The second major issue concerns your local header files. If you modify a local header file, the only way to get a fully correct build is to recompile every piece of code that includes that header file and what depends on this header may not be at all obvious.

"pgit check" provides some checks for these problems, but there is no substitute for being personally aware of these dependency issues. There are probably other ways to cause trouble, but all users must be aware of the following two issues. When in doubt, you can always "pgit quit" and go back to a normal full checkout, disconnecting from the base build.

Commands

See the base builds available:

> pgit list
2018-12-11 22:48 master/4bc67ad0/SLF6/prof
2018-12-11 22:41 master/4bc67ad0/SLF6/debug

The hex is the first 8 char of a commit hash. The list is presented with the most recent at the top.

To start a new partial build, or to convert a full checkout into a partial checkout

pgit init master/4bc67ad0/SLF6/prof

to checkout a directory:

pget get