Git

Introduction

The primary source code management system for the Mu2e Offline software is a git repository that is hosted on the Fermilab Redmine site. Git is a popular open-source software version control system and Redmine is a software project management system which contains, for each project, a log, a wiki (which may contain useful info), and possibly gantt charts and bug tracking along with the git respositories.

Based on the git repository's url, you can check out code, and, if you are a code developer, check in modifications. NB, when committing code, please follow these mu2e conventions only. Git also has capabilities for tagging code, tracking history and supports splitting off and merging development branches, among many other features.

Depending on your experience or needs, please also see the basic code recipe. You can continue with the quick start on this page or the expanded git introduction page. See the references for more details on Redmine and git.

Some older parts of Mu2e software is managed using other source code management systems. Some code, both critical and specialized, is in other, smaller repositories. See Repositories below for more details on repositories.

Quick Start

Please also see the basic code recipe for an introduction to setting up the environment, checking out code and building.

If you are going to commit anything, you must understand and follow the mu2e commit instructions

If this is the first time you are using git in the mu2e environment, please follow the few steps listed in order to set up your preferences in ~.gitconfig.

A more detailed introduction to all mu2e repositories is here and our introduction to git is here.

• checkout the main repository
  • readonly:

    git clone http://cdcvs.fnal.gov/projects/mu2eofflinesoftwaremu2eoffline/Offline.git

  • with kerberos authentication for committing code

    git clone ssh://p-mu2eofflinesoftwaremu2eoffline@cdcvs.fnal.gov/cvs/projects/mu2eofflinesoftwaremu2eoffline/Offline.git

• checkout the head to a local working branch, here named "work" (recommended)

git checkout -b work

• see the history of a file

git log fileSpec

• list tags

git tag -l

• checkout a tag or branch or commit hash into the working area. Use this to switch between branches, it will NOT overwrite your work.

git checkout tagOrBranchNameorHash 

• update the local repository to the origin repository (Redmine)

git fetch

• update the local repository to changes staged in the working area (does not affect the origin repository)

git commit -m"'Your descriptive message'"

• update the origin repository (Redmine) to changes in your local repository

git push

• update the local repository to the origin repository (REDMINE)

git fetch

• update the working area to the local repository

git merge

• update the local repository and update the working area both together

git pull

• merge a branch (including master) into your branch. NB: do NOT merge your branch into master unless authorized

git checkout otherbranch 
git pull
git checkout mybranch
git pull
git checkout mybranch
git merge otherbranch

• see what the status is

git status

• command line help

git status help

• git browser (to see a graphical history of commits)

gitk --all

Please see the references below or many web resources for details on git concepts, commands and features. If you are going to commit anything, you must understand and follow the commit workflows.

Workflows for committing code

As the group of mu2e code developers grows at the same time that the need for a stable code base increases, individual code contributions may start to clash with other contributions, so we are using a "request to pull" procedure. In this procedure, the developer does code development on a git branch and commits that branch to the repository frequently. This branch does not affect other users, so it can be in any state the developer finds useful - it might not even compile. Once the developer believes the work is complete, they make a request to the Offline group heads to merge or "pull" the branch into the master branch. If the change is very simple or low-risk, the heads may ask the developer to go ahead and do that merge. If the change is large or complex, they may request additional documentation or testing before the pull. The branch should be prepared so it includes the head of the master branch, so it can be merged with "fast forward".

The procedure of developing on a branch, then requesting to merge is explained in detail on the git workflow page.

If a change is very small, or planned in advance, or urgent, the Offline group heads may allow the commit to follow the minor workflow which ends in a commit directly to the head of master.

Branch prompt

Many people work in a style where they are switching between branches frequently, and everyone does this switch at times. You can put the branch name in your prompt so you are unlikely to get confused about what branch you are on.

In .bash_profile:

parse_git_branch() {
    git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* (.*)/(1)/'
}

export PS1="\h \w $(parse_git_branch) "

Finding deleted files

As a policy we have decided to not keep code that does not compile. Some code may be useful as examples or to revive an abandoned effort, so we should be able to recover them. Here are three methods:

1. grep the release notes
grep mystring ReleaseNotes/*/*
2. if you know the file name, look at its history
git log myfile
3. search git history. The following git command will list every commit that contained the deletion of a a file and it will list the names of all files deleted in that commit:
git log --diff-filter=D --summary and can be pipe do to grep

Once you find the file name, you can check it out based on the commit that deleted it.

git checkout <deleting_commit>^ -- <file_path>

If you are deleting non-trivial files, please note the full file paths in the releases notes.

Repositories

One important piece of the mu2e offline code is the BTrk UPS product, which contains our Kalman track fit code, and is stored in the BTrk github repo.

All of the rest of the mu2e Offline code and tools are in Redmine repositories. You can browse all Redmine projects and mu2e Redmine. The mu2e Redmine has subprojects (and sub-subprojects) and each has a repository associated with them. For example in the Standalone subproject, there is a sub-subproject Mu2eCCFC. In addition, the main mu2e Offline repository has sub-repositories which can be seen on the right sidebar menu on the repository main page. An example is the dhtools sub-repo.

Here is a (possibly incomplete) list of the repositories.

content	user	name	alias
	p-mu2e_artdaq	mu2e_artdaq
raw data format	p-mu2e_artdaq	mu2e_artdaq-core
F-C limit setting	p-mu2eccfc	mu2eccfc-mu2eccfc	mu2eccfc.git
CI validation	p-mu2e_ci	mu2e_ci
	p-mu2e-daq-firmware	mu2e-daq-firmware
	p-mu2e-daq-firmware	mu2e-daq-firmware-dtc
	p-mu2e-dcs	mu2e-dcs
	p-mu2e_detector_construction_db	mu2e_detector_construction_db
	p-mu2eextmontestdaq	mu2eextmontestdaq
grid tools	p-mu2egrid	mu2egrid
Offline	p-mu2eofflinesoftwaremu2eoffline	mu2eofflinesoftwaremu2eoffline	Offline.git
Jenkins	p-mu2eofflinesoftwaremu2eoffline	mu2eofflinesoftwaremu2eoffline-codetools	codetools.git
data handling	p-mu2eofflinesoftwaremu2eoffline	mu2eofflinesoftwaremu2eoffline-dhtools	dhtools.git
	p-mu2eofflinesoftwaremu2eoffline	mu2eofflinesoftwaremu2eoffline-geomdisplay	geomdisplay.git
satellite releases	p-mu2eofflinesoftwaremu2eoffline	mu2eofflinesoftwaremu2eoffline-satellite
gridexport product	p-mu2eofflinesoftwaremu2eoffline	mu2eofflinesoftwaremu2eoffline-gridexport	gridexport.git
	p-mu2eofflinesoftwaremu2eoffline	mu2eofflinesoftwaremu2eoffline-test
obsolete	p-mu2eofflinesoftwaremu2eoffline	mu2eofflinesoftwaremu2eoffline-validation	validation.git
Stntuple framework	p-mu2eofflinesoftwarestntuple	mu2eofflinesoftwarestntuple	Stntuple.git
	p-mu2eofflinesoftwareuser	mu2eofflinesoftwareuser-betasource	betasource.git
	p-mu2eofflinesoftwareuser	mu2eofflinesoftwareuser-example01	example01.git
	p-mu2eofflinesoftwareuser	mu2eofflinesoftwareuser-streamline
	p-mu2epbarmodels	mu2epbarmodels
	p-mu2epbaridanalysis	mu2epbaridanalysis
	p-mu2e-raw-data	mu2e-raw-data
	p-mu2e-simulations	mu2e-simulations
	p-mu2estat	mu2estat
	p-mu2e-tools	mu2e-tools-bintools	mu2ebintools
FileTools	p-mu2e-tools	mu2e-tools-filetools	mu2efiletools
FileTools	p-mu2e-tools	mu2e-tools-mu2efilename	mu2efilename
generate_fcl	p-mu2e-tools	mu2e-tools-tools	mu2etools

Creating a Redmine subproject is done by the "add subproject" button on the Redmine project web page. This is probably more appropriate for a project not closely associated with the Offline. Creating a sub-repository associated to the main offline repo is explained in the mu2e instructions and SCD instructions. This is probably more appropriate for a project closely associated with the Offline. Write privileges to each repository can be controlled by the .k5login contents of each group username associated with the repo - note that this file should be edited by the Redmine project web page (settings->members), not directly. The login to the group account is controlled by the .admin file.

A git repository is accessed through a url and the url can be read-only or read/write, which allows commits back to the repo. Taking values from the table of Redmine repos above, a repo without an alias will have read-only and read/write urls:

http://cdcvs.fnal.gov/projects/name
ssh://user@cdcvs.fnal.gov/cvs/projects/name

and if it has an alias:

http://cdcvs.fnal.gov/projects/name/alias
ssh://user@cdcvs.fnal.gov/cvs/projects/name/alias

A good view of all repositories can be found by logging is as one of the user names where you have write permissions:

ssh user@cdcvs.fnal.gov

The projects can be seen

cd /cvs/projects
ls -l | grep mu2e

there is one directory per repository and the username can be seen as the owner of the repository directory. If the repository has an alias, it will appear as a subdirectory or a circular link under

/cvs/projects/name

The .k5login file for a repository is

/home/name/.k5login

GitHub

Forking

When you start work on a repo, you fork it into your own account using buttons on the GitHub website. To start making changes, you clone your fork to a local directory.

git clone https://github.com/your_use_name/repo_name_here.git

Make commits to your fork. Create pull requests from your commits.

You will need to check if the original repo have moved ahead of your work. First, point to the original repo:

git remote add upstream https://github.com/Mu2e/TestOffline.git
git remote -v

Then pull the head of master from there, and add it to your fork:

git fetch upstream
git co upstream/<branch>
git co -b <branch>
git push -u origin <branch>

Create new repo

Creating a new repo. First ceate the repo on GitHub, then checkout the old repo locally, you will be pushing the local repo into the GitHub repo

1. create the repo on github, here TestOffline
2. Fetch all of the remote branches and tags:
git fetch origin
3. View all "old repo" local and remote branches:
git branch -a
4. If some of the remotes/ branches doesn't have a local copy, checkout to create a local copy of the missing ones:
git checkout -b <branch> origin/<branch> Now we have to have all remote branches locally.
5. Add a "new repo" as a new remote origin:
git remote add new-origin git@github.com:user/TestOffline.git
6. Push all local branches and tags to a "new repo" (note we're pushing to new-origin):
git push --all new-origin
7. Push all tags:
git push --tags new-origin
8. Remove "old repo" origin and its dependencies. View existing remotes (you'll see 2 remotes for both fetch and push)
git remote -v
9. Remove "old repo" remote:
git remote rm origin
10. Rename "new repo" remote into just 'origin':
git remote rename new-origin origin

References

• redmine git troubleshooting
• mu2e git and CVS
• mu2e minor commit workflow
• mu2e major commit workflow
• old redmine commit instructions
• git home Documentation reference
• redmine repository browser
• Fermilab redmine
• talk at the Software and Simulation Meeting, Sept 17, 2014.
• Marc P's suggested git workflow for simple git projects
• Fermilab git workshop and related materials
• managing and creating new redmine repositories