GitHubWorkflow

Introduction

This page describes a recommended git workflow for use with the Mu2e Offline code in GitHub.

The Mu2e computing group manages the Mu2e organization within GitHub. To have full access to Mu2e software, Mu2e collaborators should create a GitHub account, and request to join the Mu2e organization.

The main Mu2e repository is 'Offline', which contains the algorithms, data structures, art modules and configuration used in simulation production, online filtering, offline reconstruction, and other tasks. The main development branch within Offline is 'master', which should be used for all code development. Offline also contains legacy branches associated with particular data sets, which are maintained for a limited time after the master branch has been developed past those data, as described in the table below.

Mue2/Offline branches

branch name	branch purpose	support end date
master	development	end of Mu2e
MDC2018	MDC2018 dataset support	end of 2020

Authentication

If you are following the instructions for accessing Offline as a user, not a developer, you will never need to authenticate yourself to github. If you will do work as a developer than there will be some steps for which you will need to authenticate yourself to github. The page Authentication#Authenticating to github has instructions on two ways to do so

Migrating an existing redmine clone directory to GitHub

You can repoint an existing Offline clone from redmine to GitHub as a way of making an easy transition. To follow these instructions, you must already have a GitHub account that's registered in the Mu2e Organization. The following assumes your redmine clone is in /mu2e/app/Me/MyRedmineClone/Offline, that your GitHub username is MyGitHubName, and that you are working on a branch called MyDevelopmentBranch.

1. Login to GitHub
2. Fork Mu2e/Offline in GitHub using the 'Fork' button on the top right of the Mu2e Offline repo page
3. Add GitHub as a remote to your clone

   > cd /mu2e/app/Me/MyRedmineClone/Offline

   > git remote rename origin Redmine

   > git remote add mu2e https://github.com/Mu2e/Offline

   > git remote add origin https://github.com/MyGitHubName/Offline

   These commands are very fast as no data is actually transferred. You can see your remotes with:

   > git remote -v

   Your clone is now connected to the main Mu2e github repository and your own fork in addition to the Redmine repository. Additionally, we have changed your fork to be the default remote repository (origin)

4. Push your working branch to your GitHub fork:

   > git push -u origin MyDevelopmentBranch

After this, you can continue developing in this directory according to the general GitHub workflow described below.

Downloading Offline as a user and NOT a developer

Option 1: you want the default primary version of the code (most people):

1. clone the repo:
git clone https://github.com/mu2e/Offline
2. Done!

Option 2: A particular collaborator has a version or branch you want to use:

1. Find their github user name
2. Determine what the name of the branch they are working on is (Note: this can be master!)
3. Clone their fork:
git clone https://github.com/<user name>/Offline cd Offline git checkout origin/<branch name>
4. Done!

Option 3: You want to use pgit to avoid a long compilation time (EXPERIMENTAL)

1. Create new directory to put your Offline repo in and move to that directory
mkdir Offline cd Offline
2. As in Option 2, determine fork and branch name you wish to use
3. Create a partial checkout clone
pgit2 setup https://github.com/<user name (or mu2e)>/Offline <branch name>
4. You can now use as normal:
source setup.sh scons -j 4

Developer Workflow

• To start, you will want to make sure you understand what a remote is, and the difference between a branch and a fork

1. Make sure you have a GitHub account, and that you are added to the Mu2e GitHub organization (https://github.com/orgs/Mu2e/people)
2. In order to authenticate with github, you will need to set up your ssh keys on the machine you plan to clone / push from, following the instructions here
3. Before we do any developing, we will create our own fork of the official Mu2e Offline repo using the GitHub web interface (instructions here). This fork will be your personal sandbox on GitHub; you can do anything you want to it and it will have no effect on anyone else!
4. Create a local clone of your Offline fork. This will be identical to cloning the official mu2e repo (if your fork is up to date), except that the default remote (called "origin") where you will push to by default will point to your fork instead of the official mu2e version.
git clone git@github.com:<your user name>/Offline
5. To proceed (for example with git remote add commands) you need to change to a git working directory, so cd Offline/
6. (Not required) To be more descriptive about the change you are working on, you can create a new branch to develop your feature on. Again since we will be pushing to your fork, this has no impact on the main Offline, and the only time the branch name will actually show up otherwise is if you are pointing other people to your changes or in the comment of the final merge commit back into the master branch (or if you are working on several things at once and want to keep them separate).
git checkout -b <new branch name>
7. Do your work and commit it locally
git commit -m "this is my normal commit onto my local branch"
8. If there are changes in the official master branch, you can grab them and merge them in
1. First you need to add the mu2e/Offline repo as a new "remote"
git remote add mu2e https://github.com/mu2e/Offline
2. now you can pull/fetch/merge from that repo (or from any other fork by the same procedure)
git fetch mu2e git merge mu2e/master
9. Push your work to your fork
1. If you are working on the master branch of your fork:
git push origin master
2. If you are working on a different branch, then the first time you push add the "-u" option to tell git that these branches "track" each other (see tracking branches)
git push -u origin <branch name>
10. When you are ready to have your changes propagated back into the main mu2e Offline repo, open a pull request.
1. In a web browser, open https://github.com/<your user name>/Offline
2. If you just pushed, there should automatically be a message about your new changes and a link to automatically open a pull request
3. Otherwise follow the github instructions
11. On the next software call, be prepared to discuss your changes
12. If issues are found with your changes, just commit and push again, the pull request will automatically be updated
13. After a code review, the admins will merge in your branch, and you are done!
14. Cleanup before continuing:
1. if you created a new branch and you are done with this feature completely, go ahead and delete the branch (remember no history is lost as this whole branch is now part of master, including all provenance)
git branch -d <my branch name> git push origin --delete <my branch name> (this deletes the branch from your github fork as well)
2. if you used the master branch on your fork, or you are planning to continue to develop this feature, just make sure to update it to include the merge commit created by the pull request
git fetch mu2e git merge mu2e/master

Collaborating on a feature

Sometimes you may want to collaborate on a feature branch with other developers. In this case since the main Offline repository no longer has all the development branches we need to do a couple extra steps

1. First make sure you actually need to work on the same branch. Are you actually working on the same feature? Can the problem be split into smaller features that can be developed asynchronously? Just because features are related doesn't mean they need to be developed on the same branch
2. Determine if a large number of people will be developing on the same branch for a significant amount of time. In this case it should become an official branch in the mu2e/Offline like MDC2018
3. Decide which users fork will be the primary repo for this feature branch, and which branch on that fork you are going to use. If a new branch is needed, the owner of that fork can do as usual:
git clone git@github.com:<their user name>/Offline git checkout -b <new branch name> git push -u origin <new branch name>
4. There are then a couple options for moving forward: either add all other developers as collaborators on the primary fork, or use pull requests to the primary fork
5. To add developers as collaborators:
1. The owner of the primary fork opens https://github.com/<their user name>/Offline
2. click settings on the right, then collaborators
3. In the collaborators box, type the github user name of each other developer and hit "Add collaborator"
4. The other collaborators can then either create a read/write access clone of the primary fork, or add it as a remote to an existing offline repo
git clone git@github.com:<primary user name>/Offline or git remote add primaryfork git@github.com:<primary user name>/Offline
5. The other collaborators can now push directly to the primary fork as if it was their own:
git push primaryfork <branch name>
6. To use pull requests:
1. The owner of the primary fork can just push to it as normal following the normal developer workflow
2. Other developers clone their own fork, but add the primary fork as a remote
git remote add primaryfork git@githbub.com:<primary user name>/Offline
3. Other developers can pull in and merge changes from collaborators by fetching/pulling/merging from this remote
git fetch primaryfork git merge primaryfork/<branch name>
4. Other developers push to their own fork
git push origin <branch namee>
5. Like in the normal developer workflow, they open a pull request. But then in the compare window before creating the request, change the "base repository" from mu2e/Offline to <primary user name>/Offline (see here)
6. the owner of the primary fork will need to accept and merge it in
7. everything else goes like the normal workflow

I have a branch on Redmine that's not on GitHub! How do I use it now?

The official Mu2e GitHub repo will usually have only the master branch (+ maybe some tagged release branches). Currently Redmine has >100 branches. If you have a Redmine branch that is not merged in to master, you will move this branch to your fork to work on it instead.

To start, you should read the development workflow and understand remotes and forks, and have a github account and a fork of the Mu2e Offline repo.

1. Create a local clone of your Offline fork. This will be identical to cloning the official mu2e repo (if your fork is up to date), except that the default remote (called "origin") where you will push to by default will point to your fork instead of the official mu2e version.
git clone git@github.com:<your user name>/Offline
2. Add Redmine as a remote to your local clone:
cd Offline git remote add redmine http://cdcvs.fnal.gov/projects/mu2eofflinesoftwaremu2eoffline/Offline.git
3. Fetch a list of the commits and branches from redmine, and check out whatever branch you need
git fetch redmine git checkout -b <branch name> redmine/<branch name>
4. Push this branch to your fork
git push origin <branch name>
5. You should now be able to see your branch on github.com/<your user name>/Offline. You can now continue working on that branch and pushing to your fork, and when it is ready you can submit a pull request to the main Mu2e repo using the developer workflow