GitMinorWorkflow
Introduction
This page describes a recommended git workflow for use with the Mu2e Offline code for minor revisions. It is for small changes, such as changing fcl parameters, fixing bugs, adding minor features, or starting a new module. These modifications are committed directly to the head and should leave the head in a state that compiles and runs. You should work on your local work branch, but after the commit, there should be no branch visible in the main repo.
For more comlex revision use the git major workflow.
This page tells the same story 4 times:
- An Overview comprising a set of 10 bullet points
- A picture version that show the state of the repository after each sep
- A Cheat Sheet - just the git commands without commentary
- A Detailed Version
This page presumes that you are familiar with the Mu2e Introduction to git.
If you are already familiar with the background material, you can jump directly to the Cheat Sheet.
Overview
The recommended workflow has the following steps:
- Start from a clean local master branch that is up to date with the local tracking master branch.
- Synchronize the master branch in your local repository with that in the redmine repository.
- Create a local working branch starting from the local master branch. This new branch is temporary and will not appear in the redmine repository.
- Do your work on the local working branch and make all commits to that branch.
- Check that there are no uncommitted changes on your work branch.
- Synchronize your local master branch with the redmine repository.
- Use git rebase so that your working branch appears as if you had branched it off of the new head of the local master branch. This is the place at which you may need to resolve collisions with changes committed by others.
- Use git merge to move your working branch onto the head of the local master branch.
- Use git push to synchronize both the redmine repository and your local master tracking branch with your master branch.
- Delete your working branch.
The result is that your working branch never appears in the redmine repository; its contents are merged onto to the head of the master branch before that branch is pushed to the redmine repository.
The careful reader will notice that there is a race condition between that last git fetch/pull and the git push; this is addressed in the main body of the discussion.
The above description gives the example of a single working branch but you may many such branches; for example, you might have a separate project underway on each branch. When it is time to rebase and merge one of these branches you can leave the other branches unchanged and continue to work on them at a later time.
Picture Version
This section restates the information above but adds detail and uses figures to help tell the story. The numbered steps correspond to the same numbers in the previous section.
1) Start from a clean local master branch that is up to date with the local tracking master branch.
Check that your master branch is clean
git status
which should produce the output:
# On branch master nothing to commit, working directory clean
If it does not, consult the page Cleaning your master branch.
2) Synchronize the master branch in your local repository with that in the redmine repository.
There are two ways to perform Step 2. The one step method is to issue the command
git pull
There is also a two step method.
git fetch git merge --ff-only
There is a discussion of both methods at fetch vs pull.
The following error message may be generated by fetch or pull:
X11 forwarding request failed on channel 0
You can safely ignore it.
For this workflow example we imagine the starting point to be a redmine repository in which there is a master branch that ends in two commits named m1 and m2 (m1 and m2 are mnemonic names for SHA-1 hash codes). In all of these figures, commits are denoted by outline boxes with a white interior and a mnemonic name inside; the arrow pointing from m2 to m1 denotes that m1 is the parent of m2. A git branch is just a lightweight pointer to a commit. In Figure 1 there is one branch, named master, which points to the last commit, m2. In this figure branches are denoted by solid boxes with the branch name inside.
While the Mu2e redmine repository does contain other branches, none are involved in this workflow example and they are not shown.
After issuing the above command(s), the local repository has been synchronized with the redmine repository. This is illustrated in Figure 2. These figures use the convention that figures representing the redmine repository are shown in blue while those representing the local repository are shown in red.
The local repository has both a branch named master and a tracking branch named origin/master. Both are shown in Figure 1 and both point to the commit named m2. In most of the upcoming figures, the origin/master branch points to the same commit as does the master branch. When this is the case, only the master branch is shown. When this is not the case, both master and origin/master will be drawn on the figure.
3) Create a local working branch starting from the local master branch.
git checkout -b work
Figure 3 shows the state of the local master branch after Step 3; a new solid red box has been added to represent the newly created working branch; in this example the name of the branch is simply work.
The redmine repository master branch and the local master tracking branch are unchanged by Step 3.
4) Do your work on the local working branch and make all commits to that branch.
Now suppose that you make 4 commits, named w1 through w4. Figure 4 shows the state of the local repository after those commits: there are four new commits; the local master branch still points at the commit m2; the work branch now points at commit w4.
5) Check that there are no uncommitted changes on your work branch.
git status
If there are uncommitted changes, commit them now. If the output does not say that your checked out branch is work, consult an expert.
6) Synchronize your local master branch with the redmine repository.
git checkout master git pull
An alternative to the last step is to use git fetch and git merge; see fetch vs pull.
For purposes of this example, suppose that, after you synchronized with redmine repository in Step 2, someone pushed commits m3 and m4 to the master branch in redmine repository. The state of the redmine repository is illustrated in Figure 5:
Two new commits have been added and the master branch now points at the commit named m4. This figure is drawn in blue to remind you that it represents the stat of the redmine repository, not the local repository.
In step 6, you synchronize the branches master and origin/master in your local repository with the redmine repository. After this step, your local repository will look like Figure 6:
The commits m3 and m4 have been added and the master tag has been moved to point at m4; the origin/master tag has also been moved to point at m4.
In this picture it becomes clear why we use the name "branch" for the master and work branches.
7) This step is the key to the workflow and its action is to transform Figure 6 into Figure 7. This step in called rebasing.
A full discussion of the commands you need to issue at this time don't fit in this section; for the full discussion see the discussion of Step 7 in the last section on this page. The sequence of commands begins with:
git checkout work git rebase master
Figure 7 shows the local repository after rebasing:
Note that the commits in the work branch have new names: w1 became w1-prime, and so on. In words, Step 7 tells git:
- Look up the changes need to transform commit m2 into commit w1
- Apply these changes starting at m4
- Call the result a new commit named w1-prime.
- Repeat for the changes needed to transform w1 to w2, w2 to w3 and w3 to w4.
Very often this "just works" but there are times when it does not. An example of when it will "just work" is when the set of files modified by commits m3 and m4 has no overlap with the set of files modified by w1 through w4. In this case it is said that there are no conflicts.
An example of when rebasing will not work is when both commits m3 and w1 changed the same line of one file and that the two commits made different changes. In this case git does not know what to do and will ask for your help - this case will be described in more detail below. This situation is called and unresolveable conflict.
There are also resolvable conflicts. An example might be when commit m3 deletes one line in a file and commit w1 deletes a different line in the same file. In this case git will produce w1-prime in which both lines are removed.
The git jargon is that rebasing rewrites history to produce the w1-prime through w4-prime that you would have made yourself had you started to work at m4.
If there are problems during rebasing, you can revert to the state prior to the rebase by issuing the command:
git rebase --abort
After a successful rebase the original commits w1, w2, w3 and w4 are still present in your local repository but they are no longer part of any branch. It is possible to recover them but how to do that is out of the scope of this discussion.
8) Use git merge to move your working branch onto the head of the local master branch.
git checkout master git merge --ff-only work
After this step your local repository will look like Figure 8:
Note that the origin/master branch still points at m4 but that the master branch now points to w4-prime; this is the only figure in the workflow in which master and origin/master point at different commits. If, at this point, you give the git status command, it will tell you that master is 4 commits ahead of origin/master.
9) Use git push to synchronize both the redmine repository and your local master tracking branch with your master branch.
git push
After this step, the local repository looks like Figure 9:
In this figure, master and origin/master once again point at the same place. Figure 10 shows the state of the redmine repository after the push; it does not have the work branch but it is otherwise identical to the local repository.
10) Delete your working branch.
git branch -d work
Figure 11 shows the state of the local repository after step 10, deleting the work branch.
To summarize this story: there was an intermediate state (Figure 6) when there was a manifest branch in the commit history. But, by the right use of git rebase and git merge, this branch was removed before the push to the redmine repository. The net result is that observers of the redmine repository will see a linear commit history.
Cheat Sheet
This section states the workflow with minimal comments; the sections below will describe it in detail. The workflow contains some redundant git status and git checkout commands; they are there to remind you to make sure that you have not forgotten a commit or that you have not wandered off to an incorrect branch. The numbered steps follow the same numbering scheme as the sections above.
Make sure that the local master branch is clean and is up to date with the local master tracking branch.
git status git checkout master git status
Synchronize the local master branch with the redmine repository
git pull
or
git fetch git merge --ff-only origin/master
For a discussion about the differences between these two options see fetch vs pull.
The following error message may be generated by fetch or pull:
X11 forwarding request failed on channel 0
You can safely ignore it.
Create a working branch (the branch will be temporary and the name is not important):
git checkout -b work
Do your work. Edit files; add/delete/rename files. Commit all of your work
git add newfile1
git commit -m "comment" newfile1 oldfile1 oldfile2
... and so on until your work is done ...
Make sure you have no uncommitted changes
git status
Go back to your master branch and sync with the redmine repository
git checkout master git pull
or
git checkout master git fetch git merge --ff-only origin/master
For a discussion about the differences between these two options see fetch vs pull.
Rebase. This step is not necessary if git pull did not download any commits.
git checkout work
git rebase master
... Git issues an message saying that there is a merge conflict
... You need to hand edit the indicated file to resolve the conflict.
git add hand-edited-file
git rebase --continue
... Git issues an message saying that there is a merge conflict
... You need to hand edit the indicated file to resolve the conflict.
If there are no merge conflicts the git rebase command will complete the first time it is issued.
Merge your rebased working branch into the master branch
git checkout master git merge --ff-only work
Push your commits to the redmine repository
git push
Delete the working branch
git branch -d work
There is a race condition between the last git pull and the git push; it will be discussed below.
Detailed Version
Step 1) Start from a clean local master branch that is up to date with the local tracking branch.
Check the status of your working environment:
git status
The output must look like the following:
# On branch master nothing to commit, working directory clean
If the output is anything else, you have work to do before you can proceed. The steps you need to do are described on the wiki page Cleaning your master branch.
Step 2) Synchronize Local Master with the Redmine Repository
This step is not required but it is strongly recommended. If you do it, it will minimize the work that you need to do when you get to the rebase step. To do this, issue the git command:
git pull
or
git fetch git merge --ff-only origin/master
For a discussion about the differences between these two options see fetch vs pull.
Step 3) Create a working branch
You will ultimately merge your working branch into the master branch and the existence of the working branch will be lost to history. Therefore the name of the branch is not important and we will call it work. If the intention is that the branch will become part of the git history, then you should give it a better name. To create and checkout the working branch issue the following git command:
git checkout -b work
In several places the Mu2e git instructions caution you against checking out a branch when you have uncommitted changes in your working tree - this may cause the loss of those changes. There is one prominent exception to this recommendation. If you have uncommitted changes on your master branch it is safe to create a new branch; in this case the uncommitted changes will remain in your working tree and, if you commit them, they will be committed to the new branch.
Now use the git branch command to look at the local branches:
git branch
which will make the output:
master * work
You can see that there are now two local branches; the position of the asterisk tells you that the working tree is an image of the branch named work.
Step 4) Do your work and commit it
Do the edits that you plan to make. Create new files as needed. Then follow the instructions in Wiki for adding files, deleting files, moving files and committing all of these changes.
For this example I removed the CVS meta-data comments from the file SConscript. To be specific I removed the following 4 lines:
# $Id: SConstruct,v 1.54 2014/08/02 05:23:26 gandr Exp $ # $Author: gandr $ # $Date: 2014/08/02 05:23:26 $ #
Now, check the status:
git status
which produces the output
# On branch work # Changes not staged for commit: # (use "git add <file>..." to update what will be committed) # (use "git checkout -- <file>..." to discard changes in working directory) # # modified: SConstruct # no changes added to commit (use "git add" and/or "git commit -a")
This tells you that you have modified the file SConstruct but have not yet committed it.
You can see all of the differences between the working tree and its parent branch by using the command:
git diff
If you have modified many files, you can check the changes in just the file SConstruct by giving the command:
git diff SConstruct
The next step is to commit the change:
git commit -m "Remove CVS comments which are no longer needed." SConstruct
which produced the output
[work bac5d3c] Remove CVS comments which are no longer needed. 1 file changed, 1 insertion(+), 5 deletions(-)
Step 5) Check that there are no uncommitted changes
Double check that there are no uncommitted changes:
git status # On branch work nothing to commit, working directory clean
This says that the working tree is clean
Another way to check for a clean working tree is to use the command
git diff
which will produce no output because the working tree is clean.
You can also compare the working tree to the local master branch
git diff master
Or you can compare it to the local clone of the redmine master branch
git diff origin/master
You do not need to type the "remotes" part of the branch name but you can if you want.
Since the local master and the local clone of the remote master are the same, both of the above diff commands produce:
diff --git a/SConstruct b/SConstruct index 2f520ce..9ff2699 100644 --- a/SConstruct +++ b/SConstruct @@ -2,10 +2,6 @@ # # Build a Mu2e base release or test release. # -# $Id: SConstruct,v 1.54 2014/08/02 05:23:26 gandr Exp $ -# $Author: gandr $ -# $Date: 2014/08/02 05:23:26 $ -# # Original author Rob Kutschke. # import os, re, string
In this example I only changed one file. You may change many files, add many files, delete many files or rename many files. Just be sure to commit all changes. You may make commit each file individually, commit everything with one big commit or do something in between. Normally it makes sense to commit groups of files that together make one logical change.
The most important thing is to use git status to check that your working tree is clean before proceeding to the next step.
Step 6) Sync the local master branch with redmine
First, checkout the local master branch. This changes your working tree to be an image of the local master branch and discards any uncommitted changes to tracked files.
git checkout master
You can double check that you are on the right branch using git branch, which produces the ouptut
* master work
The main part of this step is to use git pull to:
- Contact the redmine repository and update the local tracking branch origins/master.
- Merge the changes from origins/master into the local master
Because we have made not changes to the local master branch, the merge is guaranteed to work without conflicts. Issue the command:
git pull
If there have not been any changes committed to the redmine repository since you did your previous git pull, this will produce the output:
Already up-to-date.
If there have been changes to the redmine repository, the git pull command will produce output like:
remote: Counting objects: 568, done. remote: Compressing objects: 100% (142/142), done. remote: Total 405 (delta 277), reused 376 (delta 259) Receiving objects: 100% (405/405), 89.99 KiB | 68.00 KiB/s, done. Resolving deltas: 100% (277/277), completed with 94 local objects. From ssh://cdcvs.fnal.gov/cvs/projects/mu2eofflinesoftwaremu2eoffline/Offline
236c81fd3..7aa1da4ff master -> origin/master * [new branch] art_v2_09_03 -> origin/art_v2_09_03
Updating 236c81fd3..7aa1da4ff Fast-forward
CalPatRec/fcl/prolog.fcl | 14 ++- CalPatRec/inc/CalHelixFinderAlg.hh | 5 +- CalPatRec/inc/CalHelixFinderData.hh | 5 ++ CalPatRec/inc/CalHelixFinder_types.hh | 8 +-
In either case the working tree should be the checked out master branch. Check it with git status, which should show:
# On branch master nothing to commit, working directory clean
Step 7) Rebase the working branch
This step is the meat of the entire workflow. It is the place at which conflicts might arise; if they do arise they need to be resolved before moving on.
If the git pull in Step 6) told you that your local repository was already up to date, then this step will do nothing. You can choose to execute it, just to be safe; or you can skip it.
Switch back to your working branch and issue the rebase command.
git checkout work git rebase master
The action of the rebase command is described in the figures earlier on this page. If this command completes successfully, there were no unresolvable conflicts and this step is done. You can proceed to Step 8).
If there is an unresolvable conflict, git rebase will stop and issue a diagnostic telling the name of a file that contains a conflict. You should edit the the file to fix the conflict. When that is done, give the git command
git add relative-path-to-file-that-you-fixed git rebase --continue
If there is another unresolvable conflict fix that by hand and repeat the above commands. And so on. Eventually the rebase will complete and you can move on to Step 8.
Step 8) Merge the working branch into the master branch
git checkout master git merge --ff-only work
The ff in "--ff-only" tells git to do a "fast forward" merge. A fast forward merge will only work when there have been no changes to the local master branch since the last rebase of the working branch ( or since the creation of the branch if you have not rebased ). Because the previous step was to rebase, the -ff-only is formally redundant but it is a good idea to include the --ff-only option as a safety check that the rebase actually succeeded.
The output of this step should look like:
Updating aa98377..42447ae Fast-forward SConstruct | 4 ---- 1 file changed, 4 deletions(-)
Step 9) Push the results to redmine repository and clean up
Issue the command:
git push
This will push the changes from the local master branch to the redmine repository and also update the local tracking branch. The push
command will produce output like:
git push Counting objects: 8, done. Delta compression using up to 4 threads. Compressing objects: 100% (6/6), done. Writing objects: 100% (6/6), 583 bytes, done. Total 6 (delta 4), reused 0 (delta 0) To ssh://p-mu2eofflinesoftwaremu2eoffline@cdcvs.fnal.gov/cvs/projects/mu2eofflinesoftwaremu2eoffline/Offline.git aa98377..42447ae HEAD -> master
In rare cases someone may push to the redmine repository after your last pull but before you push.
Fixme: describe procedure to deal with this.
Step 10) Delete the working branch
git branch -d work
If you made an error in the above and there remain un-merged commits on this branch, then the delete will fail. You will need to figure out what the error is and how to recover from it.
If you want to force the delete of a branch that fails to delete with the -d option, try the -D option:
git branch -D work
The -D option tells git to delete the branch even if the safety checks fail.
In this workflow, the delete is done after the push succeeds. This ensures that the branch is still accessible in the event that there we need to recover from a failed push.
Additional Notes
This section has notes that need to find a home in the main body of the text.
Scenario 1
- I have a local uncommitted change in file6.txt
- I do a git fetch and discover that file6.txt has already been changed in the redmine repository.
- explore resolving the conflict using stash
git status
# On branch master
# Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
# (use "git pull" to update your local branch)
#
# Changes not staged for commit:
# (use "git add <file>..." to update what will be committed)
# (use "git checkout -- <file>..." to discard changes in working directory)
#
# modified: file6.txt
#
no changes added to commit (use "git add" and/or "git commit -a")
git diff origin/master
diff --git a/file6.txt b/file6.txt
index 978a23e..f318806 100644
--- a/file6.txt
+++ b/file6.txt
@@ -1,3 +1,3 @@
This is file 6. Added in the second repo.
Pushed before committing file 5 in the first repo.
-Added this line in the second repo and pushed.
+Added this line in first repo. Then git fetch.
git merge --ff-only origin/master
Updating 754536d..3c940d3
error: Your local changes to the following files would be overwritten by merge:
file6.txt
Please, commit your changes or stash them before you can merge.
Aborting
git rebase origin/master
Cannot rebase: You have unstaged changes.
Please commit or stash them.
git merge origin/master
Updating 754536d..3c940d3
error: Your local changes to the following files would be overwritten by merge:
file6.txt
Please, commit your changes or stash them before you can merge.
Aborting
git stash
Saved working directory and index state WIP on master: 754536d hand corrections to merge in file 7
HEAD is now at 754536d hand corrections to merge in file 7
git status
# On branch master
# Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
# (use "git pull" to update your local branch)
#
nothing to commit, working directory clean
git merge --ff-only origin/master
Updating 754536d..3c940d3
Fast-forward
file6.txt | 1 +
1 file changed, 1 insertion(+)
git status
# On branch master
nothing to commit, working directory clean
git stash pop
Auto-merging file6.txt
CONFLICT (content): Merge conflict in file6.txt
git status
# On branch master
# Unmerged paths:
# (use "git reset HEAD <file>..." to unstage)
# (use "git add <file>..." to mark resolution)
#
# both modified: file6.txt
#
no changes added to commit (use "git add" and/or "git commit -a")
git add file6.txt
git status
# On branch master
# Changes to be committed:
# (use "git reset HEAD <file>..." to unstage)
#
# modified: file6.txt
#
git commit -m "Hand merge file6" file6.txt
git status
# On branch master
# Your branch is ahead of 'origin/master' by 1 commit.
# (use "git push" to publish your local commits)
#
nothing to commit, working directory clean
gitk shows a linear history
git stash list
stash@{0}: WIP on master: 754536d hand corrections to merge in file 7
git stash drop
Dropped refs/stash@{0} (4b3c222fd438f2f0c886ddebbce760143d75fbd2)
git stash list
( no output )
Scenario 2
- I have a local committed change in file7.txt
- The redmine repo has a conflicting change to file7.txt
- I do a git pull
This will trigger a merge that will put conflict markers in file7.txt. To recover
edit file7.txt git add file7.txt git commit -m "hand corrections to merge in file 7" -a git push Explore resolving the conflict using rebase
This makes a loop in the redmine repository.