Git

From Mu2eWiki
Revision as of 19:38, 13 May 2021 by Rlc (talk | contribs)
Jump to navigation Jump to search

Introduction

The primary source code management system for the Mu2e Offline software is a git repository. Git is a popular open-source software version control system. Currently there are two primary repositories being maintained in parallel, one hosted on the Fermilab Redmine site, and one hosted on the commercial git hosting site github. Redmine is a software project management system which contains, for each project, a log, a wiki (which may contain useful info), and possibly gantt charts and bug tracking along with the git respositories. GitHub, in addition to hosting git repositories, allows the creation and documentation of issues, code comments, and pull requests.

Based on the git repository's url, you can check out code, and, if you are a code developer, check in modifications. Git also has capabilities for tagging code, tracking history and supports splitting off and merging development branches, among many other features.

Depending on your experience or needs, please also see the basic code recipe or the official GitHub workflow. You can continue with the quick start on this page or the expanded git introduction page. See the references for more details on Redmine and git. There is a page for troubleshooting redmine git ssh connections.

Some older parts of Mu2e software is managed using other source code management systems. Some code, both critical and specialized, is in other, smaller repositories. See Repositories below for more details on repositories.

Quick Start (DEPRECATED)

This section gives a quick guide to using git with the Redmine workflow. For the GitHub version, see GitHubWorkflow.

Please also see the basic code recipe for an introduction to setting up the environment, checking out code and building.

If you are going to commit anything, you must understand and follow the mu2e commit instructions

If this is the first time you are using git in the mu2e environment, please follow the few steps listed in order to set up your preferences in ~.gitconfig.

A more detailed introduction to all mu2e repositories is here and our introduction to git is here.

  • checkout the main repository
    • readonly: 
      git clone http://cdcvs.fnal.gov/projects/mu2eofflinesoftwaremu2eoffline/Offline.git
    • with kerberos authentication for committing code 
      git clone ssh://p-mu2eofflinesoftwaremu2eoffline@cdcvs.fnal.gov/cvs/projects/mu2eofflinesoftwaremu2eoffline/Offline.git
  • checkout the head to a local working branch, here named "work" (recommended)
git checkout -b work
  • see the history of a file
git log fileSpec
  • list tags
git tag -l
  • checkout a tag or branch or commit hash into the working area. Use this to switch between branches, it will NOT overwrite your work.
git checkout tagOrBranchNameorHash
  • update the local repository to the origin repository (Redmine)
git fetch
  • update the local repository to changes staged in the working area (does not affect the origin repository)
git commit -m"'Your descriptive message'"
  • update the origin repository (Redmine) to changes in your local repository
git push
  • update the local repository to the origin repository (REDMINE)
git fetch
  • update the working area to the local repository
git merge
  • update the local repository and update the working area both together
git pull
  • merge a branch (including master) into your branch. NB: do NOT merge your branch into master unless authorized
git checkout otherbranch 
git pull
git checkout mybranch
git pull
git checkout mybranch
git merge otherbranch
  • see what the status is
git status
  • command line help
git status help
  • git browser (to see a graphical history of commits)
gitk --all


Please see the references below or many web resources for details on git concepts, commands and features. If you are going to commit anything, you must understand and follow the commit workflows.

Workflows for committing code

As the group of mu2e code developers grows at the same time that the need for a stable code base increases, individual code contributions may start to clash with other contributions, so we are using a "request to pull" procedure. In this procedure, the developer does code development on a git branch and commits that branch to the repository frequently. This branch does not affect other users, so it can be in any state the developer finds useful - it might not even compile. Once the developer believes the work is complete, they make a request to the Offline group heads to merge or "pull" the branch into the master branch. If the change is very simple or low-risk, the heads may ask the developer to go ahead and do that merge. If the change is large or complex, they may request additional documentation or testing before the pull. The branch should be prepared so it includes the head of the master branch, so it can be merged with "fast forward".

The procedure of developing on a branch, then requesting to merge is explained in detail on the git workflow page.

If a change is very small, or planned in advance, or urgent, the Offline group heads may allow the commit to follow the minor workflow which ends in a commit directly to the head of master.

Once the switch to GitHub is made, we will begin following the GitHub workflow.

GitHub Pull Request Procedures and FNALbuild

When you open a Pull Request (PR) on the Mu2e/Offline GitHub repository, you will receive a greeting message from @FNALbuild, which is Fermilab's build bot account. @FNALbuild acts as a glue between the GitHub collaborative environment and Jenkins, and is used by the repository maintainers to trigger tests that may be required to pass before your changes are merged. The tests (or continuous integration, CI, actions) currently are a supplement to the existing review process.

N.B. buildmaster.fnal.gov links (Jenkins server) can only be accessed via Fermilab onsite VPN.

The supported CI actions are listed below. The first of these is triggered automatically by the creation of a PR. In addition, any CI action can be triggered at any time by posting a command in a comment on a Pull Request. fnalbuild-users GitHub Team indicates which GitHub users and Teams in the Mu2e organisation are able to trigger CI actions; some branches have additional users authorized to trigger actions; finally, the requestor of a PR may trigger actions on that PR

@FNALbuild run build test

In regular use. Merges the PR branch into the current HEAD of the base branch and builds the code, then runs a series of small jobs (10 events) to check for runtime errors.

@FNALbuild run code checks

Currently not in use. Checks the files that were changed by a PR for trailing whitespace and hard tabs - fails if modifications are needed. FNALbuild will provide a patch to fix these problems, if any.

This is not an integration test - it only tests the PR branch as-is.


@FNALbuild run validation

Not in regular use. Pulls the built code from a build test and runs a validation job (ceSimReco) over 5000 events, and produces validation plots. Builds, or pulls a cached build of master (not containing any PR changes) and runs the same validation job, producing plots. Runs valCompare between the two sets of plots, and produces a comparison which is published on the PR in a comment.

The Jenkins jobs that run the tests are located here


Where the code is stored

The scripts that are run to test a PR are kept on GitHub in Mu2e/codetools.

The FNALbuild handler scripts are kept here.

The Mu2e version of this bot was based on and re-written using the original process_pr.py script from FNALbuild/cms-bot:

  • process_pr.py: Where everything happens. given a PR, process all comments and figure out which tests to trigger based on those comments.
  • comment_gh_pr.py: this is called from the Jenkins jobs to post a comment on the PR when the result of a test has been determined, or when the status of a test has changed e.g. it is running.
  • test_suites.py: some configuration variables to set up the regular expressions that are used to search for commands that trigger tests.
  • watchers.yaml: Users may add themselves to this to 'watch' a specific folder in Offline for changes, such that they are notified when someone wishes to change it. Regex is supported.
  • auth_teams.yaml: Configuration file of which GitHub teams are able to launch CI actions on a base branch.



References

Retrieved from "http://mu2ewiki.fnal.gov/w/index.php?title=Git&oldid=18309"