Redmine

From Mu2eWiki
Jump to navigation Jump to search

Introduction

Redmine is a web-based code management system containing git repositories, issues trackers, and wikis. The lab computing division keeps most of their code and documentation in the lab redmine system. Important tools for code infrastructure, data handling and job submission are all documented here.

Mu2e used to keep our major code repositories in the lab's redmine server. Now we keep them in [GitHubWorkflow|GitHub]. As of 2021, we still have several smaller repositories on redmine, and a few are likely to stay there for the forseeable future.

This page will help with using redmine, and converting redmine to github.



Repositories

One important piece of the mu2e offline code is the BTrk UPS product, which contains our Kalman track fit code, and is stored in the BTrk github repo.

All of the rest of the mu2e Offline code and tools are in Redmine repositories. You can browse all Redmine projects and mu2e Redmine. The mu2e Redmine has subprojects (and sub-subprojects) and each has a repository associated with them. For example in the Standalone subproject, there is a sub-subproject Mu2eCCFC. In addition, the main mu2e Offline repository has sub-repositories which can be seen on the right sidebar menu on the repository main page. An example is the dhtools sub-repo.

Here is a (possibly incomplete) list of the repositories.

content user name alias
p-mu2e_artdaq mu2e_artdaq
raw data format p-mu2e_artdaq mu2e_artdaq-core
F-C limit setting p-mu2eccfc mu2eccfc-mu2eccfc mu2eccfc.git
CI validation p-mu2e_ci mu2e_ci
p-mu2e-daq-firmware mu2e-daq-firmware
p-mu2e-daq-firmware mu2e-daq-firmware-dtc
p-mu2e-dcs mu2e-dcs
p-mu2e_detector_construction_db mu2e_detector_construction_db
p-mu2eextmontestdaq mu2eextmontestdaq
grid tools p-mu2egrid mu2egrid
Offline p-mu2eofflinesoftwaremu2eoffline mu2eofflinesoftwaremu2eoffline Offline.git
Jenkins p-mu2eofflinesoftwaremu2eoffline mu2eofflinesoftwaremu2eoffline-codetools codetools.git
data handling p-mu2eofflinesoftwaremu2eoffline mu2eofflinesoftwaremu2eoffline-dhtools dhtools.git
p-mu2eofflinesoftwaremu2eoffline mu2eofflinesoftwaremu2eoffline-geomdisplay geomdisplay.git
satellite releases p-mu2eofflinesoftwaremu2eoffline mu2eofflinesoftwaremu2eoffline-satellite
gridexport product p-mu2eofflinesoftwaremu2eoffline mu2eofflinesoftwaremu2eoffline-gridexport gridexport.git
p-mu2eofflinesoftwaremu2eoffline mu2eofflinesoftwaremu2eoffline-test
obsolete p-mu2eofflinesoftwaremu2eoffline mu2eofflinesoftwaremu2eoffline-validation validation.git
Stntuple framework p-mu2eofflinesoftwarestntuple mu2eofflinesoftwarestntuple Stntuple.git
p-mu2eofflinesoftwareuser mu2eofflinesoftwareuser-betasource betasource.git
p-mu2eofflinesoftwareuser mu2eofflinesoftwareuser-example01 example01.git
p-mu2eofflinesoftwareuser mu2eofflinesoftwareuser-streamline
p-mu2epbarmodels mu2epbarmodels
p-mu2epbaridanalysis mu2epbaridanalysis
p-mu2e-raw-data mu2e-raw-data
p-mu2e-simulations mu2e-simulations
p-mu2estat mu2estat
p-mu2e-tools mu2e-tools-bintools mu2ebintools
FileTools p-mu2e-tools mu2e-tools-filetools mu2efiletools
FileTools p-mu2e-tools mu2e-tools-mu2efilename mu2efilename
generate_fcl p-mu2e-tools mu2e-tools-tools mu2etools

Creating a Redmine subproject is done by the "add subproject" button on the Redmine project web page. This is probably more appropriate for a project not closely associated with the Offline. Creating a sub-repository associated to the main offline repo is explained in the mu2e instructions and SCD instructions. This is probably more appropriate for a project closely associated with the Offline. Write privileges to each repository can be controlled by the .k5login contents of each group username associated with the repo - note that this file should be edited by the Redmine project web page (settings->members), not directly. The login to the group account is controlled by the .admin file.

A git repository is accessed through a url and the url can be read-only or read/write, which allows commits back to the repo. Taking values from the table of Redmine repos above, a repo without an alias will have read-only and read/write urls: 

http://cdcvs.fnal.gov/projects/name
ssh://user@cdcvs.fnal.gov/cvs/projects/name

and if it has an alias: 

http://cdcvs.fnal.gov/projects/name/alias
ssh://user@cdcvs.fnal.gov/cvs/projects/name/alias

A good view of all repositories can be found by logging is as one of the user names where you have write permissions: 

ssh user@cdcvs.fnal.gov

The projects can be seen 

cd /cvs/projects
ls -l | grep mu2e

there is one directory per repository and the username can be seen as the owner of the repository directory. If the repository has an alias, it will appear as a subdirectory or a circular link under 

/cvs/projects/name

The .k5login file for a repository is 

/home/name/.k5login

CVS repositories can be seen here 

ssh mu2ecvs@cdcvs.fnal.gov
ls -l /cvs/mu2e

Redmine git hooks

To add or modify a hook, log into the Mu2e Offline redmine project 

 ssh p-mu2eofflinesoftwaremu2eoffline@ccdcvsvm.fnal.gov

The repo is hosted at: 

/cvs/projects/mu2eofflinesoftwaremu2eoffline/Offline.git

Look in the hooks/ subdirectory. I think what you want is a post-receive hook. There is already a post receive hook file - this is the code that sends email to registered subscribers on every push.


Transitioning between Redmine and Github

The following sections were important during the transition into our currently recommended git workflows. Hopefully they are not relevant for current work. They have been retained for now and will be deleted at a later date.

Adapting your Existing Clones to the New Workflow

Many of you have existing clones of your own github fork that were created using an older recommended github workflow. This section discusses how to bring your clone into compliance with the new recommended workflow. We cannot give an exact script to follow since everyone's local clone is likely to be different; so the following is general guidance:

  1. If you have unpushed commits on your local master, see the instructions in the hints section below.
  2. If you have uncommitted changes in your checked out area, commit them to a local development branch that you intend to keep.
    1. If needed: git stash; create a new development branch starting from the current head; git stash pop.
  3. If the old workflow you will have have a local branch named master and a branch named remotes/origin/master. These come from the master branch of your github fork; if you still have these branches, delete them.
    1. You cannot delete the branch that is currently checked out; therefore, to delete your local master you need first to checkout a different branch.
  4. You must have a branch named remotes/mu2e/master that comes from the Offline repository in the Mu2e github instance; actually the middle "mu2e" in the branch name is just an arbitrary identifier than you can choose but it would be weird to chose anything except mu2e or Mu2e.
  5. If you have existing development branches that started from something other than mu2e/master, just keep them and proceed as described in the recommended workflow; if you are lucky they will merge cleanly. If not, follow the merge/rebase procedure.
  6. Start all new development branches with:
 git fetch mu2e master
 git checkout -b <new development branch name>

Here are a few hints and reminders of things to watch for:

  1. Use git remote -v and git branch -avv to learn what remotes you have and what branches are tracking what other branches.
  2. If your local master branch has commits on it that you need to keep, then start a new development branch from the head of master. Just develop on it normally and push it to your fork when you are ready and make a pull request when you are ready for that. If you are lucky it will merge cleanly; if not follow the merge/rebase procedure. Once you have created the new development branch you can delete the local master branch.


Migrating an existing redmine clone directory to GitHub

You can repoint an existing Offline clone from redmine to GitHub as a way of making an easy transition. To follow these instructions, you must already have a GitHub account that's registered in the Mu2e Organization. The following assumes your redmine clone is in /mu2e/app/Me/MyRedmineClone/Offline, that your GitHub username is MyGitHubName, and that you are working on a branch called MyDevelopmentBranch.

    1. Login to GitHub
    2. Fork Mu2e/Offline in GitHub using the 'Fork' button on the top right of the Mu2e Offline repo page
    3. Add GitHub as a remote to your clone
      > cd /mu2e/app/Me/MyRedmineClone/Offline
      > git remote rename origin Redmine
      > git remote add mu2e https://github.com/Mu2e/Offline
      > git remote add origin git@github.com:MyGitHubName/Offline
      These commands are very fast as no data is actually transferred. You can see your remotes with:
      > git remote -v
      Your clone is now connected to the main Mu2e github repository and your own fork in addition to the Redmine repository. Additionally, we have changed your fork to be the default remote repository (origin)
    4. Push your working branch to your GitHub fork:
      > git push -u origin MyDevelopmentBranch
    After this, you can continue developing in this directory according to the general GitHub workflow described below.


I have a branch on Redmine that's not on GitHub! How do I use it now?

The official Mu2e GitHub repo will usually have only the master branch (+ maybe some tagged release branches). Currently Redmine has >100 branches. If you have a Redmine branch that is not merged in to master, you will move this branch to your fork to work on it instead.

To start, you should read the development workflow and understand remotes and forks, and have a github account and a fork of the Mu2e Offline repo.

  1. Create a local clone of your Offline fork. This will be identical to cloning the official Mu2e repo (if your fork is up to date), except that the default remote (called "origin") where you will push to by default will point to your fork instead of the official Mu2e version.
    2. git clone https://github.com/Mu2e/Offline:<your user name>/Offline
  2. Add Redmine as a remote to your local clone:
    3. cd Offline git remote add redmine http://cdcvs.fnal.gov/projects/mu2eofflinesoftwaremu2eoffline/Offline.git
  3. Fetch a list of the commits and branches from redmine, and check out whatever branch you need
    4. git fetch redmine git checkout -b <branch name> redmine/<branch name>
  4. Push this branch to your fork
    5. git push origin <branch name>
  5. You should now be able to see your branch on github.com/<your user name>/Offline. You can now continue working on that branch and pushing to your fork, and when it is ready you can submit a pull request to the main Mu2e repo using the developer workflow



References

Retrieved from "http://mu2ewiki.fnal.gov/w/index.php?title=Redmine&oldid=23492"