Muse

Introduction

As Mu2e approaches data-taking there will be many more ntuples, calibration procedures, and analysis code developed. Including this user code in the main Offline repo will be unwieldy and hard to maintain. The main Offline repo should remain minimal, including only what is needed for simulation and reconstruction jobs, for simple maintenance and faster builds. Code for calibration, ntuple, and analysis code, maintained and used by individuals and small groups should be stored in a smaller repos controlled by the appropriate group. These smaller repos could be standalone contain modules and utilities that depend on code and functionality in other small repos, or in Offline. To make this work, we need a system to build the smaller repos against an existing Offline build, or together with the Offline, sharing include files, fcl, and cross-linking. The system should support common use cases and be flexible. The Muse system is a set of scripts developed inside Mu2e that provides this functionality. It is a UPS product containing a set of scripts and support for scons code build at the core.

In this page we will use Ana1 and Ana2 as examples of smaller, non-Offline code repos. WorkDir will represent the path to a user project area on /mu2e/app, or possible other disk where you can build code. AnaDir will represent an area where analysis files are kept : scripts, root scripts, ntuples, histograms, and notes. If the user's work is mostly concerned with code development and commits, then it may be conveneient to use WorkDir as AnaDir, or, if the works is mostly using a statis build, then they may be different. Or there may be several versions of AnaDir, or AnaDir's with different goals, all using the same WorkDir.

Quick Start

Always required:

 setup mu2e
 setup muse

Built-in help:

 muse -h
 muse <command> -h
 muse status

Muse can only be setup once in a process and can't be unsetup. To change setup choices, start in a new process. When building on mu2egpvm01-06, use "-j 4", when building on mu2ebuild01, use "-j 20", see Machines. See GitHubWorkflow for more details on using git. See mgit for more information on using partial checkout (building only parts of Offline).

Muse scripts are focused on the workDir which provides the code to be built, a place to build the code, and persistency of what the user is building and using. There can only be one workDir in a process. After muse is UPS setup ("setup muse"), the workdir is usually configured by cd'ing to the workDir, checking out repos

 cd workDir
 git clone ...

or linking in pre-build backing releases.

 cd workDir
 muse link ...

once the code is organized, then muse is setup

 cd workDir
 muse setup

or from any directory:

 muse setup workDir

Once "muse setup" has been run, then the list of repos which will be built, and the build options, are locked in for the process. If you add or delete repos, or change a link to a backing build, then you must run "muse setup" again in a new process. The status report:

 muse status

can be run before or after "muse setup". If run before, it assumes the default directory is the workDir. If run after, it can be run from anywhere.

Builds can be removed:

 muse build -c

or simply remove the build directory content:

 rm -rf workDir/build

nothing is built in the repo git area, and everything built is under the build directory.

Build Offline

Build the Offline repo locally:

 cd WorkDir
 git clone https://github.com/Mu2e/Offline
    or
 git clone git@github.com:<your GitHub username>/Offline
    optionally, add other repos:
 git clone git@github.com:<your GitHub username>/Ana1
 muse setup
   edit files, as needed..
 muse build -j 20
   run
 mu2e -c ..

Returning later

To re-setup, later, in another process

 cd WorkDir
 muse setup

or

 cd AnaDir
 muse setup WorkDir

Setup options

In one process/window:

 cd WorkDir
 muse setup
 muse build -j 20
 muse -c ...

In another process:

 cd WorkDir
 muse setup -q debug
 muse build -j 20
 gdb --args  mu2e -c ...

Backing build

Before "muse setup", link in a pre-built Offline:

 cd WorkDir
    see options for backing builds:
 muse link
    link in a release:
 muse link master/be905d45
    optionally, add smaller repos:
 git clone git@github.com:<your GitHub username>/Ana1
 muse setup
 muse build -j 20

Partial checkout with backing build

Before "muse setup", link in a pre-built Offline:

 cd WorkDir
    see options for backing builds:
 muse link
    link in a release:
 muse link master/be905d45
    optionally, add smaller repos:
 git clone git@github.com:<your GitHub username>/Ana1
    add Offline partial build
 mgit init
 cd Offline
 mgit add HelloWorld
    edits..
 mgit status
 cd ..
 muse setup
 muse build -j 20

The build command can be run from any directory after "muse setup". The developer might find it useful to have two windows open, both setup to the same workDir: one sitting in workDir where code editing is done, and the build command is issued, and a second window sitting an AnaDir where code is run and output is kept.

Tarball

After preparing and testing a build area,

 cd WorkDir
 muse setup (if not already done)
 muse tarball

Tarball location with be printed. This can be sent to the grid with --code=/path/tarball in mu2eprodsys.

Muse product

The Muse UPS product is created out of the Mu2e/Muse repo. When the product is setup,

 setup mu2e
 setup muse

typically using the current version, it will do the following.

1. add $MUSE_DIR/bin to the path
2. add MUSE_DIR/python to PYTHONPATH
3. add alias=source $MUSE_DIR/bin/muse
4. export MUSE_ENVSET_DIR=/cvmfs/mu2e.opensciencegrid.org/DataFiles/Muse

The product can be packaged from a tag by checking out the tag in the repo, then

Muse/bin/museInstall.sh <version> <path to output product area>

muse setup

The command

muse setup

sets link, fhicl, and other paths, and sets up scons and art. Typically, the user would first organize their repos in the wokrDir, or link backing release builds, before running "muse setup". This command is also typically run when returning to work on the code in workDir in a later process.

The command must be run at the bash command line so that the alias is available, causing muse to be sourced so it can alter your environment. All other commands are run in subshells and will not effect the environment.

The setup command has two forms:

 cd workDir
 muse setup

or from anywhere, if workDir is provided:

 muse setup workDir

Options can be provided:

 muse setup <workDir> -q <options>

there is a small set of allowed options, such as prof/debug, the geant visualization build options. See

 muse setup -h

for the list of options. Examples (can only run one in a process)

 muse setup
 muse setup -q debug
 muse setup workDir -q e21
 muse setup workDir -q e21:trigger:debug

The effects of running setup are the following

1. sets environmentals used by Muse, such as the workDir (MUSE_WORK_DIR)
2. uses UPS to find the machine flavor
3. parses the options
4. determines which set of UPS art products to setup, and does the setup
5. analyzes available information and determines a repo link order
6. adds the workDir area to the bin, link, include and other paths
7. for linked backing builds, links in their build to the workDir build area

The resulting environment can be examined with

muse status
muse -v status

The UPS version of art, and all the other associated products, including a few that are not always needed, but ar convenient, like valgrind, are setup as one "environmental set" or "envset". In implementation, an envset is a script to be sourced as part of "muse setup". This script choses some defaults (if the user doesn't force a choice), then does the UPS art setups. envsets are files, where N represents a digit, with name pNNN if published permanently, or uNNN if customized by the user.

The "muse setup" command uses the following priority list to find which envset to use, and therefore which version of art to setup.

1. the user has forced an explicit choice by a qualifier like "-q d000"
2. an Offline repo is a local checkout, a mgit partial checkout, or a link, and has a .muse, then use the recommendation in there, if any
3. any other local package with a .muse file with a recommendation
4. $MUSE_WORK_DIR/uNNN, where N represents a digit, exists, then take highest number there
5. use highest number from $MUSE_ENVSET_DIR

muse link

The "muse link" command is used to link a repo, that was built elsewhere, to the local workDir. The linked repo must have been built by Muse, in its own workDir.

The linked repo can be on a local disk, or on cvmfs. Running "muse link" with no argument gives a list of available cvmfs builds. There are two groups: first, a list of recent tagged and built releases, and second, a list of continuous integration (CI) builds. CI builds are made automatically every time code is merged in the main branches in the Offline repo.

The "muse link" commands, bit to see and execute links, are typically done in the workDir, while organizing code in the area, before "muse setup" is run.

muse link
  Recent published releases:
  v09_14_00
  v09_13_00
  Recent CI builds
  2021-03-29 11:05 master/be905d45
  2021-03-25 00:05 master/d3cc52b5
  2021-03-24 22:05 master/9fe21bab

The choice is then linked:

  muse link v09_14_00
     or
  muse link master/be905d45 

Some examples follow.

A user has a analysis repo they want to build against a recent Offline build. The user would clone the analysis repo in their workDir then link

 cd WorkDir
 muse link
 muse link master/be905d45
 git clone git@github.com:<your GitHub username>/Ana1
 muse setup
 muse build -j 20

A user wants to link a backing build, then do a partial checkout of Offline for a small local change, or a quick-fix develeoment.

 cd WorkDir
 muse link
 muse link master/be905d45
 git clone git@github.com:<your GitHub username>/Ana1
 mgit init
 cd Offline
 mgit add HelloWorld
    edits..
 mgit status
 cd ..
 muse setup
 muse build -j 20

Multiple users are sharing a build of a Offline and an ntuple. In the Offline builder's area, call it workDir0,

 cd WorkDir0
 git clone https://github.com/Mu2e/Offline
 git clone git@github.com:<your GitHub username>/MyNtuple
 muse setup
 muse build -j 20

then in the user's workDir

 cd workDir
 muse link workDir0/Offline
 muse link workDir0/MyNtuple
 git clone git@github.com:<your GitHub username>/Ana1
 muse setup
 muse build -j 20

The command makes links in the "link" directory, which is in workDir. When "muse setup" is run, it creates links from the local build area to the linked code build area.

cd workDir
muse link master/d3cc52b5
ls -l link
   Offline -> /cvmfs/mu2e-development.opensciencegrid.org/museCIBuild/master/d3cc52b5/Offline
muse setup
ls -l build/sl7-prof-e20-p000/link
   Offline -> /cvmfs/mu2e-development.opensciencegrid.org/museCIBuild/master/d3cc52b5/build/sl7-prof-e20-p000/Offline

Note the link command does not select prof/debug or other possible qualifier choice - those are set in the "muse setup" command. If the requested linked build area does not exist, you will get a warning during "setup".

If you link a repo name which has already been linked, the link will be updated, and a warning issued. After changing a link, the user must wipe out the current build

 muse build -c -j 20
    or
 rm -rf build

and re-run "muse setup" in a new process to finish the change.

Muse status

• user has an Offline build modified or frozen in workDir1, and the user wants to build several analyses or versions of an analysis off from this one Offline. The user would build the Offline in its own WorkDir. Then in s separate process, create an analysis workDir, link the Offline code and build to this workDir, then build the analysis repo, backed by the Offline build.
• one person maintains a workDir with Offline and an ntuple, this Offline and ntuple area are linked to the local workDir.