Difference between revisions of "KinKal"

Revision as of 15:11, 4 August 2023

Local KinKal development with Muse

Introduction

This section describes how to do simultaneous development of KinKal and Offline. Because KinKal uses a different build system than does Offline, you need to take a few additional steps when you start. Once you have taken these steps "muse build" will first build KinKal, but only if necessary, and install it into a local UPS area; it will then build Offline using KinKal from the the local UPS area.

This writeup assumes that you have a basic knowledge of UPS and of Muse; in particular you should understand what a Muse envset is and how to use it. Normally you will choose the envset recommended by Offline and this example is written assuming that's what you want to do. If you are new to this work and you need to use a different envset, your mentor will tell you which one to use. Section #Using An Alternate envset describes how the workflow must be changed in order to use a different envset.

Initial build

In a clean shell, create a Muse workDir and cd there
```
mkdir 'myworkdir'
cd 'myworkdir'
```

clone KinKal into the Muse workdir as your would normally. Add the Mu2e KinKal repo as a remote and checkout the main branch of the Mu2e KinKal remote.

git clone git@github.com:<your GitHub username>/KinKal
cd KinKal
git remote add -f mu2e https://github.com/KFTrack/KinKal
git checkout -b work_branch mu2e/main
cd ..

Clone your fork of Offline into the muse workdir as you would normally. Add the Mu2e Offline repo as a remote and checkout the main branch of the Mu2e Offline remote. If you need other muse-aware repos (Production, TrkAna, ...) for your workflow, you can add them too
```
git clone git@github.com:<your GitHub username>/Offline
cd Offline
git remote add -f mu2e https://github.com/Mu2e/Offline
git checkout -b work_branch mu2e/master
cd ..
# Optional: repeat the above pattern for any of Production, TrkAna ...
```
Setup the Mu2e development environment
```
setup mu2e
setup codetools
```
Choose a UPS version number for your local build of KinKal. The number will have no meaning outside of your muse working directory so you can use any number that is not already in use. Choosing version 99.9.0 is likely to be safe for a long time to come. The version number must have 3 dot separated fields.
Build KinKal; create a local ups directory and install the build of KinKal into that UPS area.
```
kk2ups -v "99.9.0" -b -i -m "" -j 8
```
On mu2ebuild01 you may increase 8 to 24. This command will build both the profile and debug versions of KinKal. The option -m with an empty string tells Kkk2ups to choose the version of root that matches the default for building Offline. See section #Using An Alternate envset for when you should provide a non-empty string.

If everything worked correctly, the output of this command will end with

Begin install.
Begin install of prof version.
Begin install of debug version.
KinKal_to_UPS/build completed with status 0

Look at the files created by kk2ups:
```
ls artexternals/KinKal/
```
This will produce the output:
```
v99_09_00  v99_09_00.version
```
Also do
```
ls artexternals/KinKal/v99_09_00
```
This will produce output similar to:
```
include  slf7.x86_64.debug.e20.p3915  slf7.x86_64.e20.p3915.prof  source  ups
```
As Mu2e Offline evolves to newer versions of g++ and python, the qualifiers e20 and p3915 will change.
Look at Offline/.muse to find the name of the default envset for building Offline. In these instructions we will call that pxyz. Make a local muse subdirectory and make a copy of pxyz inside of it.
```
mkdir muse
cd muse
cp /cvmfs/mu2e.opensciencegrid.org/DataFiles/Muse/pxyz unnn
```
where unnn is the name of your copy envset, for example u046. The name unnn has no meaning outside of your muse working directory.
Edit the envset unnn to make two changes
1. Find the line that sets up KinKal and replace the version number with v99_09_00
```
   setup KinKal v99_09_00  -q +${MUSE_BUILD}:+${MUSE_COMPILER_E}:+${MUSE_PYTHON}'
   
```
2. Add the following line after the last setup command:
```
 export MUSE_PREBUILD="kk2ups -v \"99.9.0\" -b -i -j 24" 
```
  This line is not the same as the command given in step 6; it omits the -m argument. When muse executes this command a version of root will already be setup in your environmen and kk2ups will use that version of root.
cd back to the muse working directory and setup muse using the envset you just created
```
cd ..
muse setup -q unnn
```
Build Offline and any other repos that you cloned in step 3; if anything has changed in KinKal since the last build, this command will also rebuild KinKal and reinstall it in the local artexternals.
```
muse build -j 8 
```
On mu2ebuild01 you may increase 8 to 24.
Now you can edit files in any of KinKal, Offline and any other locally cloned repos. When you want to rebuild you can do so with a single command:
```
muse build -j 8 
```
On mu2ebuild01 you may increase 8 to 24.

About kk2ups

The command kk2ups is short for "KinKal to UPS" and is found in codetools. The command you issued in step 6 will
1. (-b) build the existing checkout in the local clone of KinKal
2. (-i) if necessary, create a local directory named artexternals, configure it as a UPS repositor
3. (-i) copy the build of KinKal into artexternals/KinKal/v99_09_00 and configure it as a UPS product.
The full syntax of the kk2ups option -m is: -m "args", where args is will be given as arguments to muse setup. For example to request envset pabc
```
kk2ups -v "99.9.0" -b -i -m "-q pabc" -j 8
```
Internally kk2ups looks up the version of root that is chosen by "muse setup $args" and uses that version of root to build KinKal. In step 6, args is an empty string, which tells kk2ups to use the default envset. Because you have a clone of Offline in your Muse working area, the default envset is that specified by Offline/.muse.
kk2ups has an online help feature:
```
kk2ups -?
```
To learn more about kk2ups you can read https://github.com/Mu2e/codetools/blob/main/KinKal_to_UPS/README.md.

Using An Alternate envset

If you need to use an alternative envset, you need to modify two steps in the above procedure. In this example, the alternate envset is named pabc:

In step 6, supply the alternate envset as the value of the -m argument to kk2ups
```
kk2ups -v "99.9.0" -b -i -m "-q pabc" -j 8
```

In step 10.1, make a copy of the alternate envset

mkdir muse
cd muse
cp /cvmfs/mu2e.opensciencegrid.org/DataFiles/Muse/pabc unnn

Be sure to use the same envset in both places.

Rebuilding KinKal

In earlier versions of this workflow there were explicit steps required to rebuild KinKal when needed. These steps are now done automatically/

Returning to the area

If you want to restart work in a new shell, follow these steps:

cd 'workdir'
setup mu2e
setup codetools
muse setup -q unnn

where unnn is the envset you created during the initial build. You can then edit files and build as needed.

Adding debug build

Due to the way that UPS works, it is not possible to work on both prof and debug builds in the same shell. You need to do each in a separate shell.

To add a debug build to an existing prof build, start a clean shell and:

cd 'myworkdir'
setup mu2e
setup codetools
muse setup -q unnn:debug
muse build -j 8

On mu2ebuild01 you may change the 8 to 24.

This will make a debug build of KinKal and install in it the local artexternals. Then it will do a debug build of Offline and any other repos you cloned in step 3. If you have not modified KinKal since you first executed step 6, the build of KinKal will be fast because there is no work to do.

What if you update Offline?

If you update the version of Offline in your working area the first step is to compare the old and new versions of Offline to see if the envset specified in Offline/.muse has changed. If it has not changed, there are no additional steps and you can proceed as described in #Returning to the area.

If it has changed, do the folllowing in a clean shell. In this example the new envset is named pdef.

Setup the environment:

cd 'myworkdir'
setup mu2e
setup codetools

Choose a new version number for the UPS version of KinKal, say 9.9.1. and repeat step 6

kk2ups -v "99.9.1" -b -i -m "" -j 8

There is no need to specify the new envset because kk2ups will pick it up from Offline/.muse.

Choose a new name for your modified envset, say u047.
Repeat steps 9 and 10 using the new envset name, pdef, the new KinKal version 9.9.1 and the new modifed envset name u047
In step 11 use u047 instead of u046.

When this work is completed you will see both versions of KinKal in artexternals and both builds of Offline in your build subdirectory. If do not expect to return to the old builds it is safe to delete them.