KinKal: Difference between revisions

From Mu2eWiki
Jump to navigation Jump to search
No edit summary
 
(10 intermediate revisions by the same user not shown)
Line 2: Line 2:
==Local KinKal development with Muse==
==Local KinKal development with Muse==


<font color=red> Under development morning of Aug 3, 2023</font>
===Introduction===


===Introduction===
<font color=red size=+1>These instructions only work on SL7/UPS, not on AL9/Spack.</font>


This section describes how to do simultaneous development of KinKal and Offline.  Because KinKal uses a different build system than does Offline, you need to take a few additional steps when you start.  Once you have taken these steps "muse build" will first build KinKal and install it into a local UPS area; it will then build Offline using the local UPS area as the source for finding KinKal.
This section describes how to do simultaneous development of KinKal and Offline.  Because KinKal uses a different build system than does Offline, you need to take a few additional steps when you start.  Once you have taken these steps "muse build" will first build KinKal, but only if necessary, and install it into a local UPS area; it will then build Offline using KinKal from the the local UPS area.


This writeup assumes that you have a basic knowledge of [[UPS]] and of [[Muse]]; in particular you should understand what a Muse envset is and how to use it.  Normally you will choose the envset recommended by Offline and this example is written assuming that's what you want to do.  It will note where you need to make different choices if you wish to use a different envset. If you are new to this work and you need to use a different envset, your mentor will tell you.
This writeup assumes that you have a basic knowledge of [[UPS]] and of [[Muse]]; in particular you should understand what a Muse envset is and how to use it.  Normally you will choose the envset recommended by Offline and this example is written assuming that's what you want to do.  If you are new to this work and you need to use a different envset, your mentor will tell you which one to use.  Section  [[#Using An Alternate envset]] describes how the workflow must be changed in order to use a different envset.


===Initial build===
===Initial build===
To start development the first step is to decide which Muse envset you will start from.  Normally you will choose the envset recommended by Offline; if you are new to this work and it is important to use a different envset, your mentor will tell you.


<ol>
<ol>
<li>  In a clean shell, create a [[Muse]] workDir and cd there
<li>  In a clean shell, create a [[Muse]] workDir and cd there
  <pre>
<pre>
  mkdir 'myworkdir'
mkdir 'myworkdir'
  cd 'myworkdir'
cd 'myworkdir'
  </pre> </li>
</pre> </li>
<li>clone KinKal into the Muse workdir as your would normally.  Add the Mu2e KinKal repo as a remote and checkout the main branch of the Mu2e KinKal remote.
<li>clone KinKal into the Muse workdir as your would normally.  Add the Mu2e KinKal repo as a remote and checkout the main branch of the Mu2e KinKal remote.
<pre>
<pre>
Line 29: Line 26:
cd ..
cd ..
</pre>
</pre>
If you already have work in progress on work_branch, in your fork of KinKal, change the git checkout command to choose that branch.
</li>
</li>
<li> clone your fork of Offline into the muse workdir as you would normally.  Add the Mu2e Offline repo as a remote and checkout the main branch of the Mu2e Offline remote.  If you need other muse-aware repos (Production, TrkAna, ...) for your workflow, you can add them too
<li> Clone your fork of Offline into the muse workdir as you would normally.  Add the Mu2e Offline repo as a remote and checkout the main branch of the Mu2e Offline remote.  If you need other muse-aware repos (Production, TrkAna, ...) for your workflow, you can add them too
<pre>
<pre>
git clone git@github.com:<your GitHub username>/Offline
git clone git@github.com:<your GitHub username>/Offline
Line 38: Line 34:
git checkout -b work_branch mu2e/master
git checkout -b work_branch mu2e/master
cd ..
cd ..
( git clone git@github.com:<your GitHub username>/Production )
# Optional: repeat the above pattern for any of Production, TrkAna ...
( git clone git@github.com:<your GitHub username>/TrkAna )
</pre></li>
</pre></li>
<li> Setup the development environment
<li> Setup the Mu2e development environment
<pre>
<pre>
setup mu2e
mu2einit
setup codetools
setup codetools
</pre> </li>
</pre> </li>
<li> Choose a UPS version number for your local build of KinKal; you can use any number thet is not already in use. Choosing version 9.9.9 is likely to be safe for a long time to come.  The version must have 3 dot separated fields, following the idea of [[ReleaseList#Semantic_Versioning | semantic versioning]]. </li>
<li> Choose a UPS version number for your local build of KinKal.  The number will have no meaning outside of your muse working directory so you can use any number that is not already in use. Choosing version 99.9.0 is likely to be safe for a long time to come.  The version number must have 3 dot separated fields.</li>
<li> Build KinKal; create a local ups directory and install the build of KinKal into that UPS area.
<li> Build KinKal; create a local ups directory and install the build of KinKal into that UPS area.
<pre>
<pre>
kk2ups -v "9.9.9" -b -i -m "" -j 8
kk2ups -v "99.9.0" -b -i -m "" -j 8
</pre>
</pre>
On mu2ebuild02 you may increase 8 to 32.  This command will build both the profile and debug versions of KinKal.  The option -m with an empty string tells Kkk2ups to choose the version of root that matches the default for building Offline. See section [[#Using An Alternate envset]] for when you should provide a non-empty string.
</li>
</li>
<li> If everything worked correctly, the output of this command will end with
<li> If everything worked correctly, the output of this command will end with
Line 59: Line 55:
KinKal_to_UPS/build completed with status 0
KinKal_to_UPS/build completed with status 0
</pre>
</pre>
</li>
<li> The command kk2ups is short for "KinKal to UPS" and is found in codetools.  This step will
<ol>
  <li> (-b) build the existing checkout in the local clone of KinKal
  <li> (-i) if necessary, create a local directory named artexternals, configure it as a UPS repositor
  <li>  (-i) copy the build of KinKal in artexternals/KinKal/v09_09_09 and configure it as a UPS product
  </ol>
</li>
</li>
<li> Look at the files created by kk2ups:
<li> Look at the files created by kk2ups:
<pre>
<pre>
ls artexternals/KinKal/
ls artexternals/KinKal/
v09_09_09  v09_09_09.version
</pre>
</pre>
</li>
This will produce the output:
<li> The full syntax of the kk2ups option -m is: -m "value", where value is the name of an existing Muse envset. It says that, when building KinKal, use the version of root that will be found by the command "muse setup -q value".  In this example value is an empty string which tells kk2ups to use the default envset set; because you have a clone of Offline in your Muse working area, it will use the version specified by Offline/.muse.
<pre>
</li>
v99_09_00  v99_09_00.version
<li> To learn more about kk2ups you can read [https://github.com/Mu2e/codetools/blob/main/KinKal_to_UPS/README.md] or look at the output of
</pre>
Also do
<pre>
ls artexternals/KinKal/v99_09_00
</pre>
This will produce output similar to:
<pre>
<pre>
kk2ups -?
include  slf7.x86_64.debug.e20.p3915  slf7.x86_64.e20.p3915.prof  source  ups
</pre>
</pre>
As Mu2e Offline evolves to newer versions of g++ and python, the qualifiers e20 and p3915 will change.
</li>
</li>
<li> Make a copy of the envset used to build KinKal.  If you used the default of -m "" in step 6, you can find the name of the envset in the file Offline/.muse .
<li> Look at Offline/.muse to find the name of the default envset for building Offline. In these instructions we will call that pxyz.  Make a local muse subdirectory and make a copy of pxyz inside of it.
<pre>
<pre>
mkdir muse
mkdir muse
Line 86: Line 80:
cp /cvmfs/mu2e.opensciencegrid.org/DataFiles/Muse/pxyz unnn
cp /cvmfs/mu2e.opensciencegrid.org/DataFiles/Muse/pxyz unnn
</pre>
</pre>
where pxzy is the name of the envset used to build KinKal and where unnn is the name of your new envset.
where unnn is the name of your copy envset, for example u046.  The name unnn has no meaning outside of your muse working directory.
</li>
</li>
<li>  Edit the envset unnn to make two changes
<li>  Edit the envset unnn to make two changes
   <ol>
   <ol>
     <li> copy so that it references your local version of KinKal.  I the following line IE replace the version number in the line 'setup -B KinKal  v00_01_06  -q +${MUSE_BUILD}:+${MUSE_COMPILER_E}:+${MUSE_PYTHON}' with your tag (v09_09_09).
     <li> Find the line that sets up KinKal and replace the version number with v99_09_00
   <pre>
   <pre>
   setup -B KinKal vxx_yy_zz -q +${MUSE_BUILD}:+${MUSE_COMPILER_E}:+${MUSE_PYTHON}' with your tag (v09_09_09).
   setup KinKal v99_09_00 -q +${MUSE_BUILD}:+${MUSE_COMPILER_E}:+${MUSE_PYTHON}'
   </pre>
   </pre>
   change the version number to v09_09_09. </li>
   </li>
   <li> Add the following line after the last setup command:
   <li> Add the following line after the last setup command:
     <pre>
     <pre> export MUSE_PREBUILD="kk2ups -v \"99.9.0\" -b -i -j 24" </pre>
      export MUSE_PREBUILD="kk2ups -v \"999.0.0\" -b -i -j 24"
This line is not the same as the command given in step 6; it omits the -m argument.  When muse executes this command a version of root will already be setup in your environmen and kk2ups will use that version of root.
    </pre>
   </li>
   </li>
   </ol>
   </ol>
<li> cd back to the muse working directory and setup muse using the envset you just created
<li> cd back to the muse working directory and setup muse using the envset you just created
   <pre>
   <pre>cd ..
    cd ..
muse setup -q unnn</pre>
    muse setup -q unnn
<li> Build Offline and any other repos that you cloned in step 3; if anything has changed in KinKal since the last build, this command will also rebuild KinKal and reinstall it in the local artexternals.
  </pre>
   <pre>muse build -j 8 </pre>
<li> Build Offline and any other repos that you cloned in step ??
   On mu2ebuild02 you may increase 8 to 32.</li>
   <pre>
<li> Now you can edit files in any of KinKal, Offline and any other locally cloned repos. When you want to rebuild you can do so with a single command:
      muse build -j 8
  <pre>muse build -j 8 </pre>
   </pre>
  On mu2ebuild02 you may increase 8 to 32.</li>
  On mu2ebuild01 you can increase 8 to 24
</ol>
</ol>


===Rebuilding KinKal===
====About kk2ups ====
Once you have a stock build of Offline and KinKal, you can can develop both code sets coherently.
<ol>
* edit KinKal as needed
<li> The command kk2ups is short for "KinKal to UPS" and is found in codetools.  The command you issued in step 6 will
* edit Offline as needed to exploit new KinKal features
<ol>
* rebuild KinKal and update the installation
  <li> (-b) build the existing checkout in the local clone of KinKal
* rebuild Offline
  <li> (-i) if necessary, create a local directory named artexternals, configure it as a UPS repositor
  <li>  (-i) copy the build of KinKal into artexternals/KinKal/v99_09_00 and configure it as a UPS product.
  </ol>
</li>
 
<li> The full syntax of the kk2ups option -m is: -m "args", where args is will be given as arguments to muse setup.  For example to request envset pabc
<pre>
<pre>
(edit KinKal/*/*)
kk2ups -v "99.9.0" -b -i -m "-q pabc" -j 8
(edit Offline/*/*)
cd build_profile
make -j 20
make test
cd ..
KinKal_to_UPS/build -v v9.9.9 -i -d artexternals
muse build
</pre>
</pre>
===Returning to the area===
Internally kk2ups looks up the version of root that is chosen by "muse setup $args" and uses that version of root to build KinKal.  In step 6, args is an empty string, which tells kk2ups to use the default envset. Because you have a clone of Offline in your Muse working area, the default envset is that specified by Offline/.muse.
* to return in a new shell and simply run the code you've already edited and built, you only need to repeat the setups:
</li>
<li> kk2ups has an online help feature:
<pre>
<pre>
cd 'workdir'
kk2ups -?
setup mu2e
export PRODUCTS=${PWD}/artexternals:${PRODUCTS}
muse setup -q u999
</pre>
</pre>
<li> To learn more about kk2ups you can read [https://github.com/Mu2e/codetools/blob/main/KinKal_to_UPS/README.md https://github.com/Mu2e/codetools/blob/main/KinKal_to_UPS/README.md].
</ol>
===Using An Alternate envset===


===Adding debug build===
If you need to use an alternative envset, you need to modify two steps in the above procedure.  In this example, the alternate envset is named pabc:
An environment can only be setup once, so for an process, you must choose one or the other.  The steps all follow form the prof build
<ul>
* setups.  Setup muse in debug.  Since you don't have the new KinKal in debug, you can setup root by setting up the published versions
<li> In step 6, supply the alternate envset as the value of the -m argument to kk2ups
<pre>
<pre>
setup mu2e
kk2ups -v "99.9.0" -b -i -m "-q pabc" -j 8
cd 'workdir'
</pre></li>
muse setup -q debug
<li> In step 10.1, make a copy of the alternate envset
</pre>
* build debug KinKal using the same fake version as for prof
<pre>
<pre>
mkdir build_debug
mkdir muse
cd build_debug
cd muse
cmake ../KinKal  -DCMAKE_BUILD_TYPE=Debug
cp /cvmfs/mu2e.opensciencegrid.org/DataFiles/Muse/pabc unnn
make -j 20
make test
cd ..
KinKal_to_UPS/build -v v9.9.9 -i -d artexternals
</pre>
</pre>
* now that the debug version of the local KinKal product is available, you need to setup in a new shell to get the full environment
</li>
</ul>
Be sure to use the same envset in both places.
 
===Rebuilding KinKal===
In earlier versions of this workflow there were explicit steps required to rebuild KinKal when needed.  These steps are now done automatically/
 
 
===Returning to the area===
 
If you want to restart work in a new shell, follow these steps:
<pre>
<pre>
exit
ssh 'myuser@mu2egpvm04.fnal.gov'
cd 'workdir'
cd 'workdir'
setup mu2e
mu2einit
export PRODUCTS=${PWD}/artexternals:${PRODUCTS}
setup codetools
muse setup -q u999:debug
muse setup -q unnn
muse build
</pre>
</pre>
where unnn is the envset you created during the initial build.  You can then edit files and build as needed.


===What if you update Offline?===


If you update the version of Offline in your working area:
===Adding debug build===
# Compare the old and new versions of Offline to see if the version or qualifiers of root have changed.
# If they have not, you are done with this section.  If they have, then you need to do the following:
# Make a new build of KinKal using the modified root and install it in your local artexternals directory.
## Hint: you may wish to investigate the -s versionSuffix option in KinKal_to_UPS.
## Hint: remember to start a new login session
# Follow the instructions in [[Muse#Maintaining_a_Customized_Setup]] to update your local muse/uNNN file to select your new build of KinKal.


==Original Notes==
Due to the way that UPS works, it is not possible to work on both prof and debug builds in the same shell.  You need to do each in a separate shell.


To develop new code in KinKal and call it in Offline you will need to define your own version of KinKal and install it in your own private extension to UPS.  You need to do some initial setup in a throwaway shell:
To add a debug build to an existing prof build, start a clean shell and:
* create a [[Muse]] workDir and cd there
<pre>
* checkout your fork of Offline in the muse workdir as you would normally
cd 'myworkdir'
* checkout your fork of KinKal in the muse workdir .The KinKal development model is the same as Offline: start with the head of main, create your own branch, develop and commit to that, and make a PR to KinKal when your development is ready.
mu2einit
* checkout tag v0.1.4 of KinKal_to_UPS
setup codetools
muse setup -q unnn:debug
muse build -j 8
</pre>
On mu2ebuild02 you may change the 8 to 32.


* build KinKal according to the instructions in README.  make sure to run the test (make test)
This will make a debug build of KinKal and install in it the local artexternals. Then it will do a debug build of Offline and any other repos you cloned in step 3.  If you have not modified KinKal since you first executed step 6, the build of KinKal will be fast because there is no work to do.


* create the directory 'artexternals' in the muse workdir
===What if you update Offline?===
* execute 'muse setup -q (prof, debug) depending on what you want.  Note you can have both builds in the same workdir, but any given shell can be only one or the other.
* invoke KinKal_to_UPS/build -v your_version -i -b -j10, where your_version is whatever version you want (it is temporary).  I have this command saved in a script /mu2e/app/users/brownd/KFTrack/install_KinKal.sh
* define a user envset in the ./muse directory.  mine is /mu2e/app/users/brownd/KFTrack/muse/u002 The name is unimportant.  Edit that so it refers to your_version of KinKal


Then, close that shell.  From then on, you can work with KinKal and/or Offline in this muse workdir with the following commands:
If you update the version of Offline in your working area the first step is to compare the old and new versions of Offline
* create a new shell, and cd to your muse workdir
to see if the envset specified in Offline/.muse has changed.  If it has not changed, there are no additional steps and you can proceed as described in
* exectue 'export PRODUCTS=${PWD}/artexternals:${PRODUCTS}'  This tells UPS to look for KinKal in your local install
[[#Returning to the area]].
* execute 'muse setup -q (prof, debug) your_envset'
(I have these defined as  /mu2e/app/users/brownd/KFTrack/setup_prof.sh and setup_debug.sh in my area)'
* Any time you change KinKal rebuild and rerun the tests, and then rerun install_KinKal.sh
* Any time you change Mu2eKinKal (or after changing and rebuilding KinKal as above) rebuild Offline using 'muse build'


==Second version: Local KinKal development with Muse==
If it has changed, do the folllowing in a clean shell.  In this example the new envset is named pdef.


 
<ol>
===Initial build===
<li> Setup the environment:
To develop new code in KinKal for new use cases in Offline you will need to checkout KinKal and install it in your own private extension to UPS.  You need to do some initial setup in a throwaway shell:
* create a [[Muse]] workDir and cd there
* setup Muse.  This puts a root and other products in the environment, which the KinKal build will use
<pre>
<pre>
mkdir 'myworkdir'
cd 'myworkdir'
cd 'myworkdir'
mu2einit
setup codetools
</pre>
</pre>
* checkout your fork of Offline in the muse workdir as you would normally.  Add the main mu2e repo as a remote.  If you need other muse-aware repos (Production, TrkAna, ...) for your workflow, you can add them too
<li> Choose a new version number for the UPS version of KinKal, say 9.9.1. and repeat step 6</li>
<pre>
<pre>
git clone git@github.com:<your GitHub username>/Offline
kk2ups -v "99.9.1" -b -i -m "" -j 8
cd Offline
git remote add -f mu2e https://github.com/Mu2e/Offline
git checkout -b work_branch mu2e/master
cd ..
(git clone git@github.com:<your GitHub username>/TrkAna.git)
</pre>
</pre>
* create a fork of KinKal in GitHub (if you don't already have one).  In a browser, go to the [https://github.com/KFTrack/KinKal KinKal GitHub page], then under the 'fork' menu select 'Create a new fork'.  You must be logged in to your GitHub account.
There is no need to specify the new envset because kk2ups will pick it up from Offline/.muse.
* checkout your fork of KinKal in the muse workdir .The KinKal development model is the same as Offline: start with the head of main, create your own branch, develop and commit to that, and make a PR to KinKal when your development is ready.  Start with the current version of KinKal
</li>
<pre>
<li> Choose a new name for your modified envset, say u047.   
git clone git@github.com:<your GitHub username>/KinKal
<li> Repeat steps 9 and 10 using the new envset name, pdef, the new KinKal version 9.9.1 and the new modifed envset name u047
cd KinKal
<li> In step 11 use u047 instead of u046.
git remote add -f mu2e https://github.com/KFTrack/KinKal
</ol>
git checkout -b work_branch mu2e/$KINKAL_VERSION
When this work is completed you will see both versions of KinKal in artexternals and both builds of Offline in your build subdirectory.  If do not expect to return to the old builds it is safe to delete them.
cd ..
</pre>
* build KinKal according  to the instructions in README.  You can build either a profile (optimized)
or debug.  If you want both, just repeat this step.
mkdir build_profile (or build_debug)
cd build_profile (or build_debug)
cmake ../KinKal  -DCMAKE_BUILD_TYPE=Release (or Debug)
make -j 20
make test
cd ..
</pre>
* checkout KinKal_to_UPS, and use it to make your build of KinKal visible to Offline through UPS.
<pre>
git clone https://github.com/Mu2e/KinKal_to_UPS
mkdir artexternals
rsync -aur /cvmfs/mu2e.opensciencegrid.org/artexternals/.upsfiles artexternals/
</pre>
* install your build of KinKal in artexternals.  You must assign a tag to your build; the value is arbitrary, but it shouldn't correspond to an actual tag.  Below a dummy is used.
<pre>
KinKal_to_UPS/build -v v9.9.9 -i -d artexternals
</pre>
* create an envset that points to your local UPS version of KinKal. Start with a copy of the current Offline envset.
* edit the copy so that it references your local version of KinKal.  IE replace the tag in the line 'setup -B KinKal  v00_01_06  -q +${MUSE_BUILD}:+${MUSE_COMPILER_E}:+${MUSE_PYTHON}' with your tag (v09_09_09)
<pre>
mkdir muse
cp $MUSE_ENVSET_DIR/$MUSE_ENVSET muse/u999
vim muse/u999
...
</pre>
At this point, KinKal is built and UPS-installed locally, but the current setup of Offline still points to the old version of KinKal. To get Offline to use your new version of KinKal you need a new shell. Close your existing shell, and start a new one, then cd back to your work directory.  Note you only need to do this when setting up a new work directory; the following commands are needed for all subsequent use of this workdir
* close your current shell
* create a new shell, and cd to your muse workdir. 
* Tell UPS where to look find your local version of KinKal
* setup muse, specifying the envset you created, which points to your local KinKal installation
<pre>
exit (closes the current shell)
ssh 'myuser'@mu2egpvm04.fnal.gov
cd 'myworkdir'
setup mu2e
export PRODUCTS=${PWD}/artexternals:${PRODUCTS}
muse setup -q u999
</pre>
* Now build Offline (and any other muse-aware repos you've cloned)
muse build -j 20
 
===Rebuilding KinKal===
Once you have a stock build of Offline and KinKal, you can can develop both code sets coherently.
* edit KinKal as needed
* edit Offline as needed to exploit new KinKal features
* rebuild KinKal and update the installation
* rebuild Offline
<pre>
(edit KinKal/*/*)
(edit Offline/*/*)
cd build_profile
make -j 20
make test
cd ..
KinKal_to_UPS/build -v v9.9.9 -i -d artexternals
muse build
</pre>
===Returning to the area===
* to return in a new shell and simply run the code you've already edited and built, you only need to repeat the setups:
<pre>
cd 'workdir'
setup mu2e
export PRODUCTS=${PWD}/artexternals:${PRODUCTS}
muse setup -q u999
</pre>
 
===Adding debug build===
An environment can only be setup once, so for an process, you must choose one or the other.  The steps all follow form the prof build
* setups.  Setup muse in debug.  Since you don't have the new KinKal in debug, you can setup root by setting up the published versions
<pre>
setup mu2e
cd 'workdir'
muse setup -q debug
</pre>
* build debug KinKal using the same fake version as for prof
<pre>
mkdir build_debug
cd build_debug
cmake ../KinKal  -DCMAKE_BUILD_TYPE=Debug
make -j 20
make test
cd ..
KinKal_to_UPS/build -v v9.9.9 -i -d artexternals
</pre>
* now that the debug version of the local KinKal product is available, you need to setup in a new shell to get the full environment
<pre>
exit
ssh 'myuser@mu2egpvm04.fnal.gov'
cd 'workdir'
setup mu2e
export PRODUCTS=${PWD}/artexternals:${PRODUCTS}
muse setup -q u999:debug
muse build
</pre>
 
===What if you update Offline?===
 
If you update the version of Offline in your working area:
# Compare the old and new versions of Offline to see if the version or qualifiers of root have changed.
# If they have not, you are done with this section.  If they have, then you need to do the following:
# Make a new build of KinKal using the modified root and install it in your local artexternals directory.
## Hint: you may wish to investigate the -s versionSuffix option in KinKal_to_UPS.
## Hint: remember to start a new login session
# Follow the instructions in [[Muse#Maintaining_a_Customized_Setup]] to update your local muse/uNNN file to select your new build of KinKal.
 
==Original Notes==

Latest revision as of 13:27, 26 July 2024

Local KinKal development with Muse

Introduction

These instructions only work on SL7/UPS, not on AL9/Spack.

This section describes how to do simultaneous development of KinKal and Offline. Because KinKal uses a different build system than does Offline, you need to take a few additional steps when you start. Once you have taken these steps "muse build" will first build KinKal, but only if necessary, and install it into a local UPS area; it will then build Offline using KinKal from the the local UPS area.

This writeup assumes that you have a basic knowledge of UPS and of Muse; in particular you should understand what a Muse envset is and how to use it. Normally you will choose the envset recommended by Offline and this example is written assuming that's what you want to do. If you are new to this work and you need to use a different envset, your mentor will tell you which one to use. Section #Using An Alternate envset describes how the workflow must be changed in order to use a different envset.

Initial build

  1. In a clean shell, create a Muse workDir and cd there 
    mkdir 'myworkdir'
cd 'myworkdir'
  2. clone KinKal into the Muse workdir as your would normally. Add the Mu2e KinKal repo as a remote and checkout the main branch of the Mu2e KinKal remote. 
    git clone git@github.com:<your GitHub username>/KinKal
cd KinKal
git remote add -f mu2e https://github.com/KFTrack/KinKal
git checkout -b work_branch mu2e/main
cd ..
  3. Clone your fork of Offline into the muse workdir as you would normally. Add the Mu2e Offline repo as a remote and checkout the main branch of the Mu2e Offline remote. If you need other muse-aware repos (Production, TrkAna, ...) for your workflow, you can add them too 
    git clone git@github.com:<your GitHub username>/Offline
cd Offline
git remote add -f mu2e https://github.com/Mu2e/Offline
git checkout -b work_branch mu2e/master
cd ..
# Optional: repeat the above pattern for any of Production, TrkAna ...
  4. Setup the Mu2e development environment 
    mu2einit
setup codetools
  5. Choose a UPS version number for your local build of KinKal. The number will have no meaning outside of your muse working directory so you can use any number that is not already in use. Choosing version 99.9.0 is likely to be safe for a long time to come. The version number must have 3 dot separated fields.
  6. Build KinKal; create a local ups directory and install the build of KinKal into that UPS area. 
    kk2ups -v "99.9.0" -b -i -m "" -j 8

    On mu2ebuild02 you may increase 8 to 32. This command will build both the profile and debug versions of KinKal. The option -m with an empty string tells Kkk2ups to choose the version of root that matches the default for building Offline. See section #Using An Alternate envset for when you should provide a non-empty string.

  7. If everything worked correctly, the output of this command will end with 
    Begin install.
Begin install of prof version.
Begin install of debug version.
KinKal_to_UPS/build completed with status 0
  8. Look at the files created by kk2ups: 
    ls artexternals/KinKal/

    This will produce the output: 

    v99_09_00  v99_09_00.version

    Also do 

    ls artexternals/KinKal/v99_09_00

    This will produce output similar to: 

    include  slf7.x86_64.debug.e20.p3915  slf7.x86_64.e20.p3915.prof  source  ups

    As Mu2e Offline evolves to newer versions of g++ and python, the qualifiers e20 and p3915 will change.

  9. Look at Offline/.muse to find the name of the default envset for building Offline. In these instructions we will call that pxyz. Make a local muse subdirectory and make a copy of pxyz inside of it. 
    mkdir muse
cd muse
cp /cvmfs/mu2e.opensciencegrid.org/DataFiles/Muse/pxyz unnn

    where unnn is the name of your copy envset, for example u046. The name unnn has no meaning outside of your muse working directory.

  10. Edit the envset unnn to make two changes
    1. Find the line that sets up KinKal and replace the version number with v99_09_00 
         setup KinKal v99_09_00  -q +${MUSE_BUILD}:+${MUSE_COMPILER_E}:+${MUSE_PYTHON}'
    2. Add the following line after the last setup command: 
       export MUSE_PREBUILD="kk2ups -v \"99.9.0\" -b -i -j 24"

      This line is not the same as the command given in step 6; it omits the -m argument. When muse executes this command a version of root will already be setup in your environmen and kk2ups will use that version of root.

  11. cd back to the muse working directory and setup muse using the envset you just created 
    cd ..
muse setup -q unnn
  12. Build Offline and any other repos that you cloned in step 3; if anything has changed in KinKal since the last build, this command will also rebuild KinKal and reinstall it in the local artexternals. 
    muse build -j 8
    On mu2ebuild02 you may increase 8 to 32.
  13. Now you can edit files in any of KinKal, Offline and any other locally cloned repos. When you want to rebuild you can do so with a single command: 
    muse build -j 8
    On mu2ebuild02 you may increase 8 to 32.

About kk2ups

  1. The command kk2ups is short for "KinKal to UPS" and is found in codetools. The command you issued in step 6 will
    1. (-b) build the existing checkout in the local clone of KinKal
    2. (-i) if necessary, create a local directory named artexternals, configure it as a UPS repositor
    3. (-i) copy the build of KinKal into artexternals/KinKal/v99_09_00 and configure it as a UPS product.
  2. The full syntax of the kk2ups option -m is: -m "args", where args is will be given as arguments to muse setup. For example to request envset pabc 
    kk2ups -v "99.9.0" -b -i -m "-q pabc" -j 8

    Internally kk2ups looks up the version of root that is chosen by "muse setup $args" and uses that version of root to build KinKal. In step 6, args is an empty string, which tells kk2ups to use the default envset. Because you have a clone of Offline in your Muse working area, the default envset is that specified by Offline/.muse.

  3. kk2ups has an online help feature: 
    kk2ups -?
  4. To learn more about kk2ups you can read https://github.com/Mu2e/codetools/blob/main/KinKal_to_UPS/README.md.

Using An Alternate envset

If you need to use an alternative envset, you need to modify two steps in the above procedure. In this example, the alternate envset is named pabc:

  • In step 6, supply the alternate envset as the value of the -m argument to kk2ups 
    kk2ups -v "99.9.0" -b -i -m "-q pabc" -j 8
  • In step 10.1, make a copy of the alternate envset 
    mkdir muse
cd muse
cp /cvmfs/mu2e.opensciencegrid.org/DataFiles/Muse/pabc unnn

Be sure to use the same envset in both places.

Rebuilding KinKal

In earlier versions of this workflow there were explicit steps required to rebuild KinKal when needed. These steps are now done automatically/


Returning to the area

If you want to restart work in a new shell, follow these steps: 

cd 'workdir'
mu2einit
setup codetools
muse setup -q unnn

where unnn is the envset you created during the initial build. You can then edit files and build as needed.


Adding debug build

Due to the way that UPS works, it is not possible to work on both prof and debug builds in the same shell. You need to do each in a separate shell.

To add a debug build to an existing prof build, start a clean shell and: 

cd 'myworkdir'
mu2einit
setup codetools
muse setup -q unnn:debug
muse build -j 8

On mu2ebuild02 you may change the 8 to 32.

This will make a debug build of KinKal and install in it the local artexternals. Then it will do a debug build of Offline and any other repos you cloned in step 3. If you have not modified KinKal since you first executed step 6, the build of KinKal will be fast because there is no work to do.

What if you update Offline?

If you update the version of Offline in your working area the first step is to compare the old and new versions of Offline to see if the envset specified in Offline/.muse has changed. If it has not changed, there are no additional steps and you can proceed as described in #Returning to the area.

If it has changed, do the folllowing in a clean shell. In this example the new envset is named pdef.

  1. Setup the environment: 
    cd 'myworkdir'
mu2einit
setup codetools
  2. Choose a new version number for the UPS version of KinKal, say 9.9.1. and repeat step 6
    3. kk2ups -v "99.9.1" -b -i -m "" -j 8

    There is no need to specify the new envset because kk2ups will pick it up from Offline/.muse.

  3. Choose a new name for your modified envset, say u047.
  4. Repeat steps 9 and 10 using the new envset name, pdef, the new KinKal version 9.9.1 and the new modifed envset name u047
  5. In step 11 use u047 instead of u046.

When this work is completed you will see both versions of KinKal in artexternals and both builds of Offline in your build subdirectory. If do not expect to return to the old builds it is safe to delete them.

Retrieved from "http://mu2ewiki.fnal.gov/w/index.php?title=KinKal&oldid=25693"