MuseMaintenance: Difference between revisions
Line 27: | Line 27: | ||
==Publishing Production tags== | ==Publishing Production tags== | ||
Production contains most of the top-level fcl to use Offline, so is often updated and published at the same time as Offline. To "build" and publish a tag of Production with tag name ''tag'', in a scratch area. '''Note''' since Production has no default envset, you want to be sure it builds with the same envset (and other qualifiers) as the version of Offline it will be used with. This is necessary because as | Production contains most of the top-level fcl to use Offline, so is often updated and published at the same time as Offline. To "build" and publish a tag of Production with tag name ''tag'', in a scratch area. '''Note''' since Production has no default envset, you want to be sure it builds with the same envset (and other qualifiers) as the version of Offline it will be used with. This is necessary because as Muse sets up, it checks that the builds being incorporated through links all have the build being setup. | ||
setup mu2e | setup mu2e |
Revision as of 19:17, 8 July 2021
Publishing Offline tags
When Offline is tagged, it needs to be build and published on cvmfs in Muse format. A jenkins build project is used to build prof and debug in parallel. If the build fails, the project will return an error. The output is tarballs that can then be installed on cvmfs.
- go to jenkins mu2e-offline-build-muse project. (Requires an account and a cert in the browser.)
- select "Build with Parameters", enter the tag string, and hit "Build". takes about 45 min.
- When done, select "BUILDTYPE=prof"
- if anything went wrong, select "last build" and look at the console output and the log file
- under artifacts, right click on the tarball and "copy link address"
- log onto the cvmfs machine, and start a transaction
- cd to /cvmfs/mu2e.opensciencegrid.org/Musings/, wget the tarball, extract it (bzipped, use "-j")
- remove tarball
- repeat for the debug tarball
- if needed, repoint the "Offline/current" link to this new tag
- cd /cvmfs/mu2e.opensciencegrid.org/Musings/Offline
- rm current
- ln -s vxx_xx_xx current
- cd ~ and publish the transaction
- once cvmfs is updated, test with
muse setup tag
The script that this project runs is in the codetools repo,
codetools/bin/jenkinsMuseBuild.sh
When the jenkins project is run, it checks out the head of codetools to find the script.
Publishing Production tags
Production contains most of the top-level fcl to use Offline, so is often updated and published at the same time as Offline. To "build" and publish a tag of Production with tag name tag, in a scratch area. Note since Production has no default envset, you want to be sure it builds with the same envset (and other qualifiers) as the version of Offline it will be used with. This is necessary because as Muse sets up, it checks that the builds being incorporated through links all have the build being setup.
setup mu2e setup muse git clone https://github.com/Mu2e/Production git -C Production checkout -b temp tag muse setup muse build muse tarball -r Production/tag
You will get an output tarball. In a second window, in the same directory
setup mu2e setup muse muse setup -q debug muse build muse tarball -r Production/tag
You will get another tarball. Even though Production doesn't produce anything in the build area, it is necessary create the build directories so the setup command doesn't give warnings about "Production wasn't built with this setup". Muse doesn't have a concept of a repo that build with no output, yet.
Then on the cvmfs installation machine
- log onto the cvmfs machine, and start a transaction
- cd to /cvmfs/mu2e.opensciencegrid.org/Musings/, wget the tarball, extract it (bzipped, use "-j")
- remove tarball
- repeat for the debug tarball
- if needed, repoint the "Production/current" link to this new tag
- cd ~ and publish the transaction
Publishing ProdJob
ProdJob is a published Muse build area that contains both Offline and Production. It can serve as the basis for submitting production jobs. Currently, the plan is to publish Offline and Production as separate Musings, then make a ProdJob linking to these two. We expect that Production will be updated more frequently than Offline, and ProdJob allows for this case efficiently.
In a scratch area, assuming an Offline tag otag, a Production tag ptag and a targeted ProdJob tag tag,
setup mu2e setup muse muse link Offline otag muse link Production ptag muse setup muse build muse tarball -r Prodjob/tag
And, as described in previous sections, also setup, build, and tar a "debug" version in a second process. Upload the resulting tarballs to cvmfs Musings area.
Publishing Muse itself
To develop Muse, create a fork in your github account, check it out locally
setup mu2e git clone git@github.com:<your GitHub username>/Muse
and to use this local code instead of "setup muse" do
export MUSE_DIR=$PWD/Muse export PATH=$PWD/Muse/bin:$PATH export MUSE_ENVSET_DIR=$PWD/Muse/config alias muse="source muse"
now you can add envsets and test them directly (as opposed to the muse/uNNN mechanism). All the scripts should work like a normal Muse UPS setup. Develop and commit, pull and tag changes as in any other github repo. To publish tag,
mkdir products museInstall.sh tag ./products
the products directory will contain an UPS-installed version tag, and a tarball suitable to uploading to cvmfs artproducts area. This script does not checkout or attempt to confirm that the tag is correct, it is assuming the code is actually equivalent to the tagged code. To make the new version UPS-current, edit the version number into the file under the Muse/current.chain directory on cvmfs. You don't need a debug version because this is UPS, which has a NULL product qualifier. For some reason I started using the version number pattern vN_NN_NN, maybe following art.
Testing Muse
Muse contains a series of tests than can be run by hand or as a Jenkins project (mu2e_muse_test). The script can be accessed with
setup muse mu2eTest.sh -h