MuseBuildCodeTutorial: Difference between revisions
No edit summary |
|||
(37 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
==Introduction== | ==Introduction== | ||
In the case of [[DetectorIntro|Mu2e]], an [[ | In the case of [[DetectorIntro|Mu2e]], an [[Computing Concepts#art Events|event]] records the detector response during one microbunch, consisting of a short burst of millions of protons hitting the production target, thousands of muons stopping in the aluminum stopping target, and those muons interacting or decaying. This happens nominally every 1695 ns and each of these burst forms a Mu2e event. | ||
When we read and process events to simulate the detector, reconstruct raw data, or analyze data events we will need a framework to organize our executables, and way to control the behavior of the executable, and a format to write the events into. These functions are provided by the ''art | When we read and process events to simulate the detector, reconstruct raw data, or analyze data events we will need a framework to organize our executables, and way to control the behavior of the executable, and a format to write the events into. These functions are provided by the ''art'' software package. | ||
When we want to operate on the data, we write c++ code. We track code files using a code management tool called '''git''', which writes files to a commercial site, [http://www.github.com github]. We have a main repository (or '''repo''') called '''Offline''' and other smaller special-purpose repos. We compile and link the code using a set of Mu2e scripts called [[Muse]]. Inside those scripts, we use a code build system called '''scons'''. Finally, inside scons, we use linux '''gcc''' to compile and link the code. | When we want to operate on the data, we write c++ code. We track code source files using a code management tool called '''git''', which writes files to a commercial site, [http://www.github.com github]. We have a main repository (or '''repo''') called '''Offline''' and other smaller special-purpose repos. We compile and link the code using a set of Mu2e scripts called [[Muse]]. Inside those scripts, we use a code build system called '''scons'''. Finally, inside scons, we use linux '''gcc''' to compile and link the code. | ||
==Art== | ==Art== | ||
Line 132: | Line 27: | ||
==Code== | ==Code== | ||
Please be sure your environment is prepared by consulting [[CodeEnvironment]]. | |||
We will start with a tour of the Mu2e code base. We keep our code organized using a piece of software called '''git'''. This freeware package stores our code in a compact format and keep track of lots of useful things: | We will start with a tour of the Mu2e code base. We keep our code organized using a piece of software called '''git'''. This freeware package stores our code in a compact format and keep track of lots of useful things: | ||
Line 138: | Line 34: | ||
* tracking parallel development by many people, and merging all that work back together when ready. | * tracking parallel development by many people, and merging all that work back together when ready. | ||
So lets look at what is stored in our git repository. If | So lets look at what is stored in our git repository. If you are on the interactive central servers (mu2egpvm or mu2ebuild) | ||
mkdir -p /mu2e/app/users/$USER/tutorial_code | mkdir -p /mu2e/app/users/$USER/tutorial_code | ||
cd mu2e/app/users/$USER/tutorial_code | cd /mu2e/app/users/$USER/tutorial_code | ||
mu2einit | |||
Since we will be doing several builds, create some subdirectories: | Since we will be doing several builds, create some subdirectories: | ||
Line 159: | Line 51: | ||
Here are few git commands to get you started. | Here are few git commands to get you started. | ||
git status | git status | ||
tells you which branch you are on. When multiple people are working on code development, they will probably be working on their own branches - a version of the code they can control. When the changes are tested and approved, the personal, working branch is merged back into the main code branch, called " | tells you which branch you are on. When multiple people are working on code development, they will probably be working on their own branches - a version of the code they can control. When the changes are tested and approved, the personal, working branch is merged back into the main code branch, called "main". See the active branches: | ||
git branch -al | git branch -al | ||
You can switch to a different active working branch: | You can switch to a different active working branch: | ||
Line 207: | Line 99: | ||
event.put(std::move(chCol)); | event.put(std::move(chCol)); | ||
This adds a newly created data product to the event. All of these concepts will be covered in more detail later. | This adds a newly created data product to the event. All of these concepts will be covered in more detail later. | ||
Now cd back out of the Offline directory | |||
cd.. | |||
==Configuring and Building Code== | ==Configuring and Building Code== | ||
Code management can be complex and so tools are developed to do most of the work, and simplify the user's commands. The art package, scons, Muse, and gcc are all | You should now be in the directory tutorial_code/off, which contains the Offline subdirectory. | ||
Code management can be complex and so tools are developed to do most of the work, and simplify the user's commands. The art package, scons, Muse, and gcc are all provided in a system written and maintained by the lab computing division, in a system called [[UPS]]. At this point you have already told the UPS system to make certain packages and tools available in your path. For example, if you ask UPS to show you what it has added to your interactive process: | |||
ups active | ups active | ||
and you should get an answer | and you should get an answer similar to this | ||
Active ups products: | Active ups products: | ||
git v2_30_1 -f Linux64bit+3.10-2.17 -z /cvmfs/mu2e.opensciencegrid.org/artexternals | git v2_30_1 -f Linux64bit+3.10-2.17 -z /cvmfs/mu2e.opensciencegrid.org/artexternals | ||
ups v6_0_8 -f Linux64bit+3.10-2.17 -z /cvmfs/mu2e.opensciencegrid.org/artexternals | ups v6_0_8 -f Linux64bit+3.10-2.17 -z /cvmfs/mu2e.opensciencegrid.org/artexternals | ||
muse v2_07_00 -f NULL -z /cvmfs/mu2e.opensciencegrid.org/artexternals | |||
We will use UPS to get art tools and libraries, and much more, added to our paths, but first we need to discuss a few choices. We can build code for speed, we call this option "prof", or we can tell the compiler to keep the symbols organized, which runs slower, but is useful for debugging, so we call this second option "debug". There are other options for details of how to run simulations, or visualization. The build defaults to "prof", so we will not use any switches, and ask Muse to to use UPS to add art to our paths: | |||
muse setup | |||
Note that "setup muse" is causing UPS to add muse tools to our path, but "muse setup" is asking Muse scripts to add art and everything else to your paths. To prepare for a debug build we would have used "muse setup -q debug". You can only run "muse setup" once in a process so to change anything, you have to start new process and setup with your alternative choices. Try these commands to see where you are now: | |||
ups active | |||
muse status | |||
muse -v status | |||
The basic configuration of the package's versions and build options is now done. | |||
In a typical use pattern, you would modify some of the code, or add code, then you would want to compile it so you can run it. This is the job of the build system. In our case we use | In a typical use pattern, you would modify some of the code, or add code, then you would want to compile it so you can run it. This is the job of the build system. In our case, we continue to use the set of scripts called [[Muse]], which call a package named [[Scons|scons]], which in turn calls gcc to compile code. The build system has to look over the code, decide what needs to be compiled, do the compilation according to a recipe, then see what needs to be linked and do that linking. The first time you run a build, it will compile and link everything. After that, it should only do the steps which are effected by how you changed or added to the code. | ||
Now it is finally time to build: | |||
muse build -j 4 | |||
the "-j 4" starts the build running with 4 threads, which is appropriate if you are logged on a laptop or a mu2egpvm machine. For mu2ebuild02, where you would usually build code, you can use "-j 32" since it has many more cores. | |||
. | |||
Once it starts compiling, you might as well kill it with ''ctrl-c'', since it will take long time (4 min on mu2ebuild02) to run. | |||
If you log off and want to return to work here, you would | |||
mu2einit | |||
cd mu2e/app/users/$USER/tutorial_code/off | |||
muse setup | |||
and you are ready to build and run. cd'ing to the working directory first is important, so Muse knows this is where your code is. | |||
==Releases== | ==Releases== | ||
At appropriate times, we mark the code with a tag such as " | At appropriate times, we mark the code with a tag such as "v09_12_00". This marks a state of the code we want to save. Maybe we are recording the state at a point where we make a production run for simulation, or maybe we are marking major changes in the code. Take look at the [https://github.com/Mu2e/Offline/releases release list] on github. | ||
After tagging, we build the code and save it so anyone can use it - this is "release". | After tagging, we build the code and save it so anyone can use it, without building it themselves, - this is a "release". | ||
Starting a new process (not the one from the last exercise) make a new test area: | |||
mkdir -p /mu2e/app/users/$USER/tutorial_code/two | |||
cd mu2e/app/users/$USER/tutorial_code/two | |||
mu2einit | |||
Now list what releases are available | |||
muse backing | |||
You will see two sets of Offline builds, one labeled "published releases" and one labeled "CI builds". We will use the former. You should see some release versions. To select a version, and link it to this area: | |||
muse backing Offline v10_05_00 | |||
you will now see a subdirectory created, with a link to the tagged release code. | |||
Now to up the game a bit, lets add a small repo to this area: | |||
git clone https://github.com/Mu2e/Tutorial | |||
This repo is some example code, but in practice you can add repos providing ntuples, other tools, or analysis. Now that this code directory is organized, we can setup and build it. | |||
muse setup | |||
muse build -j 4 | |||
If successful, you see that a subdirectory "build" appeared. You can explore this director to find object files and shared-object libraries. You path will include these build products, so you could run them. | |||
==Partial Builds== | ==Partial Builds== | ||
When we started building the code in the exercise above, we killed it because it would take too long. On a | When we started building the code in the exercise above, we killed it because it would take too long. On a mu2egpvm node it will take an 45 min, and on mu2ebuild02 with its 32 cores, it will take 4 min. (Note that these times are for a full build from scratch - any following incremental builds are usually much faster.) These long times are a common problem, so we have developed a way to build part of the code. | ||
We need a fully built "base release" that already exists, such as the ones we just looked at. We then make a local directory with just a small piece of the code. We build this small piece, and then take all the rest of the compiled code from the "base release". | |||
In | '''Warning''' In this method you must be aware of a big possible pitfall. If any header file that is compiled into your local small piece of code is different than the same header file compiled into the base release, then the code will have corrupt memory and is likely to fail horribly in unpredictable ways. It is best to use this partial build technique only when building a small part of the code base, and only changing cc files. It works great to do simple things like add a debugging print statement, or test a new module, or modify some fcl. If you have to modify header files, or do extensive code development, it is best to clone the whole Offline repo. | ||
We will need to change the build directory organization, so we will have to start a new process. | |||
mkdir -p /mu2e/app/users/$USER/tutorial_code/part | |||
cd mu2e/app/users/$USER/tutorial_code/part | |||
mu2einit | |||
muse link | |||
Now organize the code in this area. First, take a look at the output of "muse link". Recall there were two sections, "Recent published Offline releases" which we used in the previous exercise, and the second list, the "Recent Offline CI builds". "CI" stand for continuous integration, which means every time the Offline repo changes, we build it and add it to this list. So you have a complete build of the most recent Offline code at your fingertips. You'll see the top line is the most recent, now cut and past that name to tell muse to use it as a backing build: | |||
muse link master/be905d45 | |||
Of course the latest you see will be different from this example. The hex code is a "git hash" and labels one exact state of the Offline repo. Now we have indicated to Muse to use this build for Offline libraries. | |||
Now we will introduce a small piece of Offline code. This piece of code is compiled and provided in the backing build we just linked, but if we also build it locally, Muse will prefer to use the local version, and take all the rest of Offline from the backing build. | |||
To start the local subset of Offline code: | |||
mgit init | |||
You should see an "Offline" directory appear. Now lets add the HellowWorld subdirectory from Offline to this local area | |||
cd Offline | |||
mgit add HelloWorld | |||
ls -l | |||
cd .. | |||
You should see that the HelloWorld subdirectory of Offline has been added locally. Now that the code is organized, tell Muse to finish setting up the paths: | |||
muse setup | |||
ls -l | and build | ||
muse build -j 4 | |||
You | |||
== Exploring The Partial Build== | == Exploring The Partial Build== | ||
Line 345: | Line 207: | ||
<< endl; | << endl; | ||
</pre> | </pre> | ||
The Mu2e wiki has a short writeup about [[ | The Mu2e wiki has a short writeup about [[Computing Concepts#art, EventIDs, Runs and Subruns| EventIDs]]. | ||
<li> Run the HelloWorld module: | <li> Run the HelloWorld module: | ||
<pre> | <pre> | ||
mu2e -c HelloWorld/test/hello.fcl | mu2e -c Offline/HelloWorld/test/hello.fcl | ||
</pre> | </pre> | ||
Identify the printout from the module: | Identify the printout from the module: | ||
Line 356: | Line 218: | ||
Hello, world. From analyze: run: 1 subRun: 0 event: 3 | Hello, world. From analyze: run: 1 subRun: 0 event: 3 | ||
</pre> | </pre> | ||
This will also print | This will also print some warning messages, which will be fixed. | ||
The next tutorial has a lot of information about using fcl files so this tutorial will not discuss them much. | The next tutorial has a lot of information about using fcl files so this tutorial will not discuss them much. | ||
<li> You can also run a larger number of events: | <li> You can also run a larger number of events: | ||
Line 376: | Line 225: | ||
mu2e -c HelloWorld/test/hello.fcl -n 20 | mu2e -c HelloWorld/test/hello.fcl -n 20 | ||
</pre> | </pre> | ||
<li> Edit the file HelloWorld/src/HelloWorld_module.cc and change the printout. Then rebuild it: | <li> Edit the file Offline/HelloWorld/src/HelloWorld_module.cc and change the printout. Then rebuild it: | ||
<pre> | <pre> | ||
muse build | |||
mu2e -c HelloWorld/test/hello.fcl | mu2e -c Offline/HelloWorld/test/hello.fcl | ||
</pre> | </pre> | ||
Observe that the printout made by the module has changed. | Observe that the printout made by the module has changed. | ||
Line 386: | Line 235: | ||
<li> Make a copy of HelloWorld/src/HelloWorld_module.cc | <li> Make a copy of HelloWorld/src/HelloWorld_module.cc | ||
<pre> | <pre> | ||
cp HelloWorld/src/HelloWorld_module.cc HelloWorld/src/Bonjour_module.cc | cp Offline/HelloWorld/src/HelloWorld_module.cc Offline/HelloWorld/src/Bonjour_module.cc | ||
cp HelloWorld/test/hello.fcl HelloWorld/test/bonjour.fcl | cp Offline/HelloWorld/test/hello.fcl Offline/HelloWorld/test/bonjour.fcl | ||
</pre> | </pre> | ||
<li> | <li> | ||
Line 401: | Line 250: | ||
<li> Build and run the new module | <li> Build and run the new module | ||
<pre> | <pre> | ||
muse build | |||
mu2e -c HelloWorld/test/bonjour.fcl | mu2e -c Offline/HelloWorld/test/bonjour.fcl | ||
</pre> | </pre> | ||
Observe the changed printout. If you have questions about why you did not also change the module label, you can re-read the wiki page on [[Modules]]. | Observe the changed printout. If you have questions about why you did not also change the module label, you can re-read the wiki page on [[Modules]]. | ||
Line 408: | Line 257: | ||
<li> Make a new .fcl file that runs both modules in one job: | <li> Make a new .fcl file that runs both modules in one job: | ||
<pre> | <pre> | ||
cp HelloWorld/test/hello.fcl HelloWorld/test/both.fcl | cp Offline/HelloWorld/test/hello.fcl Offline/HelloWorld/test/both.fcl | ||
</pre> | </pre> | ||
Edit both.fcl so it looks like: | Edit both.fcl so it looks like: | ||
Line 429: | Line 278: | ||
</pre> | </pre> | ||
Note that you do not need to rebuild the libraries because you did not change the source code. | Note that you do not need to rebuild the libraries because you did not change the source code. | ||
<li> Rebuild the code even though there is nothing to do. You should see the following; | <li> Rebuild the code (muse build) even though there is nothing to do. You should see the following; | ||
<pre> | <pre> | ||
scons: Reading SConscript files ... | scons: Reading SConscript files ... | ||
Line 439: | Line 288: | ||
<li> In both.fcl, swap the order of "hi" and "hello" in the definition of "e1". Rerun. Notice that the order of the printout does not change! ''art'' reserves the right to run analyzer modules in an arbitrary order and we should never write code that depends on a particular order. ''art'' does respect user specified order for producer and filter modules. | <li> In both.fcl, swap the order of "hi" and "hello" in the definition of "e1". Rerun. Notice that the order of the printout does not change! ''art'' reserves the right to run analyzer modules in an arbitrary order and we should never write code that depends on a particular order. ''art'' does respect user specified order for producer and filter modules. | ||
</ol> | </ol> | ||
You can also tell muse to make a tarball of your build area: | |||
muse tarball | |||
this is useful for when you want to run on the grid to generate or process large datasets using your the code in your build area. | |||
[[Category:Computing]] | [[Category:Computing]] | ||
[[Category:Tutorial]] | [[Category:Tutorial]] |
Latest revision as of 21:26, 19 July 2024
Introduction
In the case of Mu2e, an event records the detector response during one microbunch, consisting of a short burst of millions of protons hitting the production target, thousands of muons stopping in the aluminum stopping target, and those muons interacting or decaying. This happens nominally every 1695 ns and each of these burst forms a Mu2e event.
When we read and process events to simulate the detector, reconstruct raw data, or analyze data events we will need a framework to organize our executables, and way to control the behavior of the executable, and a format to write the events into. These functions are provided by the art software package.
When we want to operate on the data, we write c++ code. We track code source files using a code management tool called git, which writes files to a commercial site, github. We have a main repository (or repo) called Offline and other smaller special-purpose repos. We compile and link the code using a set of Mu2e scripts called Muse. Inside those scripts, we use a code build system called scons. Finally, inside scons, we use linux gcc to compile and link the code.
Art
The art package provides:
- a framework for creating executables
- the event format (we define the contents)
- a control language called "fhicl" or "fcl" (pronounced fickle)
- services such as random numbers
When we write code, typically to simulate the detector, reconstruct the data, or analyze the data, we write our code in c++ modules. We then run the framework, and tell it to call the code in our module. We also tell it what input files to use, if any. The framework opens the input files, reads data, priovides services, calls our modules with the data, event by event, and then handles any filtering of events, and writing output.
When we run executables in the art framework, we have to tell the framework which modules to run and how to configure the modules. A module configuration might include how to find the input data inside the file, and what parameters and cuts to use in performing its task. This function is provided by a fcl file - you always provide a fcl file when you start up an art executable.
art executables read and write files in the art format. This is a highly-structured format which contains the event data, processing history, and other information. Under the covers, art uses the root package (used widely in HEP) to read and write the event files. Each event contains pieces which are referred to as products. For example, in each event, there may be a product which contains raw hits, one that contains reconstructed hits, and one that contains reconstructed tracks. In simulation, there are products with truth information.
All of these concepts will be fleshed out in subsequent tutorials.
Code
Please be sure your environment is prepared by consulting CodeEnvironment.
We will start with a tour of the Mu2e code base. We keep our code organized using a piece of software called git. This freeware package stores our code in a compact format and keep track of lots of useful things:
- all the history of each file as it is developed
- coherent versions of the whole set of code (tags)
- tracking parallel development by many people, and merging all that work back together when ready.
So lets look at what is stored in our git repository. If you are on the interactive central servers (mu2egpvm or mu2ebuild)
mkdir -p /mu2e/app/users/$USER/tutorial_code cd /mu2e/app/users/$USER/tutorial_code mu2einit
Since we will be doing several builds, create some subdirectories:
mkdir off cd off
Retrieve the code base from the git repository:
git clone https://github.com/Mu2e/Offline
after a few seconds you should see a directory Offline
. Go into that dir
cd Offline
and look around. Note the directory .git
. This is where git keeps information such code history and tags. Most of the directories you see here contain our working code. Each directory contains a relatively small, logically-grouped set of code.
Here are few git commands to get you started.
git status
tells you which branch you are on. When multiple people are working on code development, they will probably be working on their own branches - a version of the code they can control. When the changes are tested and approved, the personal, working branch is merged back into the main code branch, called "main". See the active branches:
git branch -al
You can switch to a different active working branch:
git checkout MDC2018 git status
or to a fixed release:
git checkout v09_12_00
all that output text is saying this fixed point of the code and you shouldn't be trying to modify it, which is fine, makes sense.
git status
see the files which are different from the previous release:
git diff --numstat v09_11_00 | head
and then see the code that changed in that first file listed:
git diff v09_11_00 Analyses/src/CaloCalib_module.cc
and then the file history:
git log v09_11_00 Analyses/src/CaloCalib_module.cc
In the git status you should also see "working branch clean" this means your area is still the same as the central repository you just cloned. If there were new or changed files, it would tell you. You can read much more about git and how we use it, including making your own branches and committing code modifications back to the central repo.
Let's look at the code structure. For example, under
ls -l TrkHitReco
you can see directories: fcl, inc, src, test and data. Most of the top-level directories follow this pattern. The c++ includes related to the code are kept under inc, the c++ source code in src, the fcl scripts that configure modules are in fcl, and sometimes scripts are under test or text files used by the code are kept under data. Many directories don't need test or data - see the TrkReco directory for example. Take a look at what in these directories.
Recall that we write modules, which are then run by the framework to act on the event data. Any module we write is compiled into a shared object library which is loaded by the framework if our fcl tells it to. For example,
TrkHitReco/src/StrawHitReco_module.cc
would be compiled into
../build/<some string>/Offline/lib/libmu2e_TrkHitReco_StrawHitReco_module.so
which we can then ask the framework to load and run (or not). The shared object is not there, because we haven't done the compiling, which we will get to in a minute.
Just as a quick peak, open the cc file:
emacs (or vi, cat) TrkHitReco/src/StrawHitReco_module.cc
Find the line
class StrawHitReco : public art::EDProducer
Since the module code inherits from a framework base class (EDproducer), the framework can manipulate it.
Find the line
StrawHitReco::beginJob
this method is called once at the beginning of the job. Similarly
StrawHitReco::beginRun
is called when the input events change run number.
StrawHitReco::produce
is called for every event. It is called "produce" because it typically produces data to insert into the event, in this case, the reconstructed tracker hits. Other option are "filter" if this module can select a subset of events to write out, and "analyze" if the module is not writing any new data to the event.
Find the line
event.getValidHandle(_sdtoken)
this is retrieving the raw hits from the event. Find the line event.put(std::move(chCol)); This adds a newly created data product to the event. All of these concepts will be covered in more detail later.
Now cd back out of the Offline directory
cd..
Configuring and Building Code
You should now be in the directory tutorial_code/off, which contains the Offline subdirectory.
Code management can be complex and so tools are developed to do most of the work, and simplify the user's commands. The art package, scons, Muse, and gcc are all provided in a system written and maintained by the lab computing division, in a system called UPS. At this point you have already told the UPS system to make certain packages and tools available in your path. For example, if you ask UPS to show you what it has added to your interactive process:
ups active
and you should get an answer similar to this
Active ups products: git v2_30_1 -f Linux64bit+3.10-2.17 -z /cvmfs/mu2e.opensciencegrid.org/artexternals ups v6_0_8 -f Linux64bit+3.10-2.17 -z /cvmfs/mu2e.opensciencegrid.org/artexternals muse v2_07_00 -f NULL -z /cvmfs/mu2e.opensciencegrid.org/artexternals
We will use UPS to get art tools and libraries, and much more, added to our paths, but first we need to discuss a few choices. We can build code for speed, we call this option "prof", or we can tell the compiler to keep the symbols organized, which runs slower, but is useful for debugging, so we call this second option "debug". There are other options for details of how to run simulations, or visualization. The build defaults to "prof", so we will not use any switches, and ask Muse to to use UPS to add art to our paths:
muse setup
Note that "setup muse" is causing UPS to add muse tools to our path, but "muse setup" is asking Muse scripts to add art and everything else to your paths. To prepare for a debug build we would have used "muse setup -q debug". You can only run "muse setup" once in a process so to change anything, you have to start new process and setup with your alternative choices. Try these commands to see where you are now:
ups active muse status muse -v status
The basic configuration of the package's versions and build options is now done.
In a typical use pattern, you would modify some of the code, or add code, then you would want to compile it so you can run it. This is the job of the build system. In our case, we continue to use the set of scripts called Muse, which call a package named scons, which in turn calls gcc to compile code. The build system has to look over the code, decide what needs to be compiled, do the compilation according to a recipe, then see what needs to be linked and do that linking. The first time you run a build, it will compile and link everything. After that, it should only do the steps which are effected by how you changed or added to the code.
Now it is finally time to build:
muse build -j 4
the "-j 4" starts the build running with 4 threads, which is appropriate if you are logged on a laptop or a mu2egpvm machine. For mu2ebuild02, where you would usually build code, you can use "-j 32" since it has many more cores.
Once it starts compiling, you might as well kill it with ctrl-c, since it will take long time (4 min on mu2ebuild02) to run.
If you log off and want to return to work here, you would
mu2einit cd mu2e/app/users/$USER/tutorial_code/off muse setup
and you are ready to build and run. cd'ing to the working directory first is important, so Muse knows this is where your code is.
Releases
At appropriate times, we mark the code with a tag such as "v09_12_00". This marks a state of the code we want to save. Maybe we are recording the state at a point where we make a production run for simulation, or maybe we are marking major changes in the code. Take look at the release list on github.
After tagging, we build the code and save it so anyone can use it, without building it themselves, - this is a "release".
Starting a new process (not the one from the last exercise) make a new test area:
mkdir -p /mu2e/app/users/$USER/tutorial_code/two cd mu2e/app/users/$USER/tutorial_code/two mu2einit
Now list what releases are available
muse backing
You will see two sets of Offline builds, one labeled "published releases" and one labeled "CI builds". We will use the former. You should see some release versions. To select a version, and link it to this area:
muse backing Offline v10_05_00
you will now see a subdirectory created, with a link to the tagged release code.
Now to up the game a bit, lets add a small repo to this area:
git clone https://github.com/Mu2e/Tutorial
This repo is some example code, but in practice you can add repos providing ntuples, other tools, or analysis. Now that this code directory is organized, we can setup and build it.
muse setup muse build -j 4
If successful, you see that a subdirectory "build" appeared. You can explore this director to find object files and shared-object libraries. You path will include these build products, so you could run them.
Partial Builds
When we started building the code in the exercise above, we killed it because it would take too long. On a mu2egpvm node it will take an 45 min, and on mu2ebuild02 with its 32 cores, it will take 4 min. (Note that these times are for a full build from scratch - any following incremental builds are usually much faster.) These long times are a common problem, so we have developed a way to build part of the code.
We need a fully built "base release" that already exists, such as the ones we just looked at. We then make a local directory with just a small piece of the code. We build this small piece, and then take all the rest of the compiled code from the "base release".
Warning In this method you must be aware of a big possible pitfall. If any header file that is compiled into your local small piece of code is different than the same header file compiled into the base release, then the code will have corrupt memory and is likely to fail horribly in unpredictable ways. It is best to use this partial build technique only when building a small part of the code base, and only changing cc files. It works great to do simple things like add a debugging print statement, or test a new module, or modify some fcl. If you have to modify header files, or do extensive code development, it is best to clone the whole Offline repo.
We will need to change the build directory organization, so we will have to start a new process.
mkdir -p /mu2e/app/users/$USER/tutorial_code/part cd mu2e/app/users/$USER/tutorial_code/part mu2einit muse link
Now organize the code in this area. First, take a look at the output of "muse link". Recall there were two sections, "Recent published Offline releases" which we used in the previous exercise, and the second list, the "Recent Offline CI builds". "CI" stand for continuous integration, which means every time the Offline repo changes, we build it and add it to this list. So you have a complete build of the most recent Offline code at your fingertips. You'll see the top line is the most recent, now cut and past that name to tell muse to use it as a backing build:
muse link master/be905d45
Of course the latest you see will be different from this example. The hex code is a "git hash" and labels one exact state of the Offline repo. Now we have indicated to Muse to use this build for Offline libraries.
Now we will introduce a small piece of Offline code. This piece of code is compiled and provided in the backing build we just linked, but if we also build it locally, Muse will prefer to use the local version, and take all the rest of Offline from the backing build.
To start the local subset of Offline code:
mgit init
You should see an "Offline" directory appear. Now lets add the HellowWorld subdirectory from Offline to this local area
cd Offline mgit add HelloWorld ls -l cd ..
You should see that the HelloWorld subdirectory of Offline has been added locally. Now that the code is organized, tell Muse to finish setting up the paths:
muse setup
and build
muse build -j 4
Exploring The Partial Build
Here are some suggested exercises.
- Look at the source code for the module HelloWorld/src/HelloWorld_module.cc . This file has a lot of structure that you can read about on the wiki page for Modules. The "payload" of the module is the two lines that printout the EventID for each event:
cerr << "Hello, world with changes. From analyze: " << event.id() << endl;
The Mu2e wiki has a short writeup about EventIDs.
- Run the HelloWorld module:
mu2e -c Offline/HelloWorld/test/hello.fcl
Identify the printout from the module:
Hello, world. From analyze: run: 1 subRun: 0 event: 1 Hello, world. From analyze: run: 1 subRun: 0 event: 2 Hello, world. From analyze: run: 1 subRun: 0 event: 3
This will also print some warning messages, which will be fixed.
The next tutorial has a lot of information about using fcl files so this tutorial will not discuss them much.
- You can also run a larger number of events:
mu2e -c HelloWorld/test/hello.fcl -n 20
- Edit the file Offline/HelloWorld/src/HelloWorld_module.cc and change the printout. Then rebuild it:
muse build mu2e -c Offline/HelloWorld/test/hello.fcl
Observe that the printout made by the module has changed.
- Make a new module by making a copy of a module and modifying the copy:
- Make a copy of HelloWorld/src/HelloWorld_module.cc
cp Offline/HelloWorld/src/HelloWorld_module.cc Offline/HelloWorld/src/Bonjour_module.cc cp Offline/HelloWorld/test/hello.fcl Offline/HelloWorld/test/bonjour.fcl
- Edit the new .cc file and make a global change of HelloWorld to Bonjour. Change the printout again so that you can distinguish this module from the original.
- Edit the .fcl file to replace
module_type : HelloWorld
with
module_type : Bonjour
- Build and run the new module
muse build mu2e -c Offline/HelloWorld/test/bonjour.fcl
Observe the changed printout. If you have questions about why you did not also change the module label, you can re-read the wiki page on Modules.
- Make a copy of HelloWorld/src/HelloWorld_module.cc
- Make a new .fcl file that runs both modules in one job:
cp Offline/HelloWorld/test/hello.fcl Offline/HelloWorld/test/both.fcl
Edit both.fcl so it looks like:
analyzers: { hello: { module_type : HelloWorld } hi : { module_type : Bonjour } } p1 : [ ] e1 : [hi, hello]
Now run both modules in one job:
mu2e -c HelloWorld/test/both.fcl
Note that you do not need to rebuild the libraries because you did not change the source code.
- Rebuild the code (muse build) even though there is nothing to do. You should see the following;
scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... scons: `.' is up to date. scons: done building targets.
- In both.fcl, swap the order of "hi" and "hello" in the definition of "e1". Rerun. Notice that the order of the printout does not change! art reserves the right to run analyzer modules in an arbitrary order and we should never write code that depends on a particular order. art does respect user specified order for producer and filter modules.
You can also tell muse to make a tarball of your build area:
muse tarball
this is useful for when you want to run on the grid to generate or process large datasets using your the code in your build area.