Module Writing Tutorial: Difference between revisions
(53 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
== Tutorial Session Goal == | == Tutorial Session Goal == | ||
This tutorial will show how to write an art module for Mu2e. It will explain how to structure the code, | This tutorial will show how to write an art module for Mu2e. It will explain how to structure the code, | ||
define runtime parameters, produce histograms | define runtime parameters, produce histograms, and consume and produce data products. | ||
== Session Prerequisites and Advance Preparation == | == Session Prerequisites and Advance Preparation == | ||
Line 8: | Line 8: | ||
* Perform the [[ Running_Art_Tutorial | tutorial on running Mu2e art Framework jobs ]] | * Perform the [[ Running_Art_Tutorial | tutorial on running Mu2e art Framework jobs ]] | ||
* Install the following docker containers () | * Install the following docker containers () | ||
This tutorial also assumes that you are somewhat familiar with C++. | |||
== Session Introduction == | == Session Introduction == | ||
Mu2e uses the art framework to organize our event processing code. The art framework processes events through a | Mu2e uses the art framework to organize our event processing code. The art framework processes events through a | ||
configurable sequential set of modules, called a path. | configurable sequential set of modules, called a path. A module expects certain inputs, and optionally produces | ||
certain outputs. In this tutorial you will learn how to create an art module from a basic template, and perform | certain outputs. Modules have a standard interface for interacting with the art framework, giving the user access | ||
to data at run/subrun/event transitions. Modules have a standard library for defining configuration parameters that | |||
can be changed at runtime. | |||
In this tutorial you will learn how to create an art module from a basic template, and perform | |||
basic data operations in that module. You will learn how to configure your module in code and fcl, and how to produce | basic data operations in that module. You will learn how to configure your module in code and fcl, and how to produce | ||
various kinds of output. | various kinds of output. | ||
== Exercises == | '''Disclaimer!''' In this tutorial we will be copying modules to create new modules. In general, this is not a great idea because this is how bugs can propagate. If you do this in the real world make sure you trust the code you're copying! | ||
The | |||
== Basic Exercises == | |||
* | === Setup === | ||
* | For these exercises, we will create a Satellite release. | ||
* | |||
In the '''docker''' container, we assume that you have a volume mounted on your machine to /home/working_area/ in the container (i.e something like <code>docker run --rm -it -v /path/to/your/machine/working_area:/home/working_area -v $XSOCK:$XSOCK -e DISPLAY=unix$DISPLAY mu2e/user:tutorial_1-00</code> where the -v option is important) | |||
<nowiki>> source /Tutorials_2019/setup_container.sh | |||
* | > cp -r $TUTORIAL_BASE/ModuleWriting /home/working_area/ | ||
== | > cd /home/working_area/ModuleWriting | ||
> /Offline/v7_4_1/SLF6/prof/Offline/bin/createSatelliteRelease --directory . | |||
> source setup.sh</nowiki> | |||
* | We also want to create some file lists: | ||
<nowiki>> mkdir filelists | |||
> ls $TUTORIAL_BASE/data/mcs.mu2e.CeEndpoint-mix.*.art > filelists/mcs.mu2e.CeEndpoint-mix.MDC2018h.lst</nowiki> | |||
If you are running this tutorial on one of the '''mu2egpvm''' machines, then the setup instructions will be different. These will be added at a later date. | |||
=== Exercise 1: Running a simple module (Hello, Tutorial!) === | |||
In this exercise, we will run a simple module that will print a welcoming message. | |||
<ol style="list-style-type:lower-alpha"> | |||
<li>The first module has already been written for you so we can compile straight away in the Satellite release</li> | |||
<nowiki>> scons -j4</nowiki> | |||
<li>And run</li> | |||
<nowiki>mu2e -c Ex01/fcl/HelloEx01.fcl</nowiki> | |||
This will write "Hello, world" and the full event id for the first 3 events. | |||
<li>Here is a brief overview of the contents of <code>Ex01/src/HelloTutorial_module.cc</code>:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>we keep everything in a namespace for mu2e:</li> | |||
<nowiki>namespace mu2e {</nowiki> | |||
<li>the <code>HelloTutorial</code> module is class that inherits from <code>art::EDAnalyzer</code>:</li> | |||
<nowiki>class Ex01HelloTutorial : public art::EDAnalyzer {</nowiki> | |||
<li>we create a structure that will handle the module configuration parameters (see Exercise 2)</li> | |||
<nowiki>struct Config { | |||
using Name=fhicl::Name; | |||
using Comment=fhicl::Comment; | |||
}; | |||
typedef art::EDAnalyzer::Table<Config> Parameters;</nowiki> | |||
<li>there is an <code>analyze()</code> function (this is called for every <code>art::Event</code>)</li> | |||
<nowiki>void analyze(const art::Event& event);</nowiki> | |||
</ol> | |||
<li>You will also see that there is an <code>SConscript</code> file in <code>Ex01/src/</code>. This tells SCons which libraries need to be linked to this package.</li> | |||
</ol> | |||
That's it for running your first module! This will be the basis of the rest of the exercises in this tutorial. | |||
=== Exercise 2: Adding module configuration (Hello, Fhicl Validation!) === | |||
In this exercise, we will add some module configuration parameters that can be set in fcl | |||
<ol style="list-style-type:lower-alpha"> | |||
<li>Create a new directory for this exercise called <code>Ex02/</code> and add <code>src/</code> and <code>fcl/</code> subdirectories. Also copy the <code>SConscript</code> file to <code>src/</code>.</li> | |||
<li>Copy last exercise's .cc file to create a new file</li> | |||
<nowiki>> cp Ex01/src/HelloTutorial_module.cc Ex02/src/HelloFhiclValidation_module.cc</nowiki> | |||
<li>Open the new file in your favourite text editor and make the following changes</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>find and replace <code>HelloTutorial</code> with <code>HelloFhiclValidation</code></li> | |||
<li>in the <code>Config</code> struct, below the <code>using</code> commands, add the following line to declare a fhicl parameter</li> | |||
<nowiki>fhicl::Atom<int> number{Name("number"), Comment("This module will print this number")};</nowiki> | |||
<li>add a private member variable to the class that will store the value of this parameter</li> | |||
<nowiki>int _number;</nowiki> | |||
<li>fill this member variable in the constructor initialization list:</li> | |||
<nowiki>_number(conf().number())</nowiki> | |||
<li>change the <code>std::cout</code> command to print this number</li> | |||
<nowiki>std::cout << "My _number is..." << _number << std::endl;</nowiki> | |||
</ol> | |||
<li>Recompile with <code>scons -j4</code> and fix any compilation errors</li> | |||
<li>Now create a copy of the fcl file, open it in a text editor and make the following changes:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>find and replace <code>HelloTutorial</code> with <code>HelloFhiclValidation</code></li> | |||
<li>add the new fhicl parameter to the module configuration and assign it a value</li> | |||
<nowiki>hello: { | |||
module_type : HelloFhiclValidation | |||
number : 5 | |||
}</nowiki> | |||
</ol> | |||
<li>Now everything's ready to run <code>mu2e</code> so do that</li> | |||
<nowiki> mu2e -c Ex02/fcl/HelloEx02.fcl </nowiki> | |||
<li>Try doing the following and note the errors you see:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>run without the new parameter in the fcl file</li> | |||
<li>run with a typo in the parameter name</li> | |||
<li>run with a value of the parameter that is float or a string</li> | |||
</ol> | |||
This is why fhicl validation is nice. We can catch errors in the module configurations before we waste time running <code>mu2e</code>. In the past, this wasn't the case -- you could run a job, thinking that you have changed a parameter, only to find that you had a typo in your fcl. Some already-existing modules still need to be updated and will be slowly converted to use fhicl validation. | |||
<li>We can let the parameter have a default value by adding this value to end of the parameter declaration:</li> | |||
<nowiki>fhicl::Atom<int> defaultNumber{Name("defaultNumber"), Comment("This module will print this number"), 100};</nowiki> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>Add another member variable for this parameter, fill it in the constructor initialization and add another <code>cout</code> statement to print this value</li> | |||
<li>Recompile and run without changing the fcl file</li> | |||
<li>Now add the <code>defaultNumber</code> parameter to your fcl file and play with different values</li> | |||
</ol> | |||
(Note that it is often better to have default values in fcl prolog, rather than hard-coded into the module itself) | |||
<li>There are some occasions where we want a parameter to be optional. This can be achieved by declaring the parameter like so:</li> | |||
<nowiki>fhicl::OptionalAtom<int> optionalNumber{Name("optionalNumber"), Comment("This module will print this number but it is optional")};</nowiki> | |||
However, we can't initialise this in the initializer list since it might not exist. Instead we have to check it exists in the <code>analyze()</code> function: | |||
<nowiki>if (_conf.optionalNumber(_optionalNumber)) { | |||
std::cout << "My _optionalNumber is..." << _optionalNumber << std::endl; | |||
}</nowiki> | |||
<li>You will need to <code>#include</code> "fhiclcpp/types/OptionalAtom.h"</li> | |||
<li>Play with using and not using the optional parameter in your code</li> | |||
</ol> | |||
You've just added fhicl validation to your module so that a user can change the configuration without having to recompile all the code! | |||
=== Exercise 3: Reading in a mu2e data product (Hello, KalSeed!) === | |||
In this exercise, we will be reading the results of the track fit and printing their times to the terminal. | |||
<ol style="list-style-type:lower-alpha"> | |||
<li>Create a new <code>Ex03/</code> directory with <code>src/</code> and <code>fcl/</code> subdirectories. Also copy the <code>SConscript</code> file to <code>src/</code>.</li> | |||
<li>Copy the Exercise 1's .cc file to create a new module and make the following changes</li> | |||
<nowiki> cp Ex01/src/HelloTutorial_module.cc Ex03/src/HelloKalSeed_module.cc</nowiki> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>find and replace <code>HelloTutorial</code> for <code>HelloKalSeed</code> | |||
<li><code>#include</code> RecoDataProducts/inc/KalSeed.hh</code></li> | |||
<li>add a fhicl parameter of type <code>art::InputTag</code> and a member variable to store the value</li> | |||
<li>fill the member variable and print its value in the constructor</li> | |||
<li>also in the constructor, it is good practice to tell art the products that we will use:</li> | |||
<nowiki>consumes<KalSeedCollection>(_input);</nowiki> | |||
</ol> | |||
<li>Copy a previous fcl file and edit it so that runs your new module and uses the new parameter (at the moment, it doesn't matter what the value of this parameter is).</li> | |||
<li>Compile, run and make sure everything works as you expect</li> | |||
<nowiki> mu2e -c Ex03/fcl/HelloEx03.fcl </nowiki> | |||
<li>Now we want to read in an art file. In this step, we will just change the fcl file so that it can start from an art file input</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>in the <code>source</code> block of your fcl file, change <code>EmptyEvent</code> to <code>RootInput</code></li> | |||
<li>run your fcl file on this file list to make sure everything works</li> | |||
<nowiki> mu2e -c Ex03/fcl/HelloEx03.fcl -S filelists/mcs.mu2e.CeEndpoint-mix.MDC2018h.lst</nowiki> | |||
</ol> | |||
<li>Now let's actually do something with a Mu2e data product. We will be looking at tracks (class <code>KalSeed</code>) from the downstream e-minus fit. Set the value of your fcl parameter to <code>KFFDeM</code> and then make the following changes in your module's <code>analyze()</code> function</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>get a valid handle to the <code>KalSeedCollection</code> from the event:</li> | |||
<nowiki>const auto& kalSeedCollectionHandle = event.getValidHandle<KalSeedCollection>(_input);</nowiki> | |||
<li>get the <code>KalSeedCollection</code> itself</li> | |||
<nowiki>const auto& kalSeedCollection = *kalSeedCollectionHandle;</nowiki> | |||
<li>loop through and print the t0 of each <code>KalSeed</code> (you can look in <code>$MU2E_BASE_RELEASE/RecoDataProducts/inc/KalSeed.hh</code> to work out why we need two <code>.t0()</code>s)</li> | |||
<nowiki>for (const auto& i_kalSeed : kalSeedCollection) { | |||
std::cout << "t0 = " << i_kalSeed.t0().t0() << " ns" << std::endl; | |||
}</nowiki> | |||
You will also need to <code>#include</code> the file <code>RecoDataProducts/inc/KalSeed.hh</code> | |||
</ol> | |||
<li>Recompile and run <code>mu2e</code></li> | |||
</ol> | |||
You have just read an already existing Mu2e data product from a Mu2e art file! Here are some optional exercises to explore this topic further: | |||
<ol style="list-style-type:lower-alpha"> | |||
<li>(Optional): try to print the times of a different <code>KalSeedCollection</code> (e.g. <code>KFFDmuM</code>)</li> | |||
<li>(Optional): try to print some other values from the <code>KalSeed</code> (e.g. momentum (which can be found in <code>KalSegment</code>))</li> | |||
<li>(Optional): try to read a different type of Mu2e data product (you can find the list of data products in an event with <code>mu2e -c $MU2E_BASE_RELEASE/Print/fcl/dumpDataProducts.fcl -S filelist -n 1</code>) | |||
</ol> | |||
=== Exercise 4: Filling a histogram (Hello, Histogram!) === | |||
In this exercise, we will create a ROOT histogram of the times that we were printing out to the screen in Exercise 3. We will need to use art's <code>TFileService</code> to create and write ROOT objects to a ROOT file. | |||
<ol style="list-style-type:lower-alpha"> | |||
<li>Create a new <code>Ex04/</code> directory with <code>src/</code> and <code>fcl/</code> subdirectories. Also copy the <code>SConscript</code> file to <code>src/</code>.</li> | |||
<li>Copy Exercise 3's module and make the following changes:</li> | |||
<nowiki>cp Ex03/src/HelloKalSeed_module.cc Ex04/src/HelloHistogram_module.cc</nowiki> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>find and replace <code>HelloKalSeed</code> for <code>HelloHistogram</code> | |||
<li><code>#include</code> art/Framework/Services/Optional/TFileService.h and TH1F.h</li> | |||
<li>add a new private member variable of type <code>TH1F*</code></li> | |||
<li>add a new function to your module: <code>void beginJob()</code> (this is a standard art module function that will run at the beginning of your job)</li> | |||
<li>in the <code>beginJob()</code> function add:</li> | |||
<nowiki>art::ServiceHandle<art::TFileService> tfs; | |||
double min_t0 = 0; | |||
double max_t0 = 1700; | |||
double t0_width = 10; | |||
int n_t0_bins = (max_t0 - min_t0) / t0_width; | |||
_hTrackTimes = tfs->make<TH1F>("hTrackTimes","Track t0", n_t0_bins,min_t0,max_t0);</nowiki> | |||
<li>in the <code>analyze()</code> function of your module, fill the histogram with the track times:</li> | |||
<nowiki>_hTrackTimes->Fill(i_kalSeed.t0().t0());</nowiki> | |||
</ol> | |||
<li>Copy a previous fcl file and change it to make sure it runs the new module (this should just be a change to the <code>module_type</code>) | |||
<li>Compile and run with the following command:</li> | |||
<nowiki>mu2e -c Ex04/fcl/HelloEx04.fcl -S filelists/mcs.mu2e.CeEndpoint-mix.MDC2018h.lst --TFileName out.root</nowiki> | |||
Note that, instead of defining the output ROOT filename on the command line. You can set the fcl parameter <code>services.TFileService.fileName</code> in your fcl file | |||
<li>Open the <code>out.root</code> file and explore it with a <code>TBrowser</code> to find your histogram!</li> | |||
</ol> | |||
Now that you can create a histogram, you can try some of these optional exercises | |||
<ol style="list-style-type:lower-alpha"> | |||
<li>(Optional): make the histogram parameters (min, max, bin width) fcl parameters</li> | |||
<li>(Optional): create a second instance of your module and read in a different <code>KalSeedCollection</code> (e.g. <code>KFFDmuM</code>)</li> | |||
<li>(Optional): make the plot informative by adding axis labels</li> | |||
<li>(Optional): create a TTree to store the track times</li> | |||
</ol> | |||
=== Exercise 5: Creating a subset of a data product (Hello, Cool KalSeed!) === | |||
In this exercise, we will find some "cool" <code>KalSeed</code>s that arrive fashionably late by cutting on the track times and put them into their own collection. We will reuse the <code>HelloHistogram</code> module to plot these "cool" <code>KalSeed</code>s. | |||
<ol style="list-style-type:lower-alpha"> | |||
<li>Create a new <code>Ex05/</code> directory with <code>src/</code> and <code>fcl/</code> subdirectories. Also copy the <code>SConscript</code> file to <code>src/</code>.</li> | |||
<li>Copy the module from Exercise 4</li> | |||
<nowiki>cp Ex04/src/HelloHistogram_module.cc Ex04/src/HelloCoolKalSeed_module.cc</nowiki> | |||
<li>The new module will be a producer, rather than an analyzer, so do a find and replace for the following in <code>HelloCoolKalSeed_module.cc</code></li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li><code>EDAnalyzer</code> to <code>EDProducer</code></li> | |||
<li><code>analyze</code> to <code>produce</code></li> | |||
<li><code>const art::Event&</code> to <code>art::Event&</code> (because producer modules are able to modify the event)</li> | |||
<li>remove the call to <code>EDProducer</code> in the constructor initializer list</li> | |||
</ol> | |||
<li>Also remove the histogram making parts of this module</li> | |||
<li>Before we do any cutting, let's make sure we have a working fcl</li> | |||
<li>Copy the fcl file from Exercise 4 and make the following changes:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>add a new <code>producer</code> block in the <code>physics</code> block like so:</li> | |||
<nowiki>producers : { | |||
cool : { | |||
module_type : HelloCoolKalSeed | |||
input : KFFDeM | |||
} | |||
}</nowiki> | |||
<li>add <code>cool</code> to the <code>p1</code> path</li> | |||
</ol> | |||
<li>Compile and run your new fcl file. There should be no difference in the output with the previous exercise</li> | |||
<li>Now that we have the basis of our job, we can start making changes.</li> | |||
<li>In the <code>HelloCoolKalSeed</code> module, make the following changes:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>create a new fhicl parameter that will take a float value for a time cut</li> | |||
<li>in the body of the construtor, we need to tell art that the module will be producing a new <code>KalSeedCollection</code></li> | |||
<nowiki>produces<KalSeedCollection>();</nowiki> | |||
<li>in the <code>produce()</code> function, we need to:</li> | |||
* create a new <code>KalSeedCollection</code> to fill (actually we create a pointer)</li> | |||
<nowiki>std::unique_ptr<KalSeedCollection> outputKalSeeds(new KalSeedCollection());</nowiki> | |||
* fill the new <code>KalSeedCollection</code> | |||
<nowiki>if (i_kalSeed.t0().t0() > _cut) { | |||
outputKalSeeds->push_back(i_kalSeed); | |||
}</nowiki> | |||
* write the new <code>KalSeedCollection</code> to the <code>art::Event</code> once we're done | |||
<nowiki>event.put(std::move(outputKalSeeds));</nowiki> | |||
</ol> | |||
<li>In your fcl file</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>change the input tag of the histogramming module to <code>cool</code></li> | |||
</ol> | |||
<li>Recompile and run, and you should see that the histogram is cut at your cut value!</li> | |||
</ol> | |||
Now that you have created a subset of an already existing collection, you can try some of these optional exercises: | |||
<ol style="list-style-type:lower-alpha"> | |||
<li>(Optional): change the cut value in fcl and make sure the output is as you expect</li> | |||
<li>(Optional): add some cuts on different variables</li> | |||
<li>(Optional): make some of the cuts optional</li> | |||
</ol> | |||
=== Exercise 6: Writing an art file (Hello, My Art File!) === | |||
In this exercise, we will be splitting the previous exercise into two different jobs to show how to write an art file. There will be no changes to the source code since is all done in fcl. | |||
<ol style="list-style-type:lower-alpha"> | |||
<li>Create a new <code>Ex06/</code> directory with only a <code>fcl/</code> subdirectories.</li> | |||
<li>Create two copies of your Exercise 5 fcl file. One will be for reading and one will be for writing.</li> | |||
<nowiki>cp Ex05/fcl/HelloEx05.fcl Ex06/fcl/HelloEx06_read.fcl | |||
cp Ex05/fcl/HelloEx05.fcl Ex06/fcl/HelloEx06_write.fcl</nowiki> | |||
<li>In your reading fcl, delete everything related to your <code>producer</code> module</li> | |||
<li>In your writing fcl, delete everything related to your <code>analyzer</code> module</li> | |||
<li>Also in your writing fcl:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>add an <code>output</code> block (note, this is at the same level as the <code>physics</code> block):</li> | |||
<nowiki>outputs : { | |||
MyOutput : { | |||
module_type: RootOutput | |||
SelectEvents : [p1] | |||
fileName : "my-art-file.art" | |||
outputCommands : [ "drop *_*_*_*", | |||
"keep *_cool_*_*" | |||
] | |||
} | |||
}</nowiki> | |||
The output commands have the following format <code>ObjectType_ModuleLabel_InstanceName_ProcessName</code> | |||
<li>add <code>MyOutput</code> to <code>e1</code></li> | |||
</ol> | |||
<li>Run your writing fcl and then check that the resulting .art file only contains your <code>KalSeeds</code>:</li> | |||
<nowiki>mu2e -c $MU2E_BASE_RELEASE/Print/fcl/dumpDataProducts.fcl -s my-art-file.art</nowiki> | |||
<li>Now run your reading fcl on the art file you just created and you should have a new ROOT file with your time histogram in!</li> | |||
</ol> | |||
Now that you can write an art file, try some of the following optional exercises: | |||
<ol style="list-style-type:lower-alpha"> | |||
<li>(Optional): write out some other data products along with your cool <code>KalSeeds</code></li> | |||
</ol> | |||
=== Exercise 7: Filtering events we don't want (Hello, Filter!) === | |||
You might notice that the number of events in your output art file is the same as in the input, even though we are only writing out the cool KalSeeds and we are cutting on those. This is because we are still write an <code>art::Event</code> record even if it's empty. In this exercise, we will add a <code>filter</code> module to remove events that we expect to be empty. | |||
<ol style="list-style-type:lower-alpha"> | |||
<li>Create a new <code>Ex07/</code> directory with <code>src/</code> and <code>fcl/</code> subdirectories. Also copy the <code>SConscript</code> file to <code>src/</code>.</li> | |||
<li>Copy your previous producer module and make the following changes:</li> | |||
<nowiki>cp Ex05/src/HelloCoolKalSeed_module.cc Ex07/src/HelloFilter_module.cc</nowiki> | |||
<ol style="list-style-type:lower-roman"> | |||
<li><code>HelloCoolKalSeed</code> to <code>HelloFilter</code></li> | |||
<li><code>EDProducer</code> to <code>EDFilter</code></li> | |||
<li><code>produce</code> to <code>filter</code></li> | |||
<li>the <code>void</code> return-type of <code>filter()</code> to <code>bool</code> (because we will return to art, true or false whether the event passes the filter)</li> | |||
<li>delete anything related to cutting or producing a new <code>KalSeedCollection</code> (you want to keep the input <code>KalSeedCollection</code>)</li> | |||
</ol> | |||
<li>This module will run over our "cool" <code>KalSeedCollection</code> and tell art whether there are any tracks in the collection or not. Add some logic to the <code>filter()</code> function to do this.</li> | |||
<li>Now, let's set up the fcl file to write out only the events that pass this filter. Create copies of your Exercise 6 fcls and make the following changes to the writing one:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>add a <code>filters</code> block to the <code>physics</code> block:</li> | |||
<nowiki>filters : { | |||
helloFilter : { | |||
module_type : HelloFilter | |||
input : "cool" | |||
} | |||
}</nowiki> | |||
<li>add "helloFilter" to the path <code>p1</code></li> | |||
</ol> | |||
<li>Compile and run your new writing fcl file and you will see that the number of events in the output file has fallen</li> | |||
<li>You should be able to run your the reading fcl on your new art file and the output histogram should be the same</li> | |||
</ol> | |||
=== Exercise 8: Creating a new data product (Hello, My Data Product!) === | |||
Let's say we want to create a data product that just contains the track t0. This is an oversimplified example but it will illustrate all the issues. | |||
<ol style="list-style-type:lower-alpha"> | |||
<li>Create a new <code>Ex08/</code> directory with <code>src/</code>, <code>fcl/</code>, and <code>inc/</code> subdirectories. Also copy the <code>SConscript</code> file to <code>src/</code>.</li> | |||
<li>Create a new class in a header file in <code>inc/</code>:</li> | |||
<nowiki>#ifndef ModuleWriting_TrackTime_hh | |||
#define ModuleWriting_TrackTime_hh | |||
#include "RecoDataProducts/inc/KalSeed.hh" | |||
namespace mu2e { | |||
class TrackTime { | |||
public: | |||
TrackTime() : _time(0.0) {} | |||
TrackTime(const KalSeed& kseed) { | |||
_time = kseed.t0().t0(); | |||
} | |||
const double time() const { return _time; } | |||
private: | |||
double _time; | |||
}; | |||
typedef std::vector<TrackTime> TrackTimeCollection; | |||
} | |||
#endif | |||
</nowiki> | |||
<li>To be able to write this data product out to an art file we need to add two files into the src/ directory:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>classes_def.xml</li> | |||
<nowiki><lcgdict> | |||
<class name="mu2e::TrackTime"/> | |||
<class name="art::Wrapper<mu2e::TrackTime>"/> | |||
<class name="std::vector<mu2e::TrackTime>"/> | |||
<class name="art::Wrapper<std::vector<mu2e::TrackTime> >"/> | |||
</lcgdict></nowiki> | |||
The <code>art::Wrapper</code> lines are because we will write this data product to an art file. | |||
<li>classes.h</li> | |||
<nowiki>// | |||
// headers needed when genreflex creates root dictionaries | |||
// for objects written to art files | |||
// | |||
#define ENABLE_MU2E_GENREFLEX_HACKS | |||
#include <vector> | |||
#include "canvas/Persistency/Common/Wrapper.h" | |||
#include "Ex08/inc/TrackTime.hh" | |||
#undef ENABLE_MU2E_GENREFLEX_HACKS</nowiki> | |||
</ol> | |||
This is everything we need to create a new data product that we can write to an art file. | |||
<li>Now let's copy some old modules to create new modules that will:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>create a <code>TrackTimeCollection</code> from a <code>KalSeedCollection</code>;</li> | |||
<code>cp Ex05/src/HelloCoolKalSeed_module.cc Ex08/src/HelloTrackTime_module.cc</code> | |||
<li>create a subset of a <code>TrackTimeCollection</code>;</li> | |||
<code>cp Ex05/src/HelloCoolKalSeed_module.cc Ex08/src/HelloCoolTrackTime_module.cc</code> | |||
<li>filter events based on the size of a <code>TrackTimeCollection</code>; and</li> | |||
<code>cp Ex07/src/HelloFilter_module.cc Ex08/src/FilterTrackTime_module.cc</code> | |||
<li>create a histogram using <code>TrackTime</code>s</li> | |||
<code>cp Ex04/src/HelloHistogram_module.cc Ex08/src/HistogramTrackTime_module.cc</code> | |||
</ol> | |||
<li>In <code>HelloTrackTime</code>:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>remove anything to do with cutting</li> | |||
<li>write out a <code>TrackTimeCollection</code> rather than a <code>KalSeedCollection</code></li> | |||
<li>add a check that the input KalSeedCollection and the output TrackTime collection are the same size. Throw an exception if they are not (<code>throw cet::exception("Tutorial") << "message";</code>)</li> | |||
</ol> | |||
<li>In <code>HelloCoolTrackTime</code>:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>replace <code>KalSeed</code> with <code>TrackTime</code></li> | |||
<li>make sure to use the correct method of <code>TrackTime</code> to get the time</li> | |||
</ol> | |||
<li>In <code>FilterTrackTime</code>:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>replace <code>KalSeed</code> with <code>TrackTime</code></li> | |||
</ol> | |||
<li>In <code>HistogramTrackTime</code>:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>replace <code>KalSeed</code> with <code>TrackTime</code></li> | |||
<li>make sure to use the correct method of <code>TrackTime</code> to get the time</li> | |||
</ol> | |||
<li>Create copies of your writing and reading fcls from the Exercise 7 and</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>update the modules in both fcl files to use your new modules that use TrackTime (should just be changes to each <code>module_type</code>)</li> | |||
<li>add a configuration of your new producer module e.g.</li> | |||
<nowiki>trackTime : { | |||
module_type : HelloTrackTimeDataProducts | |||
input : "KFFDeM" | |||
}</nowiki> | |||
<li>add the new producer to the start of the path <code>p1</code> | |||
<li>change the input of your cutting module to use the new producer</li> | |||
</ol> | |||
<li>Recompile and run your writing module</li> | |||
<li>Use <code>$MU2E_BASE_RELEASE/Print/fcl/dumpDataProducts.fcl</code> to check that you only have <code>TrackTimeCollections</code> in your art file</li> | |||
<li>Run your reading fcl on your new art file and check that your histogram looks as expected</li> | |||
</ol> | |||
You've just created your own data product and written it to an art file! | |||
=== Exercise 9: Creating and using a pointer to a data product (Hello, Art Ptrs!) === | |||
In this exercise, we will show one way to link the <code>TrackTime</code> object to the original <code>KalSeed</code> by using an <code>art::Ptr</code>. | |||
<ol style="list-style-type:lower-alpha"> | |||
<li>Create a new <code>Ex09/</code> directory with <code>src/</code>, <code>fcl/</code> and <code>inc/</code> subdirectories. Also copy the <code>SConscript</code> file to <code>src/</code>.</li> | |||
<li>Copy your TrackTime.hh file to create a new TrackTimePtr.hh and make the following changes:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>find-and-replace "TrackTime" with "TrackTimePtr"</li> | |||
<li>add a new private member variable of type <code>art::Ptr<KalSeed></code></li> | |||
<li>add a getter function (<code>const art::Ptr<KalSeed> kalSeed() const</code>) and a setter function (<code>void setKalSeedPtr(const art::Ptr<KalSeed>& ptr)</code>)</li> | |||
</ol> | |||
<li>Copy the classes.h and classes_def.xml from Exercise 8 and do a find-and-replace of "TrackTime" with "TrackTimePtr"</li> | |||
<li>Copy your modules from Exercise 8, rename them and do a find-and-replace of "TrackTime" with "TrackTimePtr"</li> | |||
<li>Copy your fcl files from Exercise 8 and do a find-and-replace of "TrackTime" with "TrackTimePtr"</li> | |||
<li>Run your new fcl and check that things work as they did before</li> | |||
<li>In order to create an <code>art::Ptr</code>, we need to have a handle to the collection and an index into the collection. In your <code>TrackTime</code> creating module, we need to make the following changes:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>move to an index based for loop (i.e. we want something like this now):</li> | |||
<nowiki>for (auto i_kalSeed = kalSeedCollection.begin(); i_kalSeed != kalSeedCollection.end(); ++i_kalSeed) {</nowiki> | |||
<li>create an <code>art::Ptr</code> and pass it to the <code>TrackTime</code> object</li> | |||
<nowiki>TrackTime track_time(*i_kalSeed); | |||
art::Ptr<KalSeed> kseedPtr(kalSeedCollectionHandle, i_kalSeed-kalSeedCollection.begin()); | |||
track_time.setKalSeedPtr(kseedPtr);</nowiki> | |||
</ol> | |||
<li>Recompile and re-run to make sure things work as before</li> | |||
<li>In your histogram module, add a second histogram that plots the number of hits in the KalSeed (<code>kseed.hits().size()</code>) through the <code>Ptr</code> interface</li> | |||
<li>To your writing fcl script add the following to your output commands : <code>"keep mu2e::KalSeeds_*_*_*"</code>. This will make sure the KalSeeds are written out into your art file.</li> | |||
<li>Recompile and re-run to check that your new histogram is there.</li> | |||
</ol> | |||
You've just created and used an <code>art::Ptr</code>! There are a few limitations that you might want to be aware of. These are covered in the following optional exercises: | |||
<ol style="list-style-type:lower-alpha"> | |||
<li>(Optional): Try running without the added <code>keep</code> command at the end of the exercise. This will show that the object you want to access must be in the same file as the one you are reading.</li> | |||
<li>(Optional): Add a module that produces a reduced <code>KalSeedCollection</code> with a different cut (like in Exercise 5) and only write out the reduced <code>KalSeedCollection</code> to your art file. Play with putting this module before and after the module that creates your <code>TrackTimePtrs</code> (you will also have to update input tags). You will find that creating the reduced <code>KalSeedCollection</code> '''after''' creating the <code>TrackTimePtrs</code> will result in errors when trying to use the <code>art::Ptr</code>. This is because the <code>TrackTimePtrs</code> are pointing to the full <code>KalSeedCollection</code> which you are not writing out. This is an issue if we want to update a collection (e.g. for compression) after creating <code>art::Ptrs</code> to that collection. In that case we have to go through all of our data products and create brand new objects that contain <code>art::Ptrs</code> to the collection and update them to the new collection.</li> | |||
</ol> | |||
=== Exercise 10: Creating and using an assn between two data products (Hello, Art Assns!) === | |||
In this exercise, instead of linking together the <code>TrackTime</code> and <code>KalSeed</code> objects with an <code>art::Ptr</code>, we will use an <code>art::Assns</code>. An <code>art::Assns</code> has an advantage over an <code>art::Ptr</code> in that it is bidirectional (i.e. you can go from <code>TrackTime</code> to <code>KalSeed</code> and vice versa). However, it comes at a cost of being a little more complicated to use. | |||
In this exercise, you will also see how to create an <code>art::Ptr</code> when the object you want to point to is being created in the same module. | |||
<ol style="list-style-type:lower-alpha"> | |||
<li>Create a new <code>Ex10/</code> directory with <code>src/</code>, <code>fcl/</code> and <code>inc/</code> subdirectories. Also copy the <code>SConscript</code> file to <code>src/</code>.</li> | |||
<li>First, let's typedef the Assns we will create. In a new header file:</li> | |||
<nowiki>#ifndef ModuleWriting_OurAssns_hh_ | |||
#define ModuleWriting_OurAssns_hh_ | |||
#include "canvas/Persistency/Common/Assns.h" | |||
#include "RecoDataProducts/inc/KalSeed.hh" | |||
#include "Ex08/inc/TrackTime.hh" | |||
namespace mu2e { | |||
typedef art::Assns<TrackTime, KalSeed> OurAssns; | |||
} | |||
#endif</nowiki> | |||
and add <code>mu2e::OurAssns</code> and <code>art::Wrapper<mu2e::OurAssns></code> to a new classes_def.xml. You will also need to add the lines: | |||
<nowiki><class name="art::Assns<mu2e::TrackTime, mu2e::KalSeed, void>"/> | |||
<class name="art::Assns<mu2e::KalSeed, mu2e::TrackTime, void>"/> | |||
<class name="art::Wrapper<art::Assns<mu2e::TrackTime, mu2e::KalSeed, void> >"/> | |||
<class name="art::Wrapper<art::Assns<mu2e::KalSeed, mu2e::TrackTime, void> >"/></nowiki> | |||
<li>We will create the <code>art::Assns</code> at the same time we create the <code>TrackTime</code> object, so copy your <code>HelloTrackTime_module.cc</code> from Exercise 8 and call it something different. Make the following changes:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>add a <code>produces<OurAssns>();</code> statement</li> | |||
<li>create a new <code>art::Assns</code> to be written to the art event: <code>std::unique_ptr<OurAssns> outputAssns(new OurAssns());</code></li> | |||
<li>put the <code>art::Assns</code> into the event: <code>event.put(std::move(outputAssns));</code></li> | |||
</ol> | |||
<li>At the moment, we are not filling the <code>art::Assns</code>. To do this we need to create <code>art::Ptr</code>s to the <code>KalSeed</code> and the <code>TrackTime</code></li> | |||
<li>We created an <code>art::Ptr<KalSeed></code> in Exercise 9 so you can repeat the step from that exercise</li> | |||
<li>For the <code>art::Ptr<TrackTime></code> it's a bit tricker because we haven't added the <code>TrackTimeCollection</code> to the event yet and so we can't get a handle to it. We need to do the following isntead:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>get the <code>art::ProductID</code> and the <code>ProductGetter</code> for the <code>TrackTimeCollection</code> at the start of the <code>produce()</code> function:</li> | |||
<nowiki>auto TrackTimeProdID = getProductID<TrackTimeCollection>(); | |||
auto TrackTimeGetter = event.productGetter(TrackTimeProdID);</nowiki> | |||
<li>create the <code>art::Ptr<TrackTime></code> after adding it to the output <code>TrackTimeCollection</code>:</li> | |||
<nowiki>art::Ptr<TrackTime> trackTimePtr(TrackTimeProdID, outpuTrackTimes->size()-1, TrackTimeGetter);</nowiki> | |||
</ol> | |||
<li>Finally, we add a new entry to the <code>art::Assns</code> like so:</li> | |||
<nowiki>outputAssns->addSingle(trackTimePtr, kseedPtr);</nowiki> | |||
<li>Copy an old writing fcl file and edit it to use the new Assns-creating module (be sure to add the <code>art::Assns</code> to the <code>outputCommands</code>!)</li> | |||
<li>Run the writing fcl and make sure that the <code>art::Assns</code> is in the output art file as well as all of your <code>TrackTimes</code> and <code>KalSeeds</code></li> | |||
<li>Now let's use the <code>art::Assns</code> by creating a 2D histogram of track time vs the number of hits in the track</li> | |||
<li>Copy one of your previous histogram module and make the following changes:</li> | |||
<ol style="list-style-type:lower-roman"> | |||
<li>make a 2D histogram rather than a 1D histogram</li> | |||
<li>grab the <code>art::Assns</code> from the event like so:</li> | |||
<nowiki>const auto& ourAssnsHandle = event.getValidHandle<OurAssns>(_input); | |||
const auto& ourAssns = *ourAssnsHandle;</nowiki> | |||
<li>loop through the Assns and fill the histogram like this:</li> | |||
<nowiki>for (const auto& i_entry : ourAssns) { | |||
const auto& ttimePtr = i_entry.first; | |||
const auto& kseedPtr = i_entry.second; | |||
_hOurAssns->Fill(ttimePtr->time(), kseedPtr->hits().size()); | |||
}</nowiki> | |||
</ol> | |||
<li>Create a new reading fcl and run the new histogram-making module. Look in the output ROOT file and you should see the 2D histogram!</li> | |||
</ol> | |||
That's it for making and using an <code>art::Assns</code>! | |||
== Reference Materials == | == Reference Materials == | ||
* | |||
* [https://art.fnal.gov/art-workbook/ art workbook] | |||
[[Category:Computing]] | |||
[[Category:Tutorial]] | |||
[[Category:Code]] |
Latest revision as of 22:02, 24 June 2019
Tutorial Session Goal
This tutorial will show how to write an art module for Mu2e. It will explain how to structure the code, define runtime parameters, produce histograms, and consume and produce data products.
Session Prerequisites and Advance Preparation
This tutorial requires the user to:
- Perform the Tutorial on setting up the Mu2e Offline
- Perform the tutorial on running Mu2e art Framework jobs
- Install the following docker containers ()
This tutorial also assumes that you are somewhat familiar with C++.
Session Introduction
Mu2e uses the art framework to organize our event processing code. The art framework processes events through a configurable sequential set of modules, called a path. A module expects certain inputs, and optionally produces certain outputs. Modules have a standard interface for interacting with the art framework, giving the user access to data at run/subrun/event transitions. Modules have a standard library for defining configuration parameters that can be changed at runtime.
In this tutorial you will learn how to create an art module from a basic template, and perform basic data operations in that module. You will learn how to configure your module in code and fcl, and how to produce various kinds of output.
Disclaimer! In this tutorial we will be copying modules to create new modules. In general, this is not a great idea because this is how bugs can propagate. If you do this in the real world make sure you trust the code you're copying!
Basic Exercises
Setup
For these exercises, we will create a Satellite release.
In the docker container, we assume that you have a volume mounted on your machine to /home/working_area/ in the container (i.e something like docker run --rm -it -v /path/to/your/machine/working_area:/home/working_area -v $XSOCK:$XSOCK -e DISPLAY=unix$DISPLAY mu2e/user:tutorial_1-00
where the -v option is important)
> source /Tutorials_2019/setup_container.sh > cp -r $TUTORIAL_BASE/ModuleWriting /home/working_area/ > cd /home/working_area/ModuleWriting > /Offline/v7_4_1/SLF6/prof/Offline/bin/createSatelliteRelease --directory . > source setup.sh
We also want to create some file lists:
> mkdir filelists > ls $TUTORIAL_BASE/data/mcs.mu2e.CeEndpoint-mix.*.art > filelists/mcs.mu2e.CeEndpoint-mix.MDC2018h.lst
If you are running this tutorial on one of the mu2egpvm machines, then the setup instructions will be different. These will be added at a later date.
Exercise 1: Running a simple module (Hello, Tutorial!)
In this exercise, we will run a simple module that will print a welcoming message.
- The first module has already been written for you so we can compile straight away in the Satellite release > scons -j4
- And run mu2e -c Ex01/fcl/HelloEx01.fcl This will write "Hello, world" and the full event id for the first 3 events.
- Here is a brief overview of the contents of
Ex01/src/HelloTutorial_module.cc
: - we keep everything in a namespace for mu2e: namespace mu2e {
- the
HelloTutorial
module is class that inherits fromart::EDAnalyzer
:
class Ex01HelloTutorial : public art::EDAnalyzer {
- we create a structure that will handle the module configuration parameters (see Exercise 2) struct Config { using Name=fhicl::Name; using Comment=fhicl::Comment; }; typedef art::EDAnalyzer::Table<Config> Parameters;
- there is an
analyze()
function (this is called for everyart::Event
)
void analyze(const art::Event& event);
- You will also see that there is an
SConscript
file inEx01/src/
. This tells SCons which libraries need to be linked to this package.
That's it for running your first module! This will be the basis of the rest of the exercises in this tutorial.
Exercise 2: Adding module configuration (Hello, Fhicl Validation!)
In this exercise, we will add some module configuration parameters that can be set in fcl
- Create a new directory for this exercise called
Ex02/
and addsrc/
andfcl/
subdirectories. Also copy theSConscript
file tosrc/
. - Copy last exercise's .cc file to create a new file > cp Ex01/src/HelloTutorial_module.cc Ex02/src/HelloFhiclValidation_module.cc
- Open the new file in your favourite text editor and make the following changes
- find and replace
HelloTutorial
withHelloFhiclValidation
- in the
Config
struct, below theusing
commands, add the following line to declare a fhicl parameter
fhicl::Atom<int> number{Name("number"), Comment("This module will print this number")};
- add a private member variable to the class that will store the value of this parameter int _number;
- fill this member variable in the constructor initialization list: _number(conf().number())
- change the
std::cout
command to print this number
std::cout << "My _number is..." << _number << std::endl;
- Recompile with
scons -j4
and fix any compilation errors - Now create a copy of the fcl file, open it in a text editor and make the following changes:
- find and replace
HelloTutorial
withHelloFhiclValidation
- add the new fhicl parameter to the module configuration and assign it a value hello: { module_type : HelloFhiclValidation number : 5 }
- Now everything's ready to run
mu2e
so do that
mu2e -c Ex02/fcl/HelloEx02.fcl
- Try doing the following and note the errors you see:
- run without the new parameter in the fcl file
- run with a typo in the parameter name
- run with a value of the parameter that is float or a string
- We can let the parameter have a default value by adding this value to end of the parameter declaration: fhicl::Atom<int> defaultNumber{Name("defaultNumber"), Comment("This module will print this number"), 100};
- Add another member variable for this parameter, fill it in the constructor initialization and add another
cout
statement to print this value - Recompile and run without changing the fcl file
- Now add the
defaultNumber
parameter to your fcl file and play with different values - There are some occasions where we want a parameter to be optional. This can be achieved by declaring the parameter like so: fhicl::OptionalAtom<int> optionalNumber{Name("optionalNumber"), Comment("This module will print this number but it is optional")}; However, we can't initialise this in the initializer list since it might not exist. Instead we have to check it exists in the
- You will need to
#include
"fhiclcpp/types/OptionalAtom.h" - Play with using and not using the optional parameter in your code
This is why fhicl validation is nice. We can catch errors in the module configurations before we waste time running mu2e
. In the past, this wasn't the case -- you could run a job, thinking that you have changed a parameter, only to find that you had a typo in your fcl. Some already-existing modules still need to be updated and will be slowly converted to use fhicl validation.
(Note that it is often better to have default values in fcl prolog, rather than hard-coded into the module itself)
analyze()
function:
if (_conf.optionalNumber(_optionalNumber)) {
std::cout << "My _optionalNumber is..." << _optionalNumber << std::endl;
}
You've just added fhicl validation to your module so that a user can change the configuration without having to recompile all the code!
Exercise 3: Reading in a mu2e data product (Hello, KalSeed!)
In this exercise, we will be reading the results of the track fit and printing their times to the terminal.
- Create a new
Ex03/
directory withsrc/
andfcl/
subdirectories. Also copy theSConscript
file tosrc/
. - Copy the Exercise 1's .cc file to create a new module and make the following changes cp Ex01/src/HelloTutorial_module.cc Ex03/src/HelloKalSeed_module.cc
- find and replace
HelloTutorial
forHelloKalSeed
#include
RecoDataProducts/inc/KalSeed.hh- add a fhicl parameter of type
art::InputTag
and a member variable to store the value - fill the member variable and print its value in the constructor
- also in the constructor, it is good practice to tell art the products that we will use: consumes<KalSeedCollection>(_input);
- Copy a previous fcl file and edit it so that runs your new module and uses the new parameter (at the moment, it doesn't matter what the value of this parameter is).
- Compile, run and make sure everything works as you expect mu2e -c Ex03/fcl/HelloEx03.fcl
- Now we want to read in an art file. In this step, we will just change the fcl file so that it can start from an art file input
- in the
source
block of your fcl file, changeEmptyEvent
toRootInput
- run your fcl file on this file list to make sure everything works mu2e -c Ex03/fcl/HelloEx03.fcl -S filelists/mcs.mu2e.CeEndpoint-mix.MDC2018h.lst
- Now let's actually do something with a Mu2e data product. We will be looking at tracks (class
KalSeed
) from the downstream e-minus fit. Set the value of your fcl parameter toKFFDeM
and then make the following changes in your module'sanalyze()
function - get a valid handle to the
KalSeedCollection
from the event:
const auto& kalSeedCollectionHandle = event.getValidHandle<KalSeedCollection>(_input);
- get the
KalSeedCollection
itself
const auto& kalSeedCollection = *kalSeedCollectionHandle;
- loop through and print the t0 of each
KalSeed
(you can look in$MU2E_BASE_RELEASE/RecoDataProducts/inc/KalSeed.hh
to work out why we need two.t0()
s)
for (const auto& i_kalSeed : kalSeedCollection) {
std::cout << "t0 = " << i_kalSeed.t0().t0() << " ns" << std::endl;
}
You will also need to - Recompile and run
mu2e
#include
the file RecoDataProducts/inc/KalSeed.hh
You have just read an already existing Mu2e data product from a Mu2e art file! Here are some optional exercises to explore this topic further:
- (Optional): try to print the times of a different
KalSeedCollection
(e.g.KFFDmuM
) - (Optional): try to print some other values from the
KalSeed
(e.g. momentum (which can be found inKalSegment
)) - (Optional): try to read a different type of Mu2e data product (you can find the list of data products in an event with
mu2e -c $MU2E_BASE_RELEASE/Print/fcl/dumpDataProducts.fcl -S filelist -n 1
)
Exercise 4: Filling a histogram (Hello, Histogram!)
In this exercise, we will create a ROOT histogram of the times that we were printing out to the screen in Exercise 3. We will need to use art's TFileService
to create and write ROOT objects to a ROOT file.
- Create a new
Ex04/
directory withsrc/
andfcl/
subdirectories. Also copy theSConscript
file tosrc/
. - Copy Exercise 3's module and make the following changes: cp Ex03/src/HelloKalSeed_module.cc Ex04/src/HelloHistogram_module.cc
- find and replace
HelloKalSeed
forHelloHistogram
#include
art/Framework/Services/Optional/TFileService.h and TH1F.h- add a new private member variable of type
TH1F*
- add a new function to your module:
void beginJob()
(this is a standard art module function that will run at the beginning of your job) - in the
beginJob()
function add:
art::ServiceHandle<art::TFileService> tfs;
double min_t0 = 0;
double max_t0 = 1700;
double t0_width = 10;
int n_t0_bins = (max_t0 - min_t0) / t0_width;
_hTrackTimes = tfs->make<TH1F>("hTrackTimes","Track t0", n_t0_bins,min_t0,max_t0);
- in the
analyze()
function of your module, fill the histogram with the track times:
_hTrackTimes->Fill(i_kalSeed.t0().t0());
- Copy a previous fcl file and change it to make sure it runs the new module (this should just be a change to the
module_type
) - Compile and run with the following command: mu2e -c Ex04/fcl/HelloEx04.fcl -S filelists/mcs.mu2e.CeEndpoint-mix.MDC2018h.lst --TFileName out.root Note that, instead of defining the output ROOT filename on the command line. You can set the fcl parameter
- Open the
out.root
file and explore it with aTBrowser
to find your histogram!
services.TFileService.fileName
in your fcl file
Now that you can create a histogram, you can try some of these optional exercises
- (Optional): make the histogram parameters (min, max, bin width) fcl parameters
- (Optional): create a second instance of your module and read in a different
KalSeedCollection
(e.g.KFFDmuM
) - (Optional): make the plot informative by adding axis labels
- (Optional): create a TTree to store the track times
Exercise 5: Creating a subset of a data product (Hello, Cool KalSeed!)
In this exercise, we will find some "cool" KalSeed
s that arrive fashionably late by cutting on the track times and put them into their own collection. We will reuse the HelloHistogram
module to plot these "cool" KalSeed
s.
- Create a new
Ex05/
directory withsrc/
andfcl/
subdirectories. Also copy theSConscript
file tosrc/
. - Copy the module from Exercise 4 cp Ex04/src/HelloHistogram_module.cc Ex04/src/HelloCoolKalSeed_module.cc
- The new module will be a producer, rather than an analyzer, so do a find and replace for the following in
HelloCoolKalSeed_module.cc
EDAnalyzer
toEDProducer
analyze
toproduce
const art::Event&
toart::Event&
(because producer modules are able to modify the event)- remove the call to
EDProducer
in the constructor initializer list - Also remove the histogram making parts of this module
- Before we do any cutting, let's make sure we have a working fcl
- Copy the fcl file from Exercise 4 and make the following changes:
- add a new
producer
block in thephysics
block like so:
producers : {
cool : {
module_type : HelloCoolKalSeed
input : KFFDeM
}
}
- add
cool
to thep1
path - Compile and run your new fcl file. There should be no difference in the output with the previous exercise
- Now that we have the basis of our job, we can start making changes.
- In the
HelloCoolKalSeed
module, make the following changes: - create a new fhicl parameter that will take a float value for a time cut
- in the body of the construtor, we need to tell art that the module will be producing a new
KalSeedCollection
produces<KalSeedCollection>();
- in the
produce()
function, we need to: - create a new
KalSeedCollection
to fill (actually we create a pointer) - fill the new
KalSeedCollection
- write the new
KalSeedCollection
to theart::Event
once we're done - In your fcl file
- change the input tag of the histogramming module to
cool
- Recompile and run, and you should see that the histogram is cut at your cut value!
Now that you have created a subset of an already existing collection, you can try some of these optional exercises:
- (Optional): change the cut value in fcl and make sure the output is as you expect
- (Optional): add some cuts on different variables
- (Optional): make some of the cuts optional
Exercise 6: Writing an art file (Hello, My Art File!)
In this exercise, we will be splitting the previous exercise into two different jobs to show how to write an art file. There will be no changes to the source code since is all done in fcl.
- Create a new
Ex06/
directory with only afcl/
subdirectories. - Create two copies of your Exercise 5 fcl file. One will be for reading and one will be for writing. cp Ex05/fcl/HelloEx05.fcl Ex06/fcl/HelloEx06_read.fcl cp Ex05/fcl/HelloEx05.fcl Ex06/fcl/HelloEx06_write.fcl
- In your reading fcl, delete everything related to your
producer
module - In your writing fcl, delete everything related to your
analyzer
module - Also in your writing fcl:
- add an
output
block (note, this is at the same level as thephysics
block):
outputs : {
MyOutput : {
module_type: RootOutput
SelectEvents : [p1]
fileName : "my-art-file.art"
outputCommands : [ "drop *_*_*_*",
"keep *_cool_*_*"
]
}
}
The output commands have the following format - add
MyOutput
toe1
- Run your writing fcl and then check that the resulting .art file only contains your
KalSeeds
:
mu2e -c $MU2E_BASE_RELEASE/Print/fcl/dumpDataProducts.fcl -s my-art-file.art
- Now run your reading fcl on the art file you just created and you should have a new ROOT file with your time histogram in!
ObjectType_ModuleLabel_InstanceName_ProcessName
Now that you can write an art file, try some of the following optional exercises:
- (Optional): write out some other data products along with your cool
KalSeeds
Exercise 7: Filtering events we don't want (Hello, Filter!)
You might notice that the number of events in your output art file is the same as in the input, even though we are only writing out the cool KalSeeds and we are cutting on those. This is because we are still write an art::Event
record even if it's empty. In this exercise, we will add a filter
module to remove events that we expect to be empty.
- Create a new
Ex07/
directory withsrc/
andfcl/
subdirectories. Also copy theSConscript
file tosrc/
. - Copy your previous producer module and make the following changes: cp Ex05/src/HelloCoolKalSeed_module.cc Ex07/src/HelloFilter_module.cc
HelloCoolKalSeed
toHelloFilter
EDProducer
toEDFilter
produce
tofilter
- the
void
return-type offilter()
tobool
(because we will return to art, true or false whether the event passes the filter) - delete anything related to cutting or producing a new
KalSeedCollection
(you want to keep the inputKalSeedCollection
) - This module will run over our "cool"
KalSeedCollection
and tell art whether there are any tracks in the collection or not. Add some logic to thefilter()
function to do this. - Now, let's set up the fcl file to write out only the events that pass this filter. Create copies of your Exercise 6 fcls and make the following changes to the writing one:
- add a
filters
block to thephysics
block:
filters : {
helloFilter : {
module_type : HelloFilter
input : "cool"
}
}
- add "helloFilter" to the path
p1
- Compile and run your new writing fcl file and you will see that the number of events in the output file has fallen
- You should be able to run your the reading fcl on your new art file and the output histogram should be the same
Exercise 8: Creating a new data product (Hello, My Data Product!)
Let's say we want to create a data product that just contains the track t0. This is an oversimplified example but it will illustrate all the issues.
- Create a new
Ex08/
directory withsrc/
,fcl/
, andinc/
subdirectories. Also copy theSConscript
file tosrc/
. - Create a new class in a header file in
inc/
:
#ifndef ModuleWriting_TrackTime_hh
#define ModuleWriting_TrackTime_hh
#include "RecoDataProducts/inc/KalSeed.hh"
namespace mu2e {
class TrackTime {
public:
TrackTime() : _time(0.0) {}
TrackTime(const KalSeed& kseed) {
_time = kseed.t0().t0();
}
const double time() const { return _time; }
private:
double _time;
};
typedef std::vector<TrackTime> TrackTimeCollection;
}
#endif
- To be able to write this data product out to an art file we need to add two files into the src/ directory:
- classes_def.xml <lcgdict> <class name="mu2e::TrackTime"/> <class name="art::Wrapper<mu2e::TrackTime>"/> <class name="std::vector<mu2e::TrackTime>"/> <class name="art::Wrapper<std::vector<mu2e::TrackTime> >"/> </lcgdict> The
- classes.h // // headers needed when genreflex creates root dictionaries // for objects written to art files // #define ENABLE_MU2E_GENREFLEX_HACKS #include <vector> #include "canvas/Persistency/Common/Wrapper.h" #include "Ex08/inc/TrackTime.hh" #undef ENABLE_MU2E_GENREFLEX_HACKS
- Now let's copy some old modules to create new modules that will:
- create a
TrackTimeCollection
from aKalSeedCollection
; - create a subset of a
TrackTimeCollection
; - filter events based on the size of a
TrackTimeCollection
; and - create a histogram using
TrackTime
s - In
HelloTrackTime
: - remove anything to do with cutting
- write out a
TrackTimeCollection
rather than aKalSeedCollection
- add a check that the input KalSeedCollection and the output TrackTime collection are the same size. Throw an exception if they are not (
throw cet::exception("Tutorial") << "message";
) - In
HelloCoolTrackTime
: - replace
KalSeed
withTrackTime
- make sure to use the correct method of
TrackTime
to get the time - In
FilterTrackTime
: - replace
KalSeed
withTrackTime
- In
HistogramTrackTime
: - replace
KalSeed
withTrackTime
- make sure to use the correct method of
TrackTime
to get the time - Create copies of your writing and reading fcls from the Exercise 7 and
- update the modules in both fcl files to use your new modules that use TrackTime (should just be changes to each
module_type
) - add a configuration of your new producer module e.g. trackTime : { module_type : HelloTrackTimeDataProducts input : "KFFDeM" }
- add the new producer to the start of the path
p1
- change the input of your cutting module to use the new producer
- Recompile and run your writing module
- Use
$MU2E_BASE_RELEASE/Print/fcl/dumpDataProducts.fcl
to check that you only haveTrackTimeCollections
in your art file - Run your reading fcl on your new art file and check that your histogram looks as expected
art::Wrapper
lines are because we will write this data product to an art file.
This is everything we need to create a new data product that we can write to an art file.
cp Ex05/src/HelloCoolKalSeed_module.cc Ex08/src/HelloTrackTime_module.cc
cp Ex05/src/HelloCoolKalSeed_module.cc Ex08/src/HelloCoolTrackTime_module.cc
cp Ex07/src/HelloFilter_module.cc Ex08/src/FilterTrackTime_module.cc
cp Ex04/src/HelloHistogram_module.cc Ex08/src/HistogramTrackTime_module.cc
You've just created your own data product and written it to an art file!
Exercise 9: Creating and using a pointer to a data product (Hello, Art Ptrs!)
In this exercise, we will show one way to link the TrackTime
object to the original KalSeed
by using an art::Ptr
.
- Create a new
Ex09/
directory withsrc/
,fcl/
andinc/
subdirectories. Also copy theSConscript
file tosrc/
. - Copy your TrackTime.hh file to create a new TrackTimePtr.hh and make the following changes:
- find-and-replace "TrackTime" with "TrackTimePtr"
- add a new private member variable of type
art::Ptr<KalSeed>
- add a getter function (
const art::Ptr<KalSeed> kalSeed() const
) and a setter function (void setKalSeedPtr(const art::Ptr<KalSeed>& ptr)
) - Copy the classes.h and classes_def.xml from Exercise 8 and do a find-and-replace of "TrackTime" with "TrackTimePtr"
- Copy your modules from Exercise 8, rename them and do a find-and-replace of "TrackTime" with "TrackTimePtr"
- Copy your fcl files from Exercise 8 and do a find-and-replace of "TrackTime" with "TrackTimePtr"
- Run your new fcl and check that things work as they did before
- In order to create an
art::Ptr
, we need to have a handle to the collection and an index into the collection. In yourTrackTime
creating module, we need to make the following changes: - move to an index based for loop (i.e. we want something like this now): for (auto i_kalSeed = kalSeedCollection.begin(); i_kalSeed != kalSeedCollection.end(); ++i_kalSeed) {
- create an
art::Ptr
and pass it to theTrackTime
object
TrackTime track_time(*i_kalSeed);
art::Ptr<KalSeed> kseedPtr(kalSeedCollectionHandle, i_kalSeed-kalSeedCollection.begin());
track_time.setKalSeedPtr(kseedPtr);
- Recompile and re-run to make sure things work as before
- In your histogram module, add a second histogram that plots the number of hits in the KalSeed (
kseed.hits().size()
) through thePtr
interface - To your writing fcl script add the following to your output commands :
"keep mu2e::KalSeeds_*_*_*"
. This will make sure the KalSeeds are written out into your art file. - Recompile and re-run to check that your new histogram is there.
You've just created and used an art::Ptr
! There are a few limitations that you might want to be aware of. These are covered in the following optional exercises:
- (Optional): Try running without the added
keep
command at the end of the exercise. This will show that the object you want to access must be in the same file as the one you are reading. - (Optional): Add a module that produces a reduced
KalSeedCollection
with a different cut (like in Exercise 5) and only write out the reducedKalSeedCollection
to your art file. Play with putting this module before and after the module that creates yourTrackTimePtrs
(you will also have to update input tags). You will find that creating the reducedKalSeedCollection
after creating theTrackTimePtrs
will result in errors when trying to use theart::Ptr
. This is because theTrackTimePtrs
are pointing to the fullKalSeedCollection
which you are not writing out. This is an issue if we want to update a collection (e.g. for compression) after creatingart::Ptrs
to that collection. In that case we have to go through all of our data products and create brand new objects that containart::Ptrs
to the collection and update them to the new collection.
Exercise 10: Creating and using an assn between two data products (Hello, Art Assns!)
In this exercise, instead of linking together the TrackTime
and KalSeed
objects with an art::Ptr
, we will use an art::Assns
. An art::Assns
has an advantage over an art::Ptr
in that it is bidirectional (i.e. you can go from TrackTime
to KalSeed
and vice versa). However, it comes at a cost of being a little more complicated to use.
In this exercise, you will also see how to create an art::Ptr
when the object you want to point to is being created in the same module.
- Create a new
Ex10/
directory withsrc/
,fcl/
andinc/
subdirectories. Also copy theSConscript
file tosrc/
. - First, let's typedef the Assns we will create. In a new header file: #ifndef ModuleWriting_OurAssns_hh_ #define ModuleWriting_OurAssns_hh_ #include "canvas/Persistency/Common/Assns.h" #include "RecoDataProducts/inc/KalSeed.hh" #include "Ex08/inc/TrackTime.hh" namespace mu2e { typedef art::Assns<TrackTime, KalSeed> OurAssns; } #endif and add
- We will create the
art::Assns
at the same time we create theTrackTime
object, so copy yourHelloTrackTime_module.cc
from Exercise 8 and call it something different. Make the following changes: - add a
produces<OurAssns>();
statement - create a new
art::Assns
to be written to the art event:std::unique_ptr<OurAssns> outputAssns(new OurAssns());
- put the
art::Assns
into the event:event.put(std::move(outputAssns));
- At the moment, we are not filling the
art::Assns
. To do this we need to createart::Ptr
s to theKalSeed
and theTrackTime
- We created an
art::Ptr<KalSeed>
in Exercise 9 so you can repeat the step from that exercise - For the
art::Ptr<TrackTime>
it's a bit tricker because we haven't added theTrackTimeCollection
to the event yet and so we can't get a handle to it. We need to do the following isntead: - get the
art::ProductID
and theProductGetter
for theTrackTimeCollection
at the start of theproduce()
function:
auto TrackTimeProdID = getProductID<TrackTimeCollection>();
auto TrackTimeGetter = event.productGetter(TrackTimeProdID);
- create the
art::Ptr<TrackTime>
after adding it to the outputTrackTimeCollection
:
art::Ptr<TrackTime> trackTimePtr(TrackTimeProdID, outpuTrackTimes->size()-1, TrackTimeGetter);
- Finally, we add a new entry to the
art::Assns
like so:
outputAssns->addSingle(trackTimePtr, kseedPtr);
- Copy an old writing fcl file and edit it to use the new Assns-creating module (be sure to add the
art::Assns
to theoutputCommands
!) - Run the writing fcl and make sure that the
art::Assns
is in the output art file as well as all of yourTrackTimes
andKalSeeds
- Now let's use the
art::Assns
by creating a 2D histogram of track time vs the number of hits in the track - Copy one of your previous histogram module and make the following changes:
- make a 2D histogram rather than a 1D histogram
- grab the
art::Assns
from the event like so:
const auto& ourAssnsHandle = event.getValidHandle<OurAssns>(_input);
const auto& ourAssns = *ourAssnsHandle;
- loop through the Assns and fill the histogram like this: for (const auto& i_entry : ourAssns) { const auto& ttimePtr = i_entry.first; const auto& kseedPtr = i_entry.second; _hOurAssns->Fill(ttimePtr->time(), kseedPtr->hits().size()); }
- Create a new reading fcl and run the new histogram-making module. Look in the output ROOT file and you should see the 2D histogram!
mu2e::OurAssns
and art::Wrapper<mu2e::OurAssns>
to a new classes_def.xml. You will also need to add the lines:
<class name="art::Assns<mu2e::TrackTime, mu2e::KalSeed, void>"/>
<class name="art::Assns<mu2e::KalSeed, mu2e::TrackTime, void>"/>
<class name="art::Wrapper<art::Assns<mu2e::TrackTime, mu2e::KalSeed, void> >"/>
<class name="art::Wrapper<art::Assns<mu2e::KalSeed, mu2e::TrackTime, void> >"/>
That's it for making and using an art::Assns
!