Module Writing Tutorial: Difference between revisions

From Mu2eWiki
Jump to navigation Jump to search
 
(14 intermediate revisions by 2 users not shown)
Line 28: Line 28:
For these exercises, we will create a Satellite release.
For these exercises, we will create a Satellite release.


In the '''docker''':
In the '''docker''' container, we assume that you have a volume mounted on your machine to /home/working_area/ in the container (i.e something like <code>docker run --rm -it -v /path/to/your/machine/working_area:/home/working_area -v $XSOCK:$XSOCK -e DISPLAY=unix$DISPLAY mu2e/user:tutorial_1-00</code> where the -v option is important)
  <nowiki>> source /setupmu2e-art.sh
  <nowiki>> source /Tutorials_2019/setup_container.sh
> [source script that gives $TUTORIAL_BASE]
> cp -r $TUTORIAL_BASE/ModuleWriting /home/working_area/
> cd $TUTORIAL_BASE/TrkAna
> cd /home/working_area/ModuleWriting
> /Offline/v7_4_1/SLF6/prof/Offline/bin/createSatelliteRelease --directory .
> /Offline/v7_4_1/SLF6/prof/Offline/bin/createSatelliteRelease --directory .
> source setup.sh</nowiki>
> source setup.sh</nowiki>
Line 121: Line 121:
   std::cout << "My _optionalNumber is..." << _optionalNumber << std::endl;
   std::cout << "My _optionalNumber is..." << _optionalNumber << std::endl;
}</nowiki>
}</nowiki>
<li>You will need to <code>#include</code> "fhiclcpp/types/OptionalAtom.h"</li>
  <li>Play with using and not using the optional parameter in your code</li>
  <li>Play with using and not using the optional parameter in your code</li>
</ol>
</ol>
Line 135: Line 136:
  <ol style="list-style-type:lower-roman">
  <ol style="list-style-type:lower-roman">
   <li>find and replace <code>HelloTutorial</code> for <code>HelloKalSeed</code>
   <li>find and replace <code>HelloTutorial</code> for <code>HelloKalSeed</code>
  <li><code>#include</code> RecoDataProducts/inc/KalSeed.hh</code></li>
   <li>add a fhicl parameter of type <code>art::InputTag</code> and a member variable to store the value</li>
   <li>add a fhicl parameter of type <code>art::InputTag</code> and a member variable to store the value</li>
   <li>fill the member variable and print its value in the constructor</li>
   <li>fill the member variable and print its value in the constructor</li>
Line 269: Line 271:


=== Exercise 6: Writing an art file (Hello, My Art File!) ===
=== Exercise 6: Writing an art file (Hello, My Art File!) ===
In this exercise, we will be splitting the previous exercise into two different jobs to show how to write an art file. There will be no changes to the source code since this can all be done in fcl.
In this exercise, we will be splitting the previous exercise into two different jobs to show how to write an art file. There will be no changes to the source code since is all done in fcl.


<ol style="list-style-type:lower-alpha">
<ol style="list-style-type:lower-alpha">
  <li>Create a new <code>Ex06/</code> directory with only a <code>fcl/</code> subdirectories. Also copy the <code>SConscript</code> file to <code>src/</code>.</li>
  <li>Create a new <code>Ex06/</code> directory with only a <code>fcl/</code> subdirectories.</li>
  <li>Create two copies of your Exercise 5 fcl file. One will be for reading and one will be for writing.</li>
  <li>Create two copies of your Exercise 5 fcl file. One will be for reading and one will be for writing.</li>
  <nowiki>cp Ex05/fcl/HelloEx05.fcl Ex05/fcl/HelloEx05_read.fcl
  <nowiki>cp Ex05/fcl/HelloEx05.fcl Ex06/fcl/HelloEx06_read.fcl
cp Ex05/fcl/HelloEx05.fcl Ex05/fcl/HelloEx05_write.fcl</nowiki>
cp Ex05/fcl/HelloEx05.fcl Ex06/fcl/HelloEx06_write.fcl</nowiki>
  <li>In your reading fcl, delete everything related to your producer module</li>
  <li>In your reading fcl, delete everything related to your <code>producer</code> module</li>
  <li>In your writing fcl, delete everything related to your analyzer module</li>
  <li>In your writing fcl, delete everything related to your <code>analyzer</code> module</li>
  <li>Also in your writing fcl:</li>
  <li>Also in your writing fcl:</li>
  <ol style="list-style-type:lower-roman">
  <ol style="list-style-type:lower-roman">
Line 291: Line 293:
   }
   }
}</nowiki>
}</nowiki>
The output commands have the following format <code>ObjectType_ModuleLabel_InstanceName_ProcessName</code>
   <li>add <code>MyOutput</code> to <code>e1</code></li>
   <li>add <code>MyOutput</code> to <code>e1</code></li>
  </ol>
  </ol>
  <li>Run your writing fcl and then check that the resulting .art file only contains your KalSeeds:</li>
  <li>Run your writing fcl and then check that the resulting .art file only contains your <code>KalSeeds</code>:</li>
  <nowiki>mu2e -c $MU2E_BASE_RELEASE/Print/fcl/dumpDataProducts.fcl -s my-art-file.art</nowiki>
  <nowiki>mu2e -c $MU2E_BASE_RELEASE/Print/fcl/dumpDataProducts.fcl -s my-art-file.art</nowiki>
  <li>Now run your reading fcl on the art file you just created and you should have a new ROOT file with your time histogram in!</li>
  <li>Now run your reading fcl on the art file you just created and you should have a new ROOT file with your time histogram in!</li>
Line 300: Line 303:
Now that you can write an art file, try some of the following optional exercises:
Now that you can write an art file, try some of the following optional exercises:
<ol style="list-style-type:lower-alpha">
<ol style="list-style-type:lower-alpha">
  <li>(Optional): write out some other data products along with your cool KalSeeds</li>
  <li>(Optional): write out some other data products along with your cool <code>KalSeeds</code></li>
</ol>
</ol>


=== Exercise 7: Filtering events we don't want (Hello, Filter!) ===
=== Exercise 7: Filtering events we don't want (Hello, Filter!) ===
You might notice that the number of events in your output art file is the same as in the input, even though we are only writing out the cool KalSeeds and we are cutting on those. This is because we are still write art::Event even if it's empty. In this exercise, we will add a filter module to remove events that we expect to be empty.
You might notice that the number of events in your output art file is the same as in the input, even though we are only writing out the cool KalSeeds and we are cutting on those. This is because we are still write an <code>art::Event</code> record even if it's empty. In this exercise, we will add a <code>filter</code> module to remove events that we expect to be empty.


<ol style="list-style-type:lower-alpha">
<ol style="list-style-type:lower-alpha">
Line 314: Line 317:
   <li><code>EDProducer</code> to <code>EDFilter</code></li>
   <li><code>EDProducer</code> to <code>EDFilter</code></li>
   <li><code>produce</code> to <code>filter</code></li>
   <li><code>produce</code> to <code>filter</code></li>
   <li>the <code>void</code> return-type of <code>filter()</code> to <code>bool</code> (because we will return to art, true or false whether it passes the filter)</li>
   <li>the <code>void</code> return-type of <code>filter()</code> to <code>bool</code> (because we will return to art, true or false whether the event passes the filter)</li>
   <li>delete anything related to cutting or producing a new <code>KalSeedCollection</code> (you want to keep the input <code>KalSeedCollection</code>)</li>
   <li>delete anything related to cutting or producing a new <code>KalSeedCollection</code> (you want to keep the input <code>KalSeedCollection</code>)</li>
  </ol>
  </ol>
  <li>This module will run over our "cool" KalSeedCollection and tell art whether there are any tracks in the collection or not. Add some logic to the <code>filter()</code> function to do this.</li>
  <li>This module will run over our "cool" <code>KalSeedCollection</code> and tell art whether there are any tracks in the collection or not. Add some logic to the <code>filter()</code> function to do this.</li>
  <li>Now, let's set up the fcl file to write out only the events that pass this filter. Create a copies of your Exercise 6 fcls and make the following changes to the writing one:</li>
  <li>Now, let's set up the fcl file to write out only the events that pass this filter. Create copies of your Exercise 6 fcls and make the following changes to the writing one:</li>
  <ol style="list-style-type:lower-roman">
  <ol style="list-style-type:lower-roman">
   <li>add a <code>filters</code> block to the <code>physics</code> block:</li>
   <li>add a <code>filters</code> block to the <code>physics</code> block:</li>
Line 372: Line 375:
  <class name="art::Wrapper<std::vector<mu2e::TrackTime> >"/>
  <class name="art::Wrapper<std::vector<mu2e::TrackTime> >"/>
</lcgdict></nowiki>
</lcgdict></nowiki>
The <code>art::Wrapper</code> lines are because we will write this data product to an art file.
   <li>classes.h</li>
   <li>classes.h</li>
  <nowiki>//                                                                                                                                                                                       
  <nowiki>//                                                                                                                                                                                       
Line 397: Line 401:
   <li>create a histogram using <code>TrackTime</code>s</li>
   <li>create a histogram using <code>TrackTime</code>s</li>
  <code>cp Ex04/src/HelloHistogram_module.cc Ex08/src/HistogramTrackTime_module.cc</code>
  <code>cp Ex04/src/HelloHistogram_module.cc Ex08/src/HistogramTrackTime_module.cc</code>
</ol>
  <li>In <code>HelloTrackTime</code>:</li>
  <li>In <code>HelloTrackTime</code>:</li>
  <ol style="list-style-type:lower-roman">
  <ol style="list-style-type:lower-roman">
Line 419: Line 424:
  <li>Create copies of your writing and reading fcls from the Exercise 7 and</li>
  <li>Create copies of your writing and reading fcls from the Exercise 7 and</li>
  <ol style="list-style-type:lower-roman">
  <ol style="list-style-type:lower-roman">
   <li>update the modules in both fcl files to use your new modules that use TrackTime (should just be changes to each <code>module_type</code></li>
   <li>update the modules in both fcl files to use your new modules that use TrackTime (should just be changes to each <code>module_type</code>)</li>
   <li>add a configuration of your new producer module e.g.</li>
   <li>add a configuration of your new producer module e.g.</li>
  <nowiki>trackTime : {
  <nowiki>trackTime : {
Line 429: Line 434:
  </ol>
  </ol>
  <li>Recompile and run your writing module</li>
  <li>Recompile and run your writing module</li>
  <li>Use $MU2E_BASE_RELEASE/Print/fcl/dumpDataProducts.fcl to check that you only have TrackTimeCollections in your art file</li>
  <li>Use <code>$MU2E_BASE_RELEASE/Print/fcl/dumpDataProducts.fcl</code> to check that you only have <code>TrackTimeCollections</code> in your art file</li>
  <li>Run your reading fcl on your new art file and check that your histogram looks as expected</li>
  <li>Run your reading fcl on your new art file and check that your histogram looks as expected</li>
</ol>
</ol>
You've just created your own data product and written it to an art file!


=== Exercise 9: Creating and using a pointer to a data product (Hello, Art Ptrs!) ===
=== Exercise 9: Creating and using a pointer to a data product (Hello, Art Ptrs!) ===
In this exercise, we will show one way to link the TrackTime object to the original KalSeed by using an <code>art::Ptr</code>.
In this exercise, we will show one way to link the <code>TrackTime</code> object to the original <code>KalSeed</code> by using an <code>art::Ptr</code>.


<ol style="list-style-type:lower-alpha">
<ol style="list-style-type:lower-alpha">
Line 448: Line 455:
  <li>Copy your fcl files from Exercise 8 and do a find-and-replace of "TrackTime" with "TrackTimePtr"</li>
  <li>Copy your fcl files from Exercise 8 and do a find-and-replace of "TrackTime" with "TrackTimePtr"</li>
  <li>Run your new fcl and check that things work as they did before</li>
  <li>Run your new fcl and check that things work as they did before</li>
  <li>In order to create a Ptr, we need to have a handle to the collection and an index into the collection. In your TrackTime creating module, we need to make the following changes:</li>
  <li>In order to create an <code>art::Ptr</code>, we need to have a handle to the collection and an index into the collection. In your <code>TrackTime</code> creating module, we need to make the following changes:</li>
  <ol style="list-style-type:lower-roman">
  <ol style="list-style-type:lower-roman">
   <li>move to an index based for loop (i.e. we want something like this now):</li>
   <li>move to an index based for loop (i.e. we want something like this now):</li>
  <nowiki>for (auto i_kalSeed = kalSeedCollection.begin(); i_kalSeed != kalSeedCollection.end(); ++i_kalSeed) {</nowiki>
  <nowiki>for (auto i_kalSeed = kalSeedCollection.begin(); i_kalSeed != kalSeedCollection.end(); ++i_kalSeed) {</nowiki>
   <li>create an art::Ptr and set it in the TrackTime class</li>
   <li>create an <code>art::Ptr</code> and pass it to the <code>TrackTime</code> object</li>
  <nowiki>TrackTime track_time(*i_kalSeed);
  <nowiki>TrackTime track_time(*i_kalSeed);
art::Ptr<KalSeed> kseedPtr(kalSeedCollectionHandle, i_kalSeed-kalSeedCollection.begin());
art::Ptr<KalSeed> kseedPtr(kalSeedCollectionHandle, i_kalSeed-kalSeedCollection.begin());
Line 466: Line 473:
<ol style="list-style-type:lower-alpha">
<ol style="list-style-type:lower-alpha">
  <li>(Optional): Try running without the added <code>keep</code> command at the end of the exercise. This will show that the object you want to access must be in the same file as the one you are reading.</li>
  <li>(Optional): Try running without the added <code>keep</code> command at the end of the exercise. This will show that the object you want to access must be in the same file as the one you are reading.</li>
  <li>(Optional): Add a module that produces a reduced KalSeedCollection with a different cut (like in Exercise 5) and only write out the reduced KalSeedCollection. Play with putting it before and after the TrackTimePtr-creating module it before and after. You will find that putting it after will result in errors when trying to use the Ptr later. This is an issue if we want to update a collection (e.g. for compression), then we also have to update all collections that contain Ptrs to that collection.</li>
  <li>(Optional): Add a module that produces a reduced <code>KalSeedCollection</code> with a different cut (like in Exercise 5) and only write out the reduced <code>KalSeedCollection</code> to your art file. Play with putting this module before and after the module that creates your <code>TrackTimePtrs</code> (you will also have to update input tags). You will find that creating the reduced <code>KalSeedCollection</code> '''after''' creating the <code>TrackTimePtrs</code> will result in errors when trying to use the <code>art::Ptr</code>. This is because the <code>TrackTimePtrs</code> are pointing to the full <code>KalSeedCollection</code> which you are not writing out. This is an issue if we want to update a collection (e.g. for compression) after creating <code>art::Ptrs</code> to that collection. In that case we have to go through all of our data products and create brand new objects that contain <code>art::Ptrs</code> to the collection and update them to the new collection.</li>
</ol>
</ol>


=== Exercise 10: Creating and using an assn between two data products (Hello, Art Assns!) ===
=== Exercise 10: Creating and using an assn between two data products (Hello, Art Assns!) ===
In this exercise, instead of linking together the TrackTime and KalSeed objects with a Ptr, we will use an <code>art::Assns</code>. An Assns has an advantage over a Ptr in that it is bidirectional (i.e. you can go from TrackTime to KalSeed and vice versa). However, it comes at a cost of being a little more complicated to use.
In this exercise, instead of linking together the <code>TrackTime</code> and <code>KalSeed</code> objects with an <code>art::Ptr</code>, we will use an <code>art::Assns</code>. An <code>art::Assns</code> has an advantage over an <code>art::Ptr</code> in that it is bidirectional (i.e. you can go from <code>TrackTime</code> to <code>KalSeed</code> and vice versa). However, it comes at a cost of being a little more complicated to use.


In this exercise, you will also see how to create an art::Ptr when the object you want to point to is being created in the same module.
In this exercise, you will also see how to create an <code>art::Ptr</code> when the object you want to point to is being created in the same module.


<ol style="list-style-type:lower-alpha">
<ol style="list-style-type:lower-alpha">
Line 486: Line 493:


namespace mu2e {
namespace mu2e {
   typedef art::Assns<TrackTime, KalSeed> OutAssns;
   typedef art::Assns<TrackTime, KalSeed> OurAssns;
}
}


Line 495: Line 502:
<class name="art::Wrapper<art::Assns<mu2e::TrackTime, mu2e::KalSeed, void> >"/>
<class name="art::Wrapper<art::Assns<mu2e::TrackTime, mu2e::KalSeed, void> >"/>
<class name="art::Wrapper<art::Assns<mu2e::KalSeed, mu2e::TrackTime, void> >"/></nowiki>
<class name="art::Wrapper<art::Assns<mu2e::KalSeed, mu2e::TrackTime, void> >"/></nowiki>
  <li>We will create the Assns at the same time we create the TrackTime object, so copy your <code>HelloTrackTime_module.cc</code> from Exercise 8 and call it something different. Make the following changes:</li>
  <li>We will create the <code>art::Assns</code> at the same time we create the <code>TrackTime</code> object, so copy your <code>HelloTrackTime_module.cc</code> from Exercise 8 and call it something different. Make the following changes:</li>
  <ol style="list-style-type:lower-roman">
  <ol style="list-style-type:lower-roman">
   <li>add a <code>produces<OurAssns>();</code> statement</li>
   <li>add a <code>produces<OurAssns>();</code> statement</li>
   <li>create a new Assns to be written to the art event: <code>std::unique_ptr<OurAssns> outputAssns(new OurAssns());</code></li>
   <li>create a new <code>art::Assns</code> to be written to the art event: <code>std::unique_ptr<OurAssns> outputAssns(new OurAssns());</code></li>
   <li>put the Assns into the event: <code>event.put(std::move(outputAssns));</code></li>
   <li>put the <code>art::Assns</code> into the event: <code>event.put(std::move(outputAssns));</code></li>
  </ol>
  </ol>
  <li>At the moment, we are not filling the Assns. To do this we need to create <code>art::Ptr</code>s to the <code>KalSeed</code> and the <code>TrackTime</code></li>
  <li>At the moment, we are not filling the <code>art::Assns</code>. To do this we need to create <code>art::Ptr</code>s to the <code>KalSeed</code> and the <code>TrackTime</code></li>
  <li>We created an <code>art::Ptr<KalSeed></code> in Exercise 9 so repeat that step</li>
  <li>We created an <code>art::Ptr<KalSeed></code> in Exercise 9 so you can repeat the step from that exercise</li>
  <li>For the <code>art::Ptr<TrackTime></code> it's a bit tricker because we haven't added the TrackTimeCollection to the event yet. We need to do the following:</li>
  <li>For the <code>art::Ptr<TrackTime></code> it's a bit tricker because we haven't added the <code>TrackTimeCollection</code> to the event yet and so we can't get a handle to it. We need to do the following isntead:</li>
  <ol style="list-style-type:lower-roman">
  <ol style="list-style-type:lower-roman">
   <li>get the <code>art::ProductID</code> and the <code>ProductGetter</code> at the start of the <code>produce()</code> function:</li>
   <li>get the <code>art::ProductID</code> and the <code>ProductGetter</code> for the <code>TrackTimeCollection</code> at the start of the <code>produce()</code> function:</li>
  <nowiki>auto TrackTimeProdID = getProductID<TrackTimeCollection>();
  <nowiki>auto TrackTimeProdID = getProductID<TrackTimeCollection>();
auto TrackTimeGetter = event.productGetter(TrackTimeProdID);</nowiki>
auto TrackTimeGetter = event.productGetter(TrackTimeProdID);</nowiki>
   <li>create the TrackTime Ptr after adding it to the output TrackTimeCollection:</li>
   <li>create the <code>art::Ptr<TrackTime></code> after adding it to the output <code>TrackTimeCollection</code>:</li>
  <nowiki>art::Ptr<TrackTime> trackTimePtr(TrackTimeProdID, outpuTrackTimes->size()-1, TrackTimeGetter);</nowiki>
  <nowiki>art::Ptr<TrackTime> trackTimePtr(TrackTimeProdID, outpuTrackTimes->size()-1, TrackTimeGetter);</nowiki>
  </ol>
  </ol>
  <li>Finally, we add a new entry to the Assns like so:</li>
  <li>Finally, we add a new entry to the <code>art::Assns</code> like so:</li>
  <nowiki>outputAssns->addSingle(trackTimePtr, kseedPtr);</nowiki>
  <nowiki>outputAssns->addSingle(trackTimePtr, kseedPtr);</nowiki>
  <li>Copy an old writing fcl file and edit it to use the new Assns-creating module (be sure to add the Assns to the output commands!)</li>
  <li>Copy an old writing fcl file and edit it to use the new Assns-creating module (be sure to add the <code>art::Assns</code> to the <code>outputCommands</code>!)</li>
  <li>Run the writing fcl and make sure that the Assns is in the output art file as well as all of your TrackTimes</li>
  <li>Run the writing fcl and make sure that the <code>art::Assns</code> is in the output art file as well as all of your <code>TrackTimes</code> and <code>KalSeeds</code></li>
  <li>Now let's use the Assns by creating a 2D histogram of TrackTime vs the number of hits in the track</li>
  <li>Now let's use the <code>art::Assns</code> by creating a 2D histogram of track time vs the number of hits in the track</li>
  <li>Copy one of your previous histogram module and make the following changes:</li>
  <li>Copy one of your previous histogram module and make the following changes:</li>
  <ol style="list-style-type:lower-roman">
  <ol style="list-style-type:lower-roman">
   <li>make a 2D histogram rather than a 1D histogram</li>
   <li>make a 2D histogram rather than a 1D histogram</li>
   <li>grab the Assns from the event like so:</li>
   <li>grab the <code>art::Assns</code> from the event like so:</li>
  <nowiki>const auto& ourAssnsHandle = event.getValidHandle<OurAssns>(_input);
  <nowiki>const auto& ourAssnsHandle = event.getValidHandle<OurAssns>(_input);
const auto& ourAssns = *ourAssnsHandle;</nowiki>
const auto& ourAssns = *ourAssnsHandle;</nowiki>
Line 532: Line 539:
</ol>
</ol>


That's it for making and using <code>art::Assns</code>.
That's it for making and using an <code>art::Assns</code>!


== Reference Materials ==
== Reference Materials ==
* art workbook
 
* [https://art.fnal.gov/art-workbook/ art workbook]
 
[[Category:Computing]]
[[Category:Tutorial]]
[[Category:Code]]

Latest revision as of 22:02, 24 June 2019

Tutorial Session Goal

This tutorial will show how to write an art module for Mu2e. It will explain how to structure the code, define runtime parameters, produce histograms, and consume and produce data products.

Session Prerequisites and Advance Preparation

This tutorial requires the user to:

This tutorial also assumes that you are somewhat familiar with C++.

Session Introduction

Mu2e uses the art framework to organize our event processing code. The art framework processes events through a configurable sequential set of modules, called a path. A module expects certain inputs, and optionally produces certain outputs. Modules have a standard interface for interacting with the art framework, giving the user access to data at run/subrun/event transitions. Modules have a standard library for defining configuration parameters that can be changed at runtime.

In this tutorial you will learn how to create an art module from a basic template, and perform basic data operations in that module. You will learn how to configure your module in code and fcl, and how to produce various kinds of output.

Disclaimer! In this tutorial we will be copying modules to create new modules. In general, this is not a great idea because this is how bugs can propagate. If you do this in the real world make sure you trust the code you're copying!

Basic Exercises

Setup

For these exercises, we will create a Satellite release.

In the docker container, we assume that you have a volume mounted on your machine to /home/working_area/ in the container (i.e something like docker run --rm -it -v /path/to/your/machine/working_area:/home/working_area -v $XSOCK:$XSOCK -e DISPLAY=unix$DISPLAY mu2e/user:tutorial_1-00 where the -v option is important)

> source /Tutorials_2019/setup_container.sh
> cp -r $TUTORIAL_BASE/ModuleWriting /home/working_area/
> cd /home/working_area/ModuleWriting
> /Offline/v7_4_1/SLF6/prof/Offline/bin/createSatelliteRelease --directory .
> source setup.sh

We also want to create some file lists:

> mkdir filelists
> ls $TUTORIAL_BASE/data/mcs.mu2e.CeEndpoint-mix.*.art > filelists/mcs.mu2e.CeEndpoint-mix.MDC2018h.lst

If you are running this tutorial on one of the mu2egpvm machines, then the setup instructions will be different. These will be added at a later date.

Exercise 1: Running a simple module (Hello, Tutorial!)

In this exercise, we will run a simple module that will print a welcoming message.

  1. The first module has already been written for you so we can compile straight away in the Satellite release
  2. > scons -j4
  3. And run
  4. mu2e -c Ex01/fcl/HelloEx01.fcl This will write "Hello, world" and the full event id for the first 3 events.
  5. Here is a brief overview of the contents of Ex01/src/HelloTutorial_module.cc:
    1. we keep everything in a namespace for mu2e:
    2. namespace mu2e {
    3. the HelloTutorial module is class that inherits from art::EDAnalyzer:
    4. class Ex01HelloTutorial : public art::EDAnalyzer {
    5. we create a structure that will handle the module configuration parameters (see Exercise 2)
    6. struct Config { using Name=fhicl::Name; using Comment=fhicl::Comment; }; typedef art::EDAnalyzer::Table<Config> Parameters;
    7. there is an analyze() function (this is called for every art::Event)
    8. void analyze(const art::Event& event);
  6. You will also see that there is an SConscript file in Ex01/src/. This tells SCons which libraries need to be linked to this package.

That's it for running your first module! This will be the basis of the rest of the exercises in this tutorial.

Exercise 2: Adding module configuration (Hello, Fhicl Validation!)

In this exercise, we will add some module configuration parameters that can be set in fcl

  1. Create a new directory for this exercise called Ex02/ and add src/ and fcl/ subdirectories. Also copy the SConscript file to src/.
  2. Copy last exercise's .cc file to create a new file
  3. > cp Ex01/src/HelloTutorial_module.cc Ex02/src/HelloFhiclValidation_module.cc
  4. Open the new file in your favourite text editor and make the following changes
    1. find and replace HelloTutorial with HelloFhiclValidation
    2. in the Config struct, below the using commands, add the following line to declare a fhicl parameter
    3. fhicl::Atom<int> number{Name("number"), Comment("This module will print this number")};
    4. add a private member variable to the class that will store the value of this parameter
    5. int _number;
    6. fill this member variable in the constructor initialization list:
    7. _number(conf().number())
    8. change the std::cout command to print this number
    9. std::cout << "My _number is..." << _number << std::endl;
  5. Recompile with scons -j4 and fix any compilation errors
  6. Now create a copy of the fcl file, open it in a text editor and make the following changes:
    1. find and replace HelloTutorial with HelloFhiclValidation
    2. add the new fhicl parameter to the module configuration and assign it a value
    3. hello: { module_type : HelloFhiclValidation number : 5 }
  7. Now everything's ready to run mu2e so do that
  8. mu2e -c Ex02/fcl/HelloEx02.fcl
  9. Try doing the following and note the errors you see:
    1. run without the new parameter in the fcl file
    2. run with a typo in the parameter name
    3. run with a value of the parameter that is float or a string

    This is why fhicl validation is nice. We can catch errors in the module configurations before we waste time running mu2e. In the past, this wasn't the case -- you could run a job, thinking that you have changed a parameter, only to find that you had a typo in your fcl. Some already-existing modules still need to be updated and will be slowly converted to use fhicl validation.

  10. We can let the parameter have a default value by adding this value to end of the parameter declaration:
  11. fhicl::Atom<int> defaultNumber{Name("defaultNumber"), Comment("This module will print this number"), 100};
    1. Add another member variable for this parameter, fill it in the constructor initialization and add another cout statement to print this value
    2. Recompile and run without changing the fcl file
    3. Now add the defaultNumber parameter to your fcl file and play with different values

    (Note that it is often better to have default values in fcl prolog, rather than hard-coded into the module itself)

  12. There are some occasions where we want a parameter to be optional. This can be achieved by declaring the parameter like so:
  13. fhicl::OptionalAtom<int> optionalNumber{Name("optionalNumber"), Comment("This module will print this number but it is optional")}; However, we can't initialise this in the initializer list since it might not exist. Instead we have to check it exists in the analyze() function: if (_conf.optionalNumber(_optionalNumber)) { std::cout << "My _optionalNumber is..." << _optionalNumber << std::endl; }
  14. You will need to #include "fhiclcpp/types/OptionalAtom.h"
  15. Play with using and not using the optional parameter in your code

You've just added fhicl validation to your module so that a user can change the configuration without having to recompile all the code!

Exercise 3: Reading in a mu2e data product (Hello, KalSeed!)

In this exercise, we will be reading the results of the track fit and printing their times to the terminal.

  1. Create a new Ex03/ directory with src/ and fcl/ subdirectories. Also copy the SConscript file to src/.
  2. Copy the Exercise 1's .cc file to create a new module and make the following changes
  3. cp Ex01/src/HelloTutorial_module.cc Ex03/src/HelloKalSeed_module.cc
    1. find and replace HelloTutorial for HelloKalSeed
    2. #include RecoDataProducts/inc/KalSeed.hh
    3. add a fhicl parameter of type art::InputTag and a member variable to store the value
    4. fill the member variable and print its value in the constructor
    5. also in the constructor, it is good practice to tell art the products that we will use:
    6. consumes<KalSeedCollection>(_input);
  4. Copy a previous fcl file and edit it so that runs your new module and uses the new parameter (at the moment, it doesn't matter what the value of this parameter is).
  5. Compile, run and make sure everything works as you expect
  6. mu2e -c Ex03/fcl/HelloEx03.fcl
  7. Now we want to read in an art file. In this step, we will just change the fcl file so that it can start from an art file input
    1. in the source block of your fcl file, change EmptyEvent to RootInput
    2. run your fcl file on this file list to make sure everything works
    3. mu2e -c Ex03/fcl/HelloEx03.fcl -S filelists/mcs.mu2e.CeEndpoint-mix.MDC2018h.lst
  8. Now let's actually do something with a Mu2e data product. We will be looking at tracks (class KalSeed) from the downstream e-minus fit. Set the value of your fcl parameter to KFFDeM and then make the following changes in your module's analyze() function
    1. get a valid handle to the KalSeedCollection from the event:
    2. const auto& kalSeedCollectionHandle = event.getValidHandle<KalSeedCollection>(_input);
    3. get the KalSeedCollection itself
    4. const auto& kalSeedCollection = *kalSeedCollectionHandle;
    5. loop through and print the t0 of each KalSeed (you can look in $MU2E_BASE_RELEASE/RecoDataProducts/inc/KalSeed.hh to work out why we need two .t0()s)
    6. for (const auto& i_kalSeed : kalSeedCollection) { std::cout << "t0 = " << i_kalSeed.t0().t0() << " ns" << std::endl; } You will also need to #include the file RecoDataProducts/inc/KalSeed.hh
  9. Recompile and run mu2e

You have just read an already existing Mu2e data product from a Mu2e art file! Here are some optional exercises to explore this topic further:

  1. (Optional): try to print the times of a different KalSeedCollection (e.g. KFFDmuM)
  2. (Optional): try to print some other values from the KalSeed (e.g. momentum (which can be found in KalSegment))
  3. (Optional): try to read a different type of Mu2e data product (you can find the list of data products in an event with mu2e -c $MU2E_BASE_RELEASE/Print/fcl/dumpDataProducts.fcl -S filelist -n 1)

Exercise 4: Filling a histogram (Hello, Histogram!)

In this exercise, we will create a ROOT histogram of the times that we were printing out to the screen in Exercise 3. We will need to use art's TFileService to create and write ROOT objects to a ROOT file.

  1. Create a new Ex04/ directory with src/ and fcl/ subdirectories. Also copy the SConscript file to src/.
  2. Copy Exercise 3's module and make the following changes:
  3. cp Ex03/src/HelloKalSeed_module.cc Ex04/src/HelloHistogram_module.cc
    1. find and replace HelloKalSeed for HelloHistogram
    2. #include art/Framework/Services/Optional/TFileService.h and TH1F.h
    3. add a new private member variable of type TH1F*
    4. add a new function to your module: void beginJob() (this is a standard art module function that will run at the beginning of your job)
    5. in the beginJob() function add:
    6. art::ServiceHandle<art::TFileService> tfs; double min_t0 = 0; double max_t0 = 1700; double t0_width = 10; int n_t0_bins = (max_t0 - min_t0) / t0_width; _hTrackTimes = tfs->make<TH1F>("hTrackTimes","Track t0", n_t0_bins,min_t0,max_t0);
    7. in the analyze() function of your module, fill the histogram with the track times:
    8. _hTrackTimes->Fill(i_kalSeed.t0().t0());
  4. Copy a previous fcl file and change it to make sure it runs the new module (this should just be a change to the module_type)
  5. Compile and run with the following command:
  6. mu2e -c Ex04/fcl/HelloEx04.fcl -S filelists/mcs.mu2e.CeEndpoint-mix.MDC2018h.lst --TFileName out.root Note that, instead of defining the output ROOT filename on the command line. You can set the fcl parameter services.TFileService.fileName in your fcl file
  7. Open the out.root file and explore it with a TBrowser to find your histogram!

Now that you can create a histogram, you can try some of these optional exercises

  1. (Optional): make the histogram parameters (min, max, bin width) fcl parameters
  2. (Optional): create a second instance of your module and read in a different KalSeedCollection (e.g. KFFDmuM)
  3. (Optional): make the plot informative by adding axis labels
  4. (Optional): create a TTree to store the track times

Exercise 5: Creating a subset of a data product (Hello, Cool KalSeed!)

In this exercise, we will find some "cool" KalSeeds that arrive fashionably late by cutting on the track times and put them into their own collection. We will reuse the HelloHistogram module to plot these "cool" KalSeeds.

  1. Create a new Ex05/ directory with src/ and fcl/ subdirectories. Also copy the SConscript file to src/.
  2. Copy the module from Exercise 4
  3. cp Ex04/src/HelloHistogram_module.cc Ex04/src/HelloCoolKalSeed_module.cc
  4. The new module will be a producer, rather than an analyzer, so do a find and replace for the following in HelloCoolKalSeed_module.cc
    1. EDAnalyzer to EDProducer
    2. analyze to produce
    3. const art::Event& to art::Event& (because producer modules are able to modify the event)
    4. remove the call to EDProducer in the constructor initializer list
  5. Also remove the histogram making parts of this module
  6. Before we do any cutting, let's make sure we have a working fcl
  7. Copy the fcl file from Exercise 4 and make the following changes:
    1. add a new producer block in the physics block like so:
    2. producers : { cool : { module_type : HelloCoolKalSeed input : KFFDeM } }
    3. add cool to the p1 path
  8. Compile and run your new fcl file. There should be no difference in the output with the previous exercise
  9. Now that we have the basis of our job, we can start making changes.
  10. In the HelloCoolKalSeed module, make the following changes:
    1. create a new fhicl parameter that will take a float value for a time cut
    2. in the body of the construtor, we need to tell art that the module will be producing a new KalSeedCollection
    3. produces<KalSeedCollection>();
    4. in the produce() function, we need to:
      • create a new KalSeedCollection to fill (actually we create a pointer)
      std::unique_ptr<KalSeedCollection> outputKalSeeds(new KalSeedCollection());
      • fill the new KalSeedCollection
      if (i_kalSeed.t0().t0() > _cut) { outputKalSeeds->push_back(i_kalSeed); }
      • write the new KalSeedCollection to the art::Event once we're done
      event.put(std::move(outputKalSeeds));
  11. In your fcl file
    1. change the input tag of the histogramming module to cool
  12. Recompile and run, and you should see that the histogram is cut at your cut value!

Now that you have created a subset of an already existing collection, you can try some of these optional exercises:

  1. (Optional): change the cut value in fcl and make sure the output is as you expect
  2. (Optional): add some cuts on different variables
  3. (Optional): make some of the cuts optional

Exercise 6: Writing an art file (Hello, My Art File!)

In this exercise, we will be splitting the previous exercise into two different jobs to show how to write an art file. There will be no changes to the source code since is all done in fcl.

  1. Create a new Ex06/ directory with only a fcl/ subdirectories.
  2. Create two copies of your Exercise 5 fcl file. One will be for reading and one will be for writing.
  3. cp Ex05/fcl/HelloEx05.fcl Ex06/fcl/HelloEx06_read.fcl cp Ex05/fcl/HelloEx05.fcl Ex06/fcl/HelloEx06_write.fcl
  4. In your reading fcl, delete everything related to your producer module
  5. In your writing fcl, delete everything related to your analyzer module
  6. Also in your writing fcl:
    1. add an output block (note, this is at the same level as the physics block):
    2. outputs : { MyOutput : { module_type: RootOutput SelectEvents : [p1] fileName : "my-art-file.art" outputCommands : [ "drop *_*_*_*", "keep *_cool_*_*" ] } } The output commands have the following format ObjectType_ModuleLabel_InstanceName_ProcessName
    3. add MyOutput to e1
  7. Run your writing fcl and then check that the resulting .art file only contains your KalSeeds:
  8. mu2e -c $MU2E_BASE_RELEASE/Print/fcl/dumpDataProducts.fcl -s my-art-file.art
  9. Now run your reading fcl on the art file you just created and you should have a new ROOT file with your time histogram in!

Now that you can write an art file, try some of the following optional exercises:

  1. (Optional): write out some other data products along with your cool KalSeeds

Exercise 7: Filtering events we don't want (Hello, Filter!)

You might notice that the number of events in your output art file is the same as in the input, even though we are only writing out the cool KalSeeds and we are cutting on those. This is because we are still write an art::Event record even if it's empty. In this exercise, we will add a filter module to remove events that we expect to be empty.

  1. Create a new Ex07/ directory with src/ and fcl/ subdirectories. Also copy the SConscript file to src/.
  2. Copy your previous producer module and make the following changes:
  3. cp Ex05/src/HelloCoolKalSeed_module.cc Ex07/src/HelloFilter_module.cc
    1. HelloCoolKalSeed to HelloFilter
    2. EDProducer to EDFilter
    3. produce to filter
    4. the void return-type of filter() to bool (because we will return to art, true or false whether the event passes the filter)
    5. delete anything related to cutting or producing a new KalSeedCollection (you want to keep the input KalSeedCollection)
  4. This module will run over our "cool" KalSeedCollection and tell art whether there are any tracks in the collection or not. Add some logic to the filter() function to do this.
  5. Now, let's set up the fcl file to write out only the events that pass this filter. Create copies of your Exercise 6 fcls and make the following changes to the writing one:
    1. add a filters block to the physics block:
    2. filters : { helloFilter : { module_type : HelloFilter input : "cool" } }
    3. add "helloFilter" to the path p1
  6. Compile and run your new writing fcl file and you will see that the number of events in the output file has fallen
  7. You should be able to run your the reading fcl on your new art file and the output histogram should be the same

Exercise 8: Creating a new data product (Hello, My Data Product!)

Let's say we want to create a data product that just contains the track t0. This is an oversimplified example but it will illustrate all the issues.

  1. Create a new Ex08/ directory with src/, fcl/, and inc/ subdirectories. Also copy the SConscript file to src/.
  2. Create a new class in a header file in inc/:
  3. #ifndef ModuleWriting_TrackTime_hh #define ModuleWriting_TrackTime_hh #include "RecoDataProducts/inc/KalSeed.hh" namespace mu2e { class TrackTime { public: TrackTime() : _time(0.0) {} TrackTime(const KalSeed& kseed) { _time = kseed.t0().t0(); } const double time() const { return _time; } private: double _time; }; typedef std::vector<TrackTime> TrackTimeCollection; } #endif
  4. To be able to write this data product out to an art file we need to add two files into the src/ directory:
    1. classes_def.xml
    2. <lcgdict> <class name="mu2e::TrackTime"/> <class name="art::Wrapper<mu2e::TrackTime>"/> <class name="std::vector<mu2e::TrackTime>"/> <class name="art::Wrapper<std::vector<mu2e::TrackTime> >"/> </lcgdict> The art::Wrapper lines are because we will write this data product to an art file.
    3. classes.h
    4. // // headers needed when genreflex creates root dictionaries // for objects written to art files // #define ENABLE_MU2E_GENREFLEX_HACKS #include <vector> #include "canvas/Persistency/Common/Wrapper.h" #include "Ex08/inc/TrackTime.hh" #undef ENABLE_MU2E_GENREFLEX_HACKS

    This is everything we need to create a new data product that we can write to an art file.

  5. Now let's copy some old modules to create new modules that will:
    1. create a TrackTimeCollection from a KalSeedCollection;
    2. cp Ex05/src/HelloCoolKalSeed_module.cc Ex08/src/HelloTrackTime_module.cc
    3. create a subset of a TrackTimeCollection;
    4. cp Ex05/src/HelloCoolKalSeed_module.cc Ex08/src/HelloCoolTrackTime_module.cc
    5. filter events based on the size of a TrackTimeCollection; and
    6. cp Ex07/src/HelloFilter_module.cc Ex08/src/FilterTrackTime_module.cc
    7. create a histogram using TrackTimes
    8. cp Ex04/src/HelloHistogram_module.cc Ex08/src/HistogramTrackTime_module.cc
  6. In HelloTrackTime:
    1. remove anything to do with cutting
    2. write out a TrackTimeCollection rather than a KalSeedCollection
    3. add a check that the input KalSeedCollection and the output TrackTime collection are the same size. Throw an exception if they are not (throw cet::exception("Tutorial") << "message";)
  7. In HelloCoolTrackTime:
    1. replace KalSeed with TrackTime
    2. make sure to use the correct method of TrackTime to get the time
  8. In FilterTrackTime:
    1. replace KalSeed with TrackTime
  9. In HistogramTrackTime:
    1. replace KalSeed with TrackTime
    2. make sure to use the correct method of TrackTime to get the time
  10. Create copies of your writing and reading fcls from the Exercise 7 and
    1. update the modules in both fcl files to use your new modules that use TrackTime (should just be changes to each module_type)
    2. add a configuration of your new producer module e.g.
    3. trackTime : { module_type : HelloTrackTimeDataProducts input : "KFFDeM" }
    4. add the new producer to the start of the path p1
    5. change the input of your cutting module to use the new producer
  11. Recompile and run your writing module
  12. Use $MU2E_BASE_RELEASE/Print/fcl/dumpDataProducts.fcl to check that you only have TrackTimeCollections in your art file
  13. Run your reading fcl on your new art file and check that your histogram looks as expected

You've just created your own data product and written it to an art file!

Exercise 9: Creating and using a pointer to a data product (Hello, Art Ptrs!)

In this exercise, we will show one way to link the TrackTime object to the original KalSeed by using an art::Ptr.

  1. Create a new Ex09/ directory with src/, fcl/ and inc/ subdirectories. Also copy the SConscript file to src/.
  2. Copy your TrackTime.hh file to create a new TrackTimePtr.hh and make the following changes:
    1. find-and-replace "TrackTime" with "TrackTimePtr"
    2. add a new private member variable of type art::Ptr<KalSeed>
    3. add a getter function (const art::Ptr<KalSeed> kalSeed() const) and a setter function (void setKalSeedPtr(const art::Ptr<KalSeed>& ptr))
  3. Copy the classes.h and classes_def.xml from Exercise 8 and do a find-and-replace of "TrackTime" with "TrackTimePtr"
  4. Copy your modules from Exercise 8, rename them and do a find-and-replace of "TrackTime" with "TrackTimePtr"
  5. Copy your fcl files from Exercise 8 and do a find-and-replace of "TrackTime" with "TrackTimePtr"
  6. Run your new fcl and check that things work as they did before
  7. In order to create an art::Ptr, we need to have a handle to the collection and an index into the collection. In your TrackTime creating module, we need to make the following changes:
    1. move to an index based for loop (i.e. we want something like this now):
    2. for (auto i_kalSeed = kalSeedCollection.begin(); i_kalSeed != kalSeedCollection.end(); ++i_kalSeed) {
    3. create an art::Ptr and pass it to the TrackTime object
    4. TrackTime track_time(*i_kalSeed); art::Ptr<KalSeed> kseedPtr(kalSeedCollectionHandle, i_kalSeed-kalSeedCollection.begin()); track_time.setKalSeedPtr(kseedPtr);
  8. Recompile and re-run to make sure things work as before
  9. In your histogram module, add a second histogram that plots the number of hits in the KalSeed (kseed.hits().size()) through the Ptr interface
  10. To your writing fcl script add the following to your output commands : "keep mu2e::KalSeeds_*_*_*". This will make sure the KalSeeds are written out into your art file.
  11. Recompile and re-run to check that your new histogram is there.

You've just created and used an art::Ptr! There are a few limitations that you might want to be aware of. These are covered in the following optional exercises:

  1. (Optional): Try running without the added keep command at the end of the exercise. This will show that the object you want to access must be in the same file as the one you are reading.
  2. (Optional): Add a module that produces a reduced KalSeedCollection with a different cut (like in Exercise 5) and only write out the reduced KalSeedCollection to your art file. Play with putting this module before and after the module that creates your TrackTimePtrs (you will also have to update input tags). You will find that creating the reduced KalSeedCollection after creating the TrackTimePtrs will result in errors when trying to use the art::Ptr. This is because the TrackTimePtrs are pointing to the full KalSeedCollection which you are not writing out. This is an issue if we want to update a collection (e.g. for compression) after creating art::Ptrs to that collection. In that case we have to go through all of our data products and create brand new objects that contain art::Ptrs to the collection and update them to the new collection.

Exercise 10: Creating and using an assn between two data products (Hello, Art Assns!)

In this exercise, instead of linking together the TrackTime and KalSeed objects with an art::Ptr, we will use an art::Assns. An art::Assns has an advantage over an art::Ptr in that it is bidirectional (i.e. you can go from TrackTime to KalSeed and vice versa). However, it comes at a cost of being a little more complicated to use.

In this exercise, you will also see how to create an art::Ptr when the object you want to point to is being created in the same module.

  1. Create a new Ex10/ directory with src/, fcl/ and inc/ subdirectories. Also copy the SConscript file to src/.
  2. First, let's typedef the Assns we will create. In a new header file:
  3. #ifndef ModuleWriting_OurAssns_hh_ #define ModuleWriting_OurAssns_hh_ #include "canvas/Persistency/Common/Assns.h" #include "RecoDataProducts/inc/KalSeed.hh" #include "Ex08/inc/TrackTime.hh" namespace mu2e { typedef art::Assns<TrackTime, KalSeed> OurAssns; } #endif and add mu2e::OurAssns and art::Wrapper<mu2e::OurAssns> to a new classes_def.xml. You will also need to add the lines: <class name="art::Assns<mu2e::TrackTime, mu2e::KalSeed, void>"/> <class name="art::Assns<mu2e::KalSeed, mu2e::TrackTime, void>"/> <class name="art::Wrapper<art::Assns<mu2e::TrackTime, mu2e::KalSeed, void> >"/> <class name="art::Wrapper<art::Assns<mu2e::KalSeed, mu2e::TrackTime, void> >"/>
  4. We will create the art::Assns at the same time we create the TrackTime object, so copy your HelloTrackTime_module.cc from Exercise 8 and call it something different. Make the following changes:
    1. add a produces<OurAssns>(); statement
    2. create a new art::Assns to be written to the art event: std::unique_ptr<OurAssns> outputAssns(new OurAssns());
    3. put the art::Assns into the event: event.put(std::move(outputAssns));
  5. At the moment, we are not filling the art::Assns. To do this we need to create art::Ptrs to the KalSeed and the TrackTime
  6. We created an art::Ptr<KalSeed> in Exercise 9 so you can repeat the step from that exercise
  7. For the art::Ptr<TrackTime> it's a bit tricker because we haven't added the TrackTimeCollection to the event yet and so we can't get a handle to it. We need to do the following isntead:
    1. get the art::ProductID and the ProductGetter for the TrackTimeCollection at the start of the produce() function:
    2. auto TrackTimeProdID = getProductID<TrackTimeCollection>(); auto TrackTimeGetter = event.productGetter(TrackTimeProdID);
    3. create the art::Ptr<TrackTime> after adding it to the output TrackTimeCollection:
    4. art::Ptr<TrackTime> trackTimePtr(TrackTimeProdID, outpuTrackTimes->size()-1, TrackTimeGetter);
  8. Finally, we add a new entry to the art::Assns like so:
  9. outputAssns->addSingle(trackTimePtr, kseedPtr);
  10. Copy an old writing fcl file and edit it to use the new Assns-creating module (be sure to add the art::Assns to the outputCommands!)
  11. Run the writing fcl and make sure that the art::Assns is in the output art file as well as all of your TrackTimes and KalSeeds
  12. Now let's use the art::Assns by creating a 2D histogram of track time vs the number of hits in the track
  13. Copy one of your previous histogram module and make the following changes:
    1. make a 2D histogram rather than a 1D histogram
    2. grab the art::Assns from the event like so:
    3. const auto& ourAssnsHandle = event.getValidHandle<OurAssns>(_input); const auto& ourAssns = *ourAssnsHandle;
    4. loop through the Assns and fill the histogram like this:
    5. for (const auto& i_entry : ourAssns) { const auto& ttimePtr = i_entry.first; const auto& kseedPtr = i_entry.second; _hOurAssns->Fill(ttimePtr->time(), kseedPtr->hits().size()); }
  14. Create a new reading fcl and run the new histogram-making module. Look in the output ROOT file and you should see the 2D histogram!

That's it for making and using an art::Assns!

Reference Materials