CodeArtFclTutorial: Difference between revisions

From Mu2eWiki
Jump to navigation Jump to search
No edit summary
Line 3: Line 3:
==Introduction==
==Introduction==


HEP data is organized around "events" which record a desired HEP occurrence. In the case of [[DetectorIntro|Mu2e]], an event records the detector response during one microbunch, consisting of a short burst of protons hitting the production target, thousands of muons stopping in the aluminum stopping target, and those muons interacting or decaying.  This happens nominally every 1695 ns.
In the case of [[DetectorIntro|Mu2e]], an [[ComputingStart|event]] records the detector response during one microbunch, consisting of a short burst of millions of protons hitting the production target, thousands of muons stopping in the aluminum stopping target, and those muons interacting or decaying.  This happens nominally every 1695 ns and each of these burst forms and event.


Event must be stored in a format, and for Mu2e that is the '''art''' format. art is software package that provides:
When we read and process these events we will need a framework to organize our executables, and way to control the behavior of the executable, and a format to write the events into.  These functions are provided by the ''art''' software package.
 
==Art==
 
The art package provides:
* executable framework
* the event format
* the event format
* executable framework
* a control language called "fcl" (pronounced fickle)
* services such as random numbers
* services such as random numbers


The art format is a highly-structured format wchih contain the events, processing history, and other items. Under the covers, art uses the root package to read and writre
===Framework===
 
The art format (developed at Fermilab) is a highly-structured format which contains the events, processing history, and other items. Under the covers, art uses the [https://root.cern.ch/root/ root] package (used widely in HEP) to read and write the files.  You can see this:
</pre>
> file /pnfs/mu2e/tape/phy-sim/sim/mu2e/DS-beam/MDC2018a/art/ff/f7/sim.mu2e.DS-beam.MDC2018a.001002_00373598.art
/pnfs/mu2e/tape/phy-sim/sim/mu2e/DS-beam/MDC2018a/art/ff/f7/sim.mu2e.DS-beam.MDC2018a.001002_00373598.art: ROOT file Version 61204 (Compression: 7)
<pre>
You will also notice the file has a name that ends in ".art".  We try to use  a small set of well-defined [[FileNames|file types]].
 
When we write code, typically to simulate the detector, reconstruct the data, or analyze the data, we write our code in '''modules'''.  We then run the framework, and tell it to call our module.  We also tell it what input files to use, if any.  The framework opens the input files, reads data, calls our module with the data, event by event, and then handles any filtering of events, and writing output.
 
In a fresh window, setup an Offline release,
source /cvmfs/mu2e.opensciencegrid.org/setupmu2e-art.sh
source /cvmfs/mu2e.opensciencegrid.org/Offline/v7_4_1/SLF6/prof/Offline/setup.sh
 
==Code==
 
Any module we write is compiled into a shared object library which is loaded by the framework on demand. For example,
ls -l $MU2E_BASE_RELEASE/TrkHitReco/src/StrawHitReco_module.cc
is compied into
ls -l $MU2E_BASE_RELEASE/lib/libmu2e_TrkHitReco_StrawHitReco_module.so
which we can then ask the framework to run (or not).
 
Just as a quick peak, open the cc file:
emacs $MU2E_BASE_RELEASE/TrkHitReco/src/StrawHitReco_module.cc
 
Find the line
class StrawHitReco : public art::EDProducer
Since the module inherits from a framework base class, the framework can manipulate it.
 
Find the line
StrawHitReco::beginJob
this moethod is called once at the beginning of the job.  Similarly
StrawHitReco::beginRun
is called when the input events change run number.
StrawHitReco::produce
is called for every event.  It is called "produce" because it typically produces date to insert in the event, in this case, the reconstructed tracker hits.
 
Find the line
event.getValidHandle(_sdtoken)
this is retrieving the raw hits from the event.  All of this will be covered in more detail later.
 
==Fcl==
 
Since all modules are available to the framework as shared objects, we have to tell the framework which modules to run.  We also need the capability to configure the modules, for example, to set thresholds for what makes a good hit.  This executable configuration handled by the fcl language.
 
Take look at a simple fcl file:
emacs $MU2E_BASE_RELEASE/Validation/fcl/ceSimReco.fcl
which generates events with a conversion electron.
 
Find the process_name line:
process_name : ceSimReco
this is important because any output from his executable run will be labeled by a string, to help track the data's processing history.
 
Find the services
services : {
    @table::service.all
}
This defines a set of standard services, such as geometry or random numbers (the "@table" says just use standard stuff defined in the includes).
 
The next part
physics : {
    producers: {
    ....
defines what modules to configure and what that configuration is. Again all the "@" is saying use stuff defined in the includes.  Let trace back one example.
 
 
 
 
***********************


where is the code (do not run git yet tho), intro to releases, cvmfs, run genReco, set output file names and Nevents
where is the code (do not run git yet tho), intro to releases, cvmfs, run genReco, set output file names and Nevents

Revision as of 22:52, 18 June 2019


Introduction

In the case of Mu2e, an event records the detector response during one microbunch, consisting of a short burst of millions of protons hitting the production target, thousands of muons stopping in the aluminum stopping target, and those muons interacting or decaying. This happens nominally every 1695 ns and each of these burst forms and event.

When we read and process these events we will need a framework to organize our executables, and way to control the behavior of the executable, and a format to write the events into. These functions are provided by the art' software package.

Art

The art package provides:

  • executable framework
  • the event format
  • a control language called "fcl" (pronounced fickle)
  • services such as random numbers

Framework

The art format (developed at Fermilab) is a highly-structured format which contains the events, processing history, and other items. Under the covers, art uses the root package (used widely in HEP) to read and write the files. You can see this:

> file /pnfs/mu2e/tape/phy-sim/sim/mu2e/DS-beam/MDC2018a/art/ff/f7/sim.mu2e.DS-beam.MDC2018a.001002_00373598.art /pnfs/mu2e/tape/phy-sim/sim/mu2e/DS-beam/MDC2018a/art/ff/f7/sim.mu2e.DS-beam.MDC2018a.001002_00373598.art: ROOT file Version 61204 (Compression: 7)

You will also notice the file has a name that ends in ".art".  We try to use  a small set of well-defined file types.

When we write code, typically to simulate the detector, reconstruct the data, or analyze the data, we write our code in modules.  We then run the framework, and tell it to call our module.  We also tell it what input files to use, if any.  The framework opens the input files, reads data, calls our module with the data, event by event, and then handles any filtering of events, and writing output.

In a fresh window, setup an Offline release,
 source /cvmfs/mu2e.opensciencegrid.org/setupmu2e-art.sh
 source /cvmfs/mu2e.opensciencegrid.org/Offline/v7_4_1/SLF6/prof/Offline/setup.sh

Code

Any module we write is compiled into a shared object library which is loaded by the framework on demand. For example, ls -l $MU2E_BASE_RELEASE/TrkHitReco/src/StrawHitReco_module.cc is compied into ls -l $MU2E_BASE_RELEASE/lib/libmu2e_TrkHitReco_StrawHitReco_module.so which we can then ask the framework to run (or not). Just as a quick peak, open the cc file: emacs $MU2E_BASE_RELEASE/TrkHitReco/src/StrawHitReco_module.cc Find the line class StrawHitReco : public art::EDProducer Since the module inherits from a framework base class, the framework can manipulate it. Find the line StrawHitReco::beginJob this moethod is called once at the beginning of the job. Similarly StrawHitReco::beginRun is called when the input events change run number. StrawHitReco::produce is called for every event. It is called "produce" because it typically produces date to insert in the event, in this case, the reconstructed tracker hits. Find the line event.getValidHandle(_sdtoken) this is retrieving the raw hits from the event. All of this will be covered in more detail later.

Fcl

Since all modules are available to the framework as shared objects, we have to tell the framework which modules to run. We also need the capability to configure the modules, for example, to set thresholds for what makes a good hit. This executable configuration handled by the fcl language. Take look at a simple fcl file: emacs $MU2E_BASE_RELEASE/Validation/fcl/ceSimReco.fcl which generates events with a conversion electron. Find the process_name line: process_name : ceSimReco this is important because any output from his executable run will be labeled by a string, to help track the data's processing history. Find the services services : { @table::service.all } This defines a set of standard services, such as geometry or random numbers (the "@table" says just use standard stuff defined in the includes). The next part physics : { producers: { .... defines what modules to configure and what that configuration is. Again all the "@" is saying use stuff defined in the includes. Let trace back one example. *********************** where is the code (do not run git yet tho), intro to releases, cvmfs, run genReco, set output file names and Nevents intro to fcl, paths and filters. maybe a couple of tutorials? * existing releases * so's * command line * print and validate * modifying fcl