Services

Introduction

A key element of art is support for an idea named "services". From the point of view of Mu2e physicists, a service acts as an agent that manages some resource and allows all other code to interact with that resource. For example, code in a module can use services to access the Mu2e geometry, the particle data table and the conditions data ( formerly known as calibration constants ). These are examples of services written and maintained by Mu2e but there are also a number of services that are native to art. Some of the native services provide capabilities used internally by art and others are used for tracing, profiling and debugging. Two art-provided services will be highly visible to Mu2e physicists, the TFileService , which helps to manage ROOT histograms and ntuples, and the [[RandomNumbers| RandomNumberGenerator] service, which manages the state of random number engines in order to ensure either uniqueness of random number sequences, or repeatability of random number sequences, as needed.

The only service that interacts with the art::Event is the RandomNumberGenerator service; all others are forbidden to do so. Services must not be used as a back door for two modules to communicate with each other; a key design rule of art is that modules may communicate with each other only via the art::Event.

Any Mu2e code may access all services using the handles mechanism ; this defines what art means by a service. Modules may get handles to services; services may also get handles to other services, so long as this does not create loop dependencies. Services cannot call code in modules.

A handful of the art provided services are always present in any art job. All other services are loaded on demand using dynamic libraries (.so). Services are run-time configurable: when art instantiates a service, it looks in the run-time configuration (.fcl file ). to find a parameter set named for that service; this parameter set is passed as an argument to the constructor of that service. Services may register with art to be called at state changes in the event loop, such as begin/end of Job, begin/end of Run, begin/end of SubRun, start/end of processing one event and the open/close of input/output files. There are other possibilities that are used for art internal uses but are not likely to be of general interest to Mu2e.

The art service model supports multi-threading, which may become important in the era of computers with many cores and limited memory per core. Specifically the model is aware of the need for both global and thread-local data.

Accessing Services

The following code fragment illustrates how to access two services:

   #include "art/Framework/Services/Registry/ServiceHandle.h"
   #include "art/Framework/Services/Optional/TFileService.h"
   #include "GeometryService/inc/GeometryService.hh"

   art::ServiceHandle<<font color=red>art::TFileService</font>>       tfs;
   art::ServiceHandle<<font color=red>mu2e::GeometryService</font>>   geom;

One requests access to a particular service by specifying the class name of the service as the template argument to the class template art::ServiceHandle. An art::ServiceHandle<T> is just a type of safe pointer to an object of type T: if art does find the service, one uses an art::ServiceHandle exactly like a pointer; if art cannot find the service, the handle will throw an exception when it is constructed. In the pre-art framework, each use of a ServiceHandle was a CPU expensive operation because it found the service anew on each use; that is no longer true. In an art::ServiceHandle, all of the CPU expensive work is done inside the constructor and the result is stored in a bare pointer; because of in-lining, using an art::ServiceHandle has exactly the same run-time performance as using a bare pointer.

Any code running within art may get a handle to any service. This is much like getting an instance pointer to a singleton; while services do have some superficial similarities to singletons, services services are not singletons .

To be a bit more specific, an art::ServiceHandle<T> behaves like a bare pointer to a non-const object of type T; that is, given an art::ServiceHandle<T>, one may call all public methods of T, including modifier methods. Most of the services that will be used by Mu2e do not have any modifier methods, only accessor methods. This helps to enforce the rule that modules may only communicate with each other via the art::Event.

By design of the service registry, once a Service has been instantiated, it's address will never change for the duration of the art job. While the internal state of the service may change considerably, its address will not. Therefore it is safe to get an art::ServiceHandle at the start of a job and cache the handle for the duration of the job; in general it is not safe to extract information from a within Service and to cache that information. Additional information about handles in general and art::ServiceHandles in particular is available. Because every service is different, it is not possible to give a general discussion of how to use a handle to a service.

art Supplied Services

The services native to art are discussed on a separate web page.

Mu2e Supplied Services

At this writing ( August, 2012 ), Mu2e supplies five services,

GeometryService:	GeometryService/src/GeometryService_service.cc
ConditionsService:	ConditionsService/src/Conditions_service.cc
GlobalConstantsService:	ConditionsService/src/GlobalConstants_service.cc
DbService:	DbService/src/DbService_service.cc
ProditionsService:	ProditionsService/src/ProditionsService_service.cc
SeedService:	SeedService/src/Seed_service.cc

Two steps have been taken to avoid name collisions between Mu2e supplied services and art supplied services. The first step is that the code for all Mu2e supplied services must be in the mu2e namespace. The second step is in the FHiCL file: when art instantiates an art supplied service named xxx, it looks in the FHiCL file for a parameter set named services.xxx; when art instantiates a mu2e supplied service with the name xxx, it looks in the FHiCL file for a parameter set named services.user.xxx. The additional element in the FHiCL name, user, fills the same role as a namespace and ensures that the names of Mu2e supplied services will not collide with those of the art supplied services.

The GeometryService provides const access to a reconstruction-centric representation of the geometry for use by reconstruction and analysis codes; it also provides tools to help construct alternate representations, such as that used in Geant4. There is a web page that discusses the big picture view of geometry within the Mu2e Offline software. The GeometryService is a reasonably advanced prototype but many details are not yet present. The GeometryService has one run-time configurable parameter, the name of the file that holds the geometry information:

inputFile : "Mu2eG4/test/geom_01.txt"

As the code matures, this information will be moved to a database and the run-time parameter will become a database key.

The Mu2e code base also provides the GeomHandle class template, which is a shortcut for using the GeometryService. It is described on the web page that discusses handles.

The ConditionsService remains a very crude prototype that provides const access to all kinds of conditions data; it is really not much more than a placeholder. The ConditionsService has one run-time configurable parameter, the name of the file that holds the conditions data:

conditionsfile : "Mu2eG4/test/conditions_01.txt"

This information will eventually be moved to a database and the run-time configurable parameter will become a database key.

The Mu2e code base also provides the ConditionsHandle class template, which is a shortcut for using the ConditionsService. It behaves similarly to the GeomHandle class template and it is described on the web page that discusses handles.

The DbService allows low-level access to a conditions database to retrieve database tables, through the appropriate intervals of validity. The ProditionService is the advanced form the ConditionsService which allows database access through DbService, and allows computing cached values based on database tables and other Proditions services - see the documentation at the ConditionsData. Eventually, the old ConditionsService will be completely replaced with Proditions, at which time ProditionsService name may be changed to ConditionsService.

The GlobalConstantsService behaves similarly to the ConditionsService. The difference is that information within the GlobalConstants service must be known at service-constructor time and it must be unchanged for the duration of the job. Two examples of such information are the particle data table and the lifetime of muonic Aluminium. In contrast, information within the ConditionsService may have time dependence and, at the earliest, will not be available until the first beginRun has been encountered. Other information within the ConditionsService may not be known until beginSubRun-time or even until the first event has been seen.

The SeedService is a tool for managing seeds for random number engines on a per module-label basis. See the instructions on using the SeedService.

Most Mu2e physicists will not need to understand the G4Helper service. Its job is to work around several weakness is the design of Geant4. One weakness of Geant4 is that there are many objects that you must create on the heap, give to Geant4 and then remember to delete at the of the job. The G4Helper service provides a primitive garbage collector; the use pattern is to create an object on the heap, register the object with the garbage collector and then to give the object to Geant4; at the end of the job, the garbage collector deletes all registered objects. The garbage collector is primitive in the sense that it only works if the order of deletion of the objects is not important; it is your responsibility to learn which G4 related objects require garbage collection and which G4 related objects are deleted by other G4 related objects.

G4Helper also provides some bookkeeping tools to retain intermediate results when constructing the Geant4 geometry. The G4Helper service is the one Mu2e provided service that has modifier methods. The G4Helper service has no run-time configuration information but an empty parameter set named services.user.G4Helper must be present in the .fcl file; if it is absent, art will throw an exception.

AntiLeakRegistry in G4Helper (aka Garbage Collector)

The design of G4 often requires code with the following pattern:

1. Instantiate on object on the heap: T* t = new T( /* arguments */ );
2. Call a G4 method giving t as an argument.
3. There are three cases:
   1. G4 takes ownership of the object pointed to by t and will call delete on t at the appropriate time, usually the end of the job.
   2. G4 does not take ownership and it is the responsibilty of the Mu2e code to call delete on t at the end of the job.
   3. G4 does not take ownership and it is the responsibilty of the Mu2e code to call delete on t at some time other than the end of the job. This case is very rare.

The remainder of this section discusses case 3b).

At the time that G4 was designed, it was common practice in HEP code that, in case 3b, the objects were never deleted; they were simply leaked at the end of the job. This was considered acceptable practice since it did not waste memory and did not impair performance; it was just a failure to clean up.

Mu2e forbids this practice; we require that all new'ed objects be deleted before the end of the job. We require this because we use tools such as valgrind to find memory leaks and we do not want to miss real problems that are hidden by many thousands of lines of printout about case 3b).

In order to support the required behaviour for case 3b), Mu2e provides a primitive garbage collector. Is is the object of type AntiLeakRegistry that is available from the G4Helper service. This registry is created at the start of the job and one may register objects with it throughout the job. At the end of the the job the registry deletes everthing that is registered with it. I mentioned that this registry is primitive: it can only properly manage objects that are allowed to be deleted in any order. If you have objects that must be deleted in a prescribed order, you will need to find another solution; although it may be possible to simply put the top level object in the registry and let it's destructor do the rest of the work.

The following code fragment from Mu2eG4/src/constructTTrackerv3.cc illustrates one way to use the registry:

  // Get the registry.
  G4Helper    & _helper   = *(art::ServiceHandle<G4Helper>());
  AntiLeakRegistry & reg = _helper.antiLeakRegistry();

  // Lots of code deleted

  // Create a new object that will be owned by the registry.
  G4RotationMatrix* rotTub = reg.add(G4RotationMatrix(RYForTrapezoids));

  // rotTub is passed as an argument to G4 code that uses it but does not take ownership.

The last line in this example could also have been written:

  G4RotationMatrix* rotTub = new G4RotationMatrix(RYForTrapezoids);
  reg.add(rotTub );

or even as

  G4RotationMatrix* rotTub = reg.add(new G4RotationMatrix(RYForTrapezoids) );

In the first case, the AntiLeakRegistry will insert a copy of its argument into an std::list of the appropriate type. In the last two cases, the AntiLeakRegistry will insert a copy of the pointer into an std::list of the appropriate type. So the first form has a trivial mount of extra overhead; for all but very, very large objects, this overhead is unimportant for operations that are done once during the initialization phase of the job. I have a preference for the first form, despite this inefficiency, because it avoids the use of new entirely.

In the registry, the objects are held in std::lists because subsequent additions to an std::list will not change the memory location of objects already in the list.

Order in which Services are Called

This section will discuss the order in which art calls the callback methods of user supplied services. Some parts of this section may also be relevant to the order in which art calls the callback methods of the optional art supplied services; it is probably totally irrelevant for the non-optional art-supplied services. The content addresses two questions: what is the native order for calling these callback methods and how does one change the order.

The user supplied services that are needed for a job must have their configuration present in the .fcl file for that job; the following example is taken from Mu2eG4/test/g4test_03.fcl

services : {

  // art supplied services go here.
  GeometryService        : { inputFile      : "Mu2eG4/test/geom_01.txt"            }
  ConditionsService      : { conditionsfile : "Mu2eG4/test/conditions_01.txt"      }
  GlobalConstantsService : { inputFile      : "Mu2eG4/test/globalConstants_01.txt" }
  G4Helper               : { }
  SeedService            : @local::automaticSeeds

}

In the fhicl::ParameterSet object for services, these 5 services will appear in the canonical order prescribed by FHiCL, which happens to be alphabetical by service name. The default behavior of art is to call the constructors of the services in the order in which the services appear in the services parameter set. All service constructors are called before any module constructors are called. Most service constructors register several callback methods with the callback registry, the art::ActivityRegistry object passed in to the c'tor of each service. Therefore the order that the callback methods appear in the registry is the order in which the services were constructed.

The key to understanding this section is the following: suppose that many services have registered a preBeginRun callback method and a postBeginRun callback method; when art encounters a beginRun in the event loop, it will:

1. call the preBeginRun callback methods in the order in which they were registered in the callback registry.
2. call the beginRun method for each module
3. call the postBeginRun callback methods in the REVERSE order in which they were registered in the callback registry.

For example, suppose we have AService, BService and some modules, the sequence of calls at beginRun-time is:

AService::preBeginRun
BService::preBeginRun
 ... beginRun methods of the modules
BService::postBeginRun
AService::postBeginRun

Note the reversal of the order for the post* calls; this is meant to follow the behaviour of nested scopes. Similarly for preBeginJob/postBeginJob, preEndRun/postEndRun and all other states in the event loop for which callback methods may be registered.

It is legal for services to depend on services; the only constraint is that the graph of which service depends on which other services must be acyclic. For example the GeometryService might some day depend on the ConditionsService, through which the GeometryService can access its alignment information. For the ConditionsService/GeometryService example to work correctly, it is important that the preBeginRun and preSubRun callback methods of the ConditionsService be called before those of the GeometryService. As it happens, this is the default behaviour because of an accident of the canonical ordering within FHiCL. In this example we do not imagine needing postBeginRun and postBeginSubRun callback methods.

The question is: how would we make this work if the canonical ordering within FHiCL gave the wrong order? The answer is as follows.

 GeometryService::GeometryService(fhicl::ParameterSet const& pset,
                                  art::ActivityRegistry&     iRegistry)
  // Initializers deleted/
  {
    art::ServiceHandle<ConditionsService> conditions("ignored");
    iRegistry.sPreBeginRun.watch(this, &GeometryService::preBeginRun);
    iRegistry.sPostEndJob.watch (this, &GeometryService::postEndJob );
  }

The key is that the request for the service handle precedes the calls to the registry. When art encounters the request for the service handle to the ConditionsService, it will first check to see if that service has already been constructed; if it has been constructed, art will return the handle; if it has not been constructed, art will construct the ConditionsService and return the handle. In either case, this guarantees that the ConditionsService registers its callback methods before those of the GeometryService.

Therefore, when art encounters a beginRun, it will call the preBeginRun callback registred for the ConditionsService before calling that of the GeometryService. Similarly for preBginJob/postBeginJob, preEndRun/postEndRun etc. This ensures the required behaviour.

For some examples that verify/illustrate this behaviour, look in:

Sandbox/src/FooService_service.cc
Sandbox/src/BarService_service.cc
Sandbox/src/XBarTest_module.cc

Sandbox/test/footest.fcl
Sandbox/test/bartest.fcl
Sandbox/test/xbartest.fcl

In the era of the conditions database, we force the following order. Every service that needs another service should create that service in its constructor. New dependencies must not violate this ordering or there may be construction loops.

GlobalConstantsService
DbService (low-level conditions database)
GeometryService (including Bfield)
RecoGeometryService
ConditionsService
ProditionsService (database-based conditions)
SeedService
G4Helper
BTrkHelper

not used:
Alignment
Sandbox/src/BarService_service.cc
Sandbox/src/Bug01Service_service.cc
Sandbox/src/FooService_service.cc
Sandbox/src/XBarService_service.cc

Making New Services

To make your own service, follow the model of the GeometryService, GeometryService/src/GeometryService_service.cc and GeometryService/inc/GeometryService.hh. The other important file is GeometryService/src/SConscript. The sections below contain the contents of those files, with the details specific to the GeometryService stripped out, leaving only the parts relevant for any service.

The stripped down header file is:

#include "fhiclcpp/ParameterSet.h"
#include "art/Framework/Services/Registry/ActivityRegistry.h"

namespace mu2e {

  class GeometryService {

public:
    GeometryService(const fhicl::ParameterSet&, art::ActivityRegistry& );

    void preBeginRun( art::Run const &run);

    // Accessor member functions

private:

    // Private data members go here.

  };

}

An art service does not inherit from a Service base class; this is in contrast to an art module that must inherit from one of the module base classes. The minimum functional service contains a constructor, its private data and accessors to that private data. In addition, the class may contain zero or more member functions that will be registered with art as callbacks to be called on certain state transitions within the event processing loop, such as the start or end of a run, the start or end of a subrun, and so on. In this case there is a member function, preBeginRun, that will be registered to be called at the start of each run.

The following text shows a stripped down version of the .cc file:

#include "GeometryService/inc/GeometryService.hh"
#include "art/Framework/Services/Registry/ServiceMacros.h"

namespace mu2e {

  GeometryService::GeometryService(fhicl::ParameterSet const& iPS,
                                   art::ActivityRegistry& iRegistry){
    iRegistry.watchPreBeginRun(this, &GeometryService::preBeginRun);
  }

  void GeometryService::preBeginRun(art::Run const &) {
      // Work to be done at the start of each run
  }

} // end namespace mu2e

DEFINE_ART_SERVICE(mu2e::GeometryService);

The constructor receives a parameter set as an argument and it may extract information from the parameter set to configure itself; that is not shown in this simple example. The one line in the body of the constructor registers the member function GeometryService::preBeginRun with the art activity registry; it is registered as a callback that will be called at the start of a new run. To be a little more specific, art permits services to register to be called before all of the module beginRun member functions are called and to register to be called after all of the module beginRun member functions have been called. This example registers a member function to be called before the module beginRun member functions are called, ie preBeginRun. In this example, the member function preBeginRun could have been given an arbitrary name; the information that the registration is for a preBeginRun callback is encoded in the name of the function called, iRegistry.watchPreBeginRun. The full list of watch* member functions can be found in the header file art/Framework/Services/Registry/ActivityRegistry.h. To figure out the required argument list for each type of callback function, you can stare at ActivityRegistry.h for a while; the magic is in the declarations

 typedef sigc::signal<void, Run const&> PreBeginRun;

This says that the callback function must return a void and have one argument that is of type "Run const&". It may be easier to look at the code for art/Framework/Services/Optional/Tracer_service.cc which registers for all callbacks and prints diagnostic information on each call. This service can be [[ArtServices| enabled from the command line or from the .fcl file] .

The question of whether to make a callback for preBeginRun or postBeginRun, both or neither depends on what the service is intended to do. The purpose of the GeometryService is to maintain an up to date representation of the geometry, which is permitted to change on run boundaries. Modules will want this information to be available when their beginRun methods are called. Therefore GeometryService has a preBeginRun callback.

The final part of the example is the last line that invokes the DEFINE_ART_SERVICE macro. Most Mu2e users will never need to know what this macro does. The short version is that a creates the tools that art needs to access the service via a plugin .so library, a factory method and a piece of code that, upon opening of the library, registers the factory method with an art internal registry that knows how to match the name of service to the factory method that can create an instance of that service.

A Service is not a Singleton

This section will only be of interest to advanced C++ users and not to most Mu2e people.

Getting a handle to a service feels a lot like getting an instance pointer to a singleton. While this similarity is certainly real, one should not think of services as singletons. One important difference is that services are instantiated by art and are only instantiated as they are needed; singletons, on the other hand, are instantiated by the loader in an order that may change with compiler/linker/loader/OS version; is difficult to manage a group of singletons which must be instantiated in a prescribed order.

Another difference is that art manages run-time configuration and distributes it to the services as art instantiates them. If one wanted to replace services with singletons it would be necessary to invent a new mechanism for transmitting run-time configuration to each singleton.

Finally, the art service model is ready to deal with multi-threading; some features can be global while others can be thread-local.