SchemaEvolution

Introduction

From time to time an Mu2e art product needs to be updated. The members of a class may be deleted, added or transformed. In this case, we could introduce a new product and keep it separate from the old product. If the redesign is extreme, this might make sense. More often, the change is minimal, with only a member or two affected. In this case we would like to let the old product evolve into the new version.

We will assume the following:

• old release will keep writing the old version
• new releases will want to read the old or new version
• new releases will only have the new version in memory and only write the new version

There are two main cases, which follow

Variable Movement

The simplest case is adding new variables with unique new names or removing variables from a class. When root writes a dictionary of a class, it examines the code and writes a summary of the variable names and types. This list is then stored in the art file with the binary instance of the class. If the change adds, deletes or moves these variables, it can follow the changes through the dictionary. For example if it sees the object on disk was declared as

class : {
  int va;
  int vb;
  int vc;
}

and it sees the same class in memory has structure:

class : {
  int vb;
  int va;
  float vd
  float vc;
}

then, after reading the object from disk, it will know to swap the memory for va and vb and make the conversion of int vc to float vc via a c++ assignment statement. The way to think of it is that root is following the variable names through the dictionary. The variable vd was added in the memory version and this memory space will not be zeroed by root. It may be adequate to zero it in the constructor. It can be reliably zeroed in xml files, as described below.

We do not need to do anything for this schema evolution to work correctly in all cases.

Transformation of Variables

Sometimes the variable needs to be transformed on input. For example, supposed in the old class there was a flag containing two bits and in the new class these are stored as two bools. root can't keep track of this by following the variable names, so we have to tell it what to do. The solution is to write a code snippet, in this case, something like

bool1 = ((flag&1)!=0)
bool2 = ((flag&2)!=0)

and inserting it in the classes_def.xml file, in the stanza for the object which requires it. You can put any amount of code here, even include files. The code is included into the root dictionary code that reads the object. You can define the transformation to apply to only certain versions of the class (version of input class or the class in memory). It appears to even be possible to read class A into class B or class A into classes B and C.

Here are the specific art instructions for inserting the transformation code.

Versions of classes

It is possible to assign a version number to each class in the classes_def.xml file. If class T changes, you need to increment the version for class T only, but not the classes that include this class as a member or by template. Versioning only becomes a convenience if a class is requiring frequent updates. We don't expect that this will be needed, except in a few cases. If a version is added to a class, it should start with version 10 (root uses lower version numbers when reading unversioned classes). All classes that include templates should be versioned with a class method in the header, not in the xml file. This allows the version number to follow in all template instantiations. Details in the art instructions.

If class A contains class B as a member, and class B is modified and has its version incremented, we do not need to change the version number of class A.

The class checksum is not sensitive to changes in the class methods, only the members and inheritance, so we do not need to increment the version with a change of interface (thought it does seem to be recommended in the art document).

Notes on classes files

When an object is written to a root file, it needs a dictionary (metadata about the class members) to be interpreted. The dictionary is made by genreflex, triggered off two files

classes.h
classes_def.xml

The classes.h simply includes the files that define the objects to be written. Then classes_def.xml contains pseudo-xml code defining the dictionaries to create.

Start with the class itself:

<class name="mu2e::CaloRecoDigi"/>

Add any classes that are data members of the main class:

 <class name="mu2e::CaloChannelId"/>

Add the object which is the container of the objects:

<class name="mu2e::CaloRecoDigiCollection"/>

If any product will contain an art::Ptr to this class, then add that object:

<class name="art::Ptr<mu2e::CaloRecoDigi>"/>

And if this or any other product will contain a vector of art::Ptr, include that:

<class name="std::vector<art::Ptr<mu2e::CaloRecoDigi> >"/>

Finally, for the object as it is actually "art::Event::put()" into the event, also define an art::Wrapper

<class name="art::Wrapper<mu2e::CaloRecoDigiCollection>"/>

Since products are simple containers that may be accessed many places, they should be at the bottom of the link order, preferably in one of the Products directories.