FileFamilies: Difference between revisions

From Mu2eWiki
Jump to navigation Jump to search
(Created page with " ==Introduction== A file family is a set of files which are grouped exclusively on the same set of tapes. File families are used to indicate files that may be treated diff...")
 
No edit summary
Line 48: Line 48:
the file family.  Outside of collaboration production,  user will probably only need usr-sim  
the file family.  Outside of collaboration production,  user will probably only need usr-sim  
(for Monte Carlo art files) usr-nts for ntuples and usr-etc for tarballs and anything else.
(for Monte Carlo art files) usr-nts for ntuples and usr-etc for tarballs and anything else.
[[Category:Computing]]
[[Category:Computing/Workflow]]

Revision as of 22:20, 27 March 2017

Introduction

A file family is a set of files which are grouped exclusively on the same set of tapes. File families are used to indicate files that may be treated differently during data-handling operations. This might include tape library location, groupings for migration, deletion, or copy offsite, groupings for access priority or dCache location or lifetime on disk. For example, we expect to group raw data, reconstructed data, and simulations on different sets of tapes.

List

Here are the mu2e file families. You should be familiar with this list if you are uploading files to tape, but the general user reading files does not need to understand this list.

  • phy-sim Monte Carlo simulated or reconstructed art files. These are official collaboration samples only, originated, produced, validated, and documented by physics groups intended for long-term use by many collabrators. Examples are the TDR and CD3 samples. The username associated with the files will be the production username "mu2e".
  • phy-nts non-art format ntuples of phy-sim
  • phy-etc configuration files, tarballs of log files, backups, and other files
  • usr-sim Monte Carlo simulated or reconstructed art files. These samples are produced by one or a few individuals for use in their personal studies. They are probably for short-term use, not documented publically, and not used by many collaborators. The username associated with these files will be the person most likely to understand how they were created and how they should be used if questions come up a year or two later - the intellectual owner of the data.
  • usr-nts Non-art format ntuples of usr-sim
  • usr-etc Other user-created tarballs of log files, backups
  • tst-cos Testbeam and cosmic data created before commissioning. This would include raw data formats as well as various possible derived formats and tarballs

For real data taking, more file families will be created to hold raw data, reconstructed data, and ntuples, etc.

When uploading files, you will need to specify the file family. Outside of collaboration production, user will probably only need usr-sim (for Monte Carlo art files) usr-nts for ntuples and usr-etc for tarballs and anything else.