Difference between revisions of "Workflows"

From Mu2eWiki
Jump to navigation Jump to search
 
Line 13: Line 13:
 
* [[SimulationFCL]] - an overview how MDC2020 fcl works
 
* [[SimulationFCL]] - an overview how MDC2020 fcl works
 
* [[GenerateFcl]] - generate fcl for simulation jobs
 
* [[GenerateFcl]] - generate fcl for simulation jobs
 +
* [[JustInTimeFcl]] - a technique for simplifying fcl generation
 
* [[SubmitJobs]] - submit, monitor and recover jobs in the workflow
 
* [[SubmitJobs]] - submit, monitor and recover jobs in the workflow
* [[JustInTimeFcl]] - a technique for simplifying fcl generation
 
 
* [[Concatenate]] - merging root and art files
 
* [[Concatenate]] - merging root and art files
 
* [[Upload]] - upload files to tape
 
* [[Upload]] - upload files to tape

Latest revision as of 15:32, 18 April 2024


Workflow for jobs

Data Handling

  • dCache - the large aggregated data disk system
  • enstore - the mass storage tape system
  • File Transfer Service (FTS) - service to create SAM records and copy files to tape
  • Raw Data Mover (RDM) - Mu2e service to copy teststand data files to tape
  • Data transfer - how to move larger datasets, necessary for grid jobs
  • File names - how to name files for upload or for production
  • File families - the logical grouping of data on tapes
  • SAM - file metadata and data handling management system
    • SAM metadata - the metadata stored for each file
    • SAM expert - some notes for the future, not of general interest
  • File tools - tools for manipulating large file datasets and metadata
  • FTS Upload - deprecated method to upload by the FTS area
  • Rucio - next-gen data handling

Operations