Difference between revisions of "Workflows"
Jump to navigation
Jump to search
| Line 8: | Line 8: | ||
== Data Handling == | == Data Handling == | ||
| − | |||
| − | |||
* [[Dcache|dCache]] - the large aggregated data disk system | * [[Dcache|dCache]] - the large aggregated data disk system | ||
* [[Enstore|enstore]] - the mass storage tape system | * [[Enstore|enstore]] - the mass storage tape system | ||
| + | * [[FileNames|File names]] - how to name files for upload or for production | ||
| + | * [[FileFamilies|File families]] - the logical grouping of data on tapes | ||
* [[SAM|SAM]] - file metadata and data handling management system | * [[SAM|SAM]] - file metadata and data handling management system | ||
| − | ** [[SamExpert| | + | ** [[SamMetadata|SAM metadata]] - the metadata stored for each file |
| − | * [[Upload]] - writing files to mass storage tape | + | ** [[SamExpert|SAM expert]] - some notes for the future, not of general interest |
| − | * [[UploadFTS|FTS Upload]] | + | * [[Upload|Upload]] - writing files to mass storage tape |
| + | ** [[MCWorkflow| MC workflow]] - how to run simulation and optionally concatenate and upload the files | ||
| + | ** [[UploadWorkflow|Upload files]] - how to upload existing files | ||
| + | ** [[FileTools|File tools]] - tools for manipulating large file datasets and metadata | ||
| + | * [[UploadFTS|FTS Upload]] - deprecated method to upload by the FTS area | ||
==Operations== | ==Operations== | ||
Revision as of 18:58, 30 March 2017
Workflow for jobs
- Grids - overview of the computing farms
- Data transfer - how to move larger datasets, necessary for grid jobs
- Workflow for running grid jobs (convert to wiki)
- Prestage - make sure files have been copied off tape and ready to use
Data Handling
- dCache - the large aggregated data disk system
- enstore - the mass storage tape system
- File names - how to name files for upload or for production
- File families - the logical grouping of data on tapes
- SAM - file metadata and data handling management system
- SAM metadata - the metadata stored for each file
- SAM expert - some notes for the future, not of general interest
- Upload - writing files to mass storage tape
- MC workflow - how to run simulation and optionally concatenate and upload the files
- Upload files - how to upload existing files
- File tools - tools for manipulating large file datasets and metadata
- FTS Upload - deprecated method to upload by the FTS area