Enstore

The Scientific Computing Division maintains a system of data tapes called enstore (manual, project), which allows us to "write data to tape". The tapes are held in libraries in Feynman Cmputing Center and the Grid Computing Center building. When files are needed, a robot arm retrieves them and inserts them in a tape drive.

There are currently two types 9/2018 of tapes and we are transitioning between them.

T10K StorageTek T10000KC and T10000KD which hold 5TB and 8TB each, respectively. We currently (2017) share about 20 tape drives with all of the Intensity Frontier. On 9/28/18 our media for new files was switched to LTO8, in library CD-LTO8F1.

LTO The LTO8 drives have a quoted maximum speed between 300MB/s and 350MB/s depending on media type (as compared to the quoted maximum of 250MB/s for the T10KC drives currently used). The LTO8 tapes have a capacity of 12TB but as you know, there is a global unavailability of those tapes, so we're using "M8" (LTO7 formatted with LTO8) tapes which have a capacity of 9 TB.

We have a total of 36 drives in production now with an additional 20 to be added within the next few weeks. By the end of this calendar year, all experiments (with the exception of CMS) will be putting all new data in this library using those drives and a significant fraction of the drives will be utilized for migrating existing data from T2 media to LTO.

mu2e tapes are divided into several file families depending on the type of data: production or user data, raw or reco, beam data or sim. The files assigned to a file family will go to one set of tapes for that file family. It can be useful to segregate data like this so it can be treated specially. For example, raw data might be stored with two copies in different buildings, while this is unnecessary for sim data.

A tape drive can read at 250MB/s so one tape can be read in about 5h. In reality, tapes are rarely read all the way though - we typically access single files at a time. Typical access times are:

• 1m to find an mount the tape
• 1m to seek to the file
• 10s to read the file
• 1m to dismount and replace the tape

Once the file is off the tape, it has to be copied to tape-backed dCache. If the file is over 300MB, it was written to tape as lone file and the copy to dCache is immediate. If it is smaller, then it was rolled in a tarball with other small files in a system called Small File Aggregation (SFA). In this case, it has to be extracted fro the tarball before being written to dCache and this can add up to 15s latency.

We can mix large and small files in our data. The largest file that can be written is something like 1TB, but as a practical matter, all mu2e files should be less than about 20GB, and when there is an option, they should be 2-5 GB.

We write to tape through the tape-backed dCache. We only write files that are properly named, organized and documented by the mu2egrid scripts.

To read files from tape, we would usually access them by the mu2egrid scripts or the /pnfs mount of dCache. See also SAM to help with the file names and locations. Large numbers of files, such as those required by grid jobs, require prestaging to make sure they are off tape and on disk before reading them.

A list of complete file listings is available, updated each day (use wget, too big for a browser).