Computing Concepts

Introduction

This page is intended for physicists who are just starting to work in the mu2e computing environment. The following is a broad overview of the major components and how they work together. There are many links into the rest of the mu2e documentation which is more terse and is intended as a reference once you get through the introductory material. Don't worry if this page leave lots of unanswered questions - it is only an overview.

Hardware and authentication

Interactive logins

Collaborators can do interactive work in several places. Probably the best place to start is the interactive machines at Fermilab. These are set of virtual machines running SL6 (a variant of linux) . The disks that the user sees are located on specialized disk server hardware and the same disks are mounted by all the interactive machines. There are five machines named mu2egpvm01.fnal.gov through mu2egpvm05.fnal.gov. You will have an account and a home area here (the same area is on all machines) and some space for data. You can read more about the interactive machines here.

Collaborators can also compile and run mu2e code on their linux desktops or laptops. You can read more about that here. The code should be manageable on Mac's unix layer but we have not pursued this option officially. Finally, several of the collaboration institutions have set up working areas at their home institutions. For these options, someone would need to copy the code distribution and supporting software tools to the local machine, or mount the distributed code disk. You can read more about these options here, but best to discuss this with your group leaders first. Generally we recommend working on the lab interactive nodes unless the networking from your institution makes that prohibitive.

Authentication

You login to the virtual machines by using kerberos authentication. You will need a permanent ID called a kerberos "principal" which is looks like "xyz@FNAL.GOV", where xyz is your username. You will have a password associated with your principal. You will use this principal and password to log into linux desktops located at Fermilab or to ssh into the interactive machines from your home institution. You typically refresh your kerberos authentication every day.

The second identity you will need is the "services" principal, which looks like xyz@services.fnal.gov and also has a password. You will need this identity to log into Fermilab email, the servicedesk web site and some other services based at the lab. You would typically only need this authentication as you log into the service.

Finally, you will need a CILogin certificate. This cert is the basis of authentication to the mu2e documents database, the computing farms, and other services. You will use this cert in two ways. The first way is to load it into your browser, which then gives you access to web pages and web services. The second is by using your kerberos authentication to access a copy of your certificate maintained in a remote database. You get this certificate once and then renew it only once a year.

Please follow the procedure on the ComputingAccounts page to setup your accounts.

Grid resources

mu2e has access to a compute farm at Fermilab, called GPGrid. This is several thousand cores and we mu2e is allocated a portion of the nodes, and we can use more if no one else needs it. Once you have used the interactive machines to build and test your code, you can submit a large job to the compute farms using the "jobsub" system. You can get typically get 1000 nodes for day before your priority goes down and you get less. If the farm is not crowded, which is not uncommon, you can get several times that.

mu2e also has access to compute farms at other institutions through a collaboration call Open Science Grid (OSG). It is easy to modify your submit command to use these resources. We do not have a quota here, so we don't really know how many nodes we can get, but it is usually at least as much as we can get on GPGrid. This system is less reliable than GPgrid so we often see unusual failure modes, or jobs restarting.

Code

mu2e simulation, reconstruction, and analysis code is available as one git repository. The code is primarily c++, built with scons, with bash scripting, and some perl and python. Our code depends heavily on external packages such as art and root. Executables, assembled from shared objects and plug-in modules, are controlled by fcl scripts. Pre-built releases are published and available world-wide on cvmfs.

Tutorials

• Testing the ROOT display
• Testing the Geant4 based event display
• Notes on dynamic libraries
• The First Step: the art workbook
• Running G4 within art: The first examples.
• Mu2e maintained FAQs: C++ FAQ, Unix/Linux FAQ, ROOT FAQ, Geant4 Notes