LoginTutorial: Difference between revisions

From Mu2eWiki
Jump to navigation Jump to search
No edit summary
 
(27 intermediate revisions by 3 users not shown)
Line 1: Line 1:
==Authentication and First Login==
==Authentication and First Login==
To login to the central machines at Fermilab, you will need first need your [[ComputingAccounts|accounts created]].  This process may take a day or two, and will result in three authentications:
* kerberos used for loggins into central machines, among others
* a services account for email, servicedesk and other uses
* CILogin certificate, for submitting jobs, transferring and uploading data
You can read more about this at [[Authentication]], which you should at least scan before continuing.


If you are sitting at a Fermilab linux computer, you can use your kerberos login at the login screen.  If you are at a university computer with kerberos installed, you may or may not be able to obtain your kerberos authentication.  If you are already on a Fermilab desktop or a Mu2e central computer, you can get this authentication with:
This page presumes that you have computer accounts and have completed the [[Day_1_CheckList#Computer_Accounts | the Day 1 Checklist]].   
kinit <your user name>@FNAL.GOV
If that is successful, you can login to the central systems:
  ssh <username>@mu2egpvm01.fnal.gov


If you have problems, please look through [[Authentication]] and the links it points to, in particular, if you are on a University machine, you might need to configure kerberos with a new krb5.conf file. 
# If you are sitting at a Fermilab linux computer, you can use your kerberos principal and password login at the login screen. 
# If you are sitting at your own computer or a university computer that is a unix computer or a Mac, follow the instructions at [[ComputingLogin#Logging_in_from_Linux_or_Mac.27s]].
#*If you have issues, the first two guesses are:
#** Your /etc/krb5.conf file is not compliant with Fermilab requirements. See [[ComputingLogin#Logging_in_from_Linux_or_Mac.27s]].
#** You have recently installed code from anaconda, miniconda or condaforge, which puts their kerberos in your PATH in front of the default kerberos. See [[ComputingLogin#Anaconda]].
#If you are otherwise on a Windows machine, follow the instructions at [[ComputingLogin#Logging_in_From_PC.27s]].  PuTTy allows you to login in a terminal window on the central machines, and xming allows you to display xwindows back on your laptop.


If you are on a windows laptop, you will want to install [[ComputingLogin#Logging_in_From_PC.27s|Putty and xming]].  PuTTy allows you to login in a terminal window on the central machines, and xming allows you to display xwindows back on your laptop. If you are on a Mac, see [[ComputingLogin#Logging_in_from_Linux_or_Mac.27s|Mac's]].
If you have problems the places to look are, [[ComputingLogin]], [[Authentication]], [[ErrorRecovery]] or [[ComputingHelp]].  You can also ask a colleague or post a question on the Mu2e [[Slack]] channel is_it_me_or_a_bug.


You probably want to try out your email, by going to email.fnal.gov and putting in your services password.  Some official communications are sent only to this address so check it regularly or forward it to a preferred account.
On your local machine or on the central machine, you can see the status of your kerberos ticket with the ''klist'' command:
 
OK, from here on we will assume you are on a central machine.
 
You can see your ticket with the '''klist''' command:
<pre>
<pre>
  > klist
  > klist
Line 31: Line 22:
         renew until 06/25/19 12:58:50
         renew until 06/25/19 12:58:50
</pre>
</pre>
If you stay logged on overnight, it will expire and you will need to renew it with '''kinit'''.
If you stay logged on overnight, it will expire and you will need to renew it with ''kinit''.  If you ''kinit'' on the central machine it will extend only that session.  If you  ''kinit'' on your local machine, you can start new sessions but it won't extend any existing sessions.
 
==If You Cannot Login==
 
When most people join Mu2e their accounts on the Mu2e computers are created automatically.  If you discover that you cannot login, check to see if the following applies to you.  There are several groups of people who need to take an extra step and must apply for their accounts on Mu2e computers. These people are those who:


You can use your kerberos identity to invoke your certificate identity with '''kx509'''.
* are newly hired by Fermilab, including people on the Italian summer student program.
<pre>
* already have an active FermiID when they join Mu2e.
  > kx509
* renewed an expired FermiID when they joined Mu2e and who were on another experiment prior to joining Mu2e.  During renewals, the workflow does not have a step to create computer accounts.
Authorizing ...... authorized
 
Fetching certificate ..... fetched
If you are in one of these categories, follow the instructions under "New to Mu2e" at [[ComputingAccounts]].
Storing certificate in /tmp/x509up_u1311
 
Your certificate is valid until: Tue Jun 25 13:18:18 2019
In all other cases, open a [[ServiceDesk]] ticket.
</pre>
 
 
==Check Your Default Shell ==
All shell scripts in Mu2e and all of the Mu2e examples are designed to work using the bash shell.  Fermilab computer accounts are created with the bash shell as the default.
 
Some Fermilab accounts created years ago may have a different default shell, possibly <code>tcsh</code>.  You can check which shell is active by
echo $SHELL
  /bin/bash
If your shell is not bash, open a [[ServiceDesk]] ticket and request that it be changed.  The command chsh does not work our machines because the passwd file on each machine is externally managed.
You will also want to port any customizations from your .login and .cshrc files to .my_bash_profile and .my_bashrc . See [[Shells]] to learn about these the two .my_ files.
 
== Create Your Login Scripts ==
 
After you have checked that bash is your default shell, create your login scripts <code>.bash_profile</code> and <code>.bashrc</code>. 
To do this, follow the instructions in [[Shells]]. Be sure to follow the instructions to test the your login scripts correctly setup UPS.
 
== Check setup mu2e==
 
This section is retired but the section header has been retained because there may be links to it. It's replacement is:
 
# [[Shells#Verify_that_Your_Login_Scripts_Work_Correctly]] if you are running on a AL9 machine
# [[#Legacy SL7 Environment Check]] if you are running in an SL7 container
 
==Legacy SL7 Environment Check==
 
This section is only relevant if you are running inside an SL7 container.  It shows how to test that your login scripts and <code>mu2einit</code> are working correctly.  The analog for AL9 is described in [[Shells#Verify_that_Your_Login_Scripts_Work_Correctly]].
 
After the login scripts execute, you have a minimal UPS environment, the only UPS product that has been setup is UPS itself.
> ups active
Active ups products:
ups              v6_0_7          -f Linux64bit+3.10-2.17                    -z /cvmfs/fermilab.opensciencegrid.org/products/common/db
And your environment only knows about one UPS repositories. 
> printenv PRODUCTS | tr ":" "\n"
/cvmfs/fermilab.opensciencegrid.org/products/common/db


You can see your cert with '''voms-proxy-info --all''':
The next step is to give the command:
<pre>
mu2einit
> voms-proxy-info --all
Now both UPS repositories are available in the environment.  You can see this by:
subject  : /DC=org/DC=cilogon/C=US/O=Fermi National Accelerator Laboratory/OU=People/CN=Raymond Culbertson/CN=UID:rlc
> printenv PRODUCTS | tr ":" "\n"
issuer    : /DC=org/DC=cilogon/C=US/O=CILogon/CN=CILogon Basic CA 1
/cvmfs/mu2e.opensciencegrid.org/artexternals
identity : /DC=org/DC=cilogon/C=US/O=CILogon/CN=CILogon Basic CA 1
  /cvmfs/fermilab.opensciencegrid.org/products/common/db
type      : unknown
strength  : 2048 bits
path      : /tmp/x509up_u1311
timeleft  : 167:51:31
key usage : Digital Signature, Key Encipherment, Data Encipherment
</pre>


Your certificate can be extended with information about your experiment and the roles you have access to.  This can be done with a script
The first UPS repository contains a subset of the software provided by Fermilab Computing.  The second UPS repository contains all of the Mu2e software plus additional software provided by Fermilab Computing.
/cvmfs/mu2e.opensciencegrid.org/bin/vomsCert
If you then run '''voms-proxy-info --all''' you will see your cert has gotten longer.


You will want to bring your certificate into your browsers since a few web pages, notably the Mu2e doc-db, are authenticated with the certificate.  [https://fermi.service-now.com/kb_view.do?sysparm_article=KB0011548 Download] your cert into your browsers.
The <code>mu2einit</code> command has also set a few environmental variables for you.  For example,
> printenv MU2E
/cvmfs/mu2e.opensciencegrid.org
> printenv MU2E_DATA_PATH
  /cvmfs/mu2e.opensciencegrid.org/DataFiles
These are used by the Mu2e software.


You will want to create an account on [https://mu2e-hnews.fnal.gov/HyperNews/Mu2e/top.pl hypernews] and subscribe to forums that are relevant for you.  If you are working on Mu2e software you should subscribe to the following forums: "Software and Simulation" and "Help! Is it me or a bug?". If you use STNTUPLE you will also want to sign up for that forum.  Speak with your colleagues to learn what other forums are relevant for you work.
The command has also setup two UPS more products:
# a modern version of git
# the Mu2e code build scripts, called [[Muse]]
you can see this with
> ups active
Active ups products:
git              v2_40_1        -f Linux64bit+3.10-2.17                    -z /cvmfs/mu2e.opensciencegrid.org/artexternals
muse              v4_06_00        -f NULL                                    -z /cvmfs/mu2e.opensciencegrid.org/artexternals
  ups              v6_1_1          -f Linux64bit+3.10-2.17                    -z /cvmfs/mu2e.opensciencegrid.org/artexternals
The exact versions you see will probably be different, since new versions come out fairly regularly.


Your environment is now configured to start work on Mu2e.


==Mu2e Machines==
==Mu2e Machines==
There are several Mu2e dedicated [[ComputingLogin#Machines|machines]] (actually virtual machines).
There are several dedicated [[ComputingLogin#Machines|Mu2e interactive machines]]; follow the link to see the names of the machines and details of their configuration.
* mu2egpvm01,2,3,4,5 general purpose machines with SL6 operating system
 
* mu2egpvm06 general purpose machines with SL7 operating system
You should use these machines for developing and building code, running executables for a short test jobs, making and viewing histograms, and writing notes.
* mu2ebuild01 machine to compile code
By "general purpose" we mean running executables for a short time, or making and viewing histograms, or writing notes.  These machines have 4 cores.  The build machine, mu2ebuild01 has 16 cores so it can compile the entire Mu2e code in a reasonable time, about 11 min.


==Mu2e Disks==
==Mu2e Disks==


You can read more about the [[Mu2e disks|Disks]], but in the tutorial, we give a quick tour.
Below is a quick introduction to the disk space available for Mu2e users.  When you have time, you can learn more at [[Disks]].


===home disk===
===home disk===
When you log in, your default directory will be your home area, which is the same for all the Mu2e centrl machines.  It may or may not be the same as a linux desktop you are sitting at, depending on how it was set up.
When you log in, your default directory will be your home area, which is the same for all the Mu2e central machines.  It may or may not be the same as a Fermilab linux desktop you are sitting at, depending on how that desktop  was set up.
<pre>
<pre>
  > pwd
  > pwd
Line 92: Line 125:
===app disk===
===app disk===


'''/mu2e/app''' is where you will put code that you checkout compile and develop.  You can create you own area:
'''/exp/mu2e/app''' is where you will put code that you checkout, compile and develop.  You can create you own area:
  mkdir -p /mu2e/app/users/$USER
  mkdir -p /exp/mu2e/app/users/$USER
and you should be able to write there
and you should be able to write there
  touch /mu2e/app/users/$USER/test
  touch /exp/mu2e/app/users/$USER/test
Note that you will have a finite quota on this disk, so do not put data files here (see next).
New Mu2e people will have have a quota of 25 GB on this disk.


This disk is not backed up.
This disk is not backed up to tape but it does maintain [[Disks#Snapshots]].
 
Until the summer of 2023, this area was called, '''/mu2e/app'''.  If you still have files in the old area, please delete files that are no longer needed and move the remaining files to your space on /exp/mu2e/app.  If you had a larger quota on '''/mu2e/app''', you will also have it on '''/exp/mu2e/app'''.


===data disk===
===data disk===
'''/mu2e/data''' is where you can put data files and ntuples that you are using now.  You can create you own area:
'''/exp/mu2e/data''' is where you can put data files and ntuples that you are using now.  You can create you own area:
  mkdir -p /mu2e/data/users/$USER
  mkdir -p /exp/mu2e/data/users/$USER
and you should be able to write there
and you should be able to write there
  touch /mu2e/data/users/$USER/test
  touch /exp/mu2e/data/users/$USER/test
This space is limited, so if you are using more than 500 GB or the files are sitting there for more than a few months, you should consider putting the files on tape or dCache (see next).
New Mu2e people will have have a quota of 150 GB on this disk.  


This disk is not backed up.
This disk is not backed up.
Until the summer of 2023, this area was called, '''/mu2e/data'''.  If you still have files in the old area, please delete files that are no longer needed and move the remaining files to your space on /exp/mu2e/data.  If you had a larger quota on '''/mu2e/data''', you will also have it on '''/exp/mu2e/data'''.


===dCache===
===dCache===
Line 138: Line 175:


===cvmfs===
===cvmfs===
We store code on the cvmfs disk.  software packages from outside Mu2e:
We store code on the /cvmfs disk.  Software packages from outside Mu2e are stored at:
  ls -l /cvmfs/mu2e.opensciencegrid.org/artexternals
  ls -l /cvmfs/mu2e.opensciencegrid.org/artexternals
and pre-built Mu2e code versions:  
Pre-built Mu2e code versions:  
  ls -l /cvmfs/mu2e.opensciencegrid.org/Offline
  ls -l /cvmfs/mu2e.opensciencegrid.org/Musings
cvmfs acts like a ready-only disk.  It is actually a set of clients, servers, and databases.  Anywhere the client is run, the identical /cvmfs disk appears on the node and can be read by the users.  So this is perfect for distributing code to interactive machines, grid worker nodes, and universities.
Our magentic field maps are at:
ls -l /cvmfs/mu2e.opensciencegrid.org/DataFiles/BFieldMaps
 
cvmfs acts like a ready-only disk.  It is actually a set of clients, servers, and databases.  Anywhere the client is run, the identical /cvmfs disk appears on the node and can be read by the users.  So this is perfect for distributing code to interactive machines, grid worker nodes, and universities.  You can even mount it on your laptop.  If you set your your local cache is large enough, you can pre-fill the cache with everything you need to work without a network.


==Setup Code==
You can read me at the [[Cvmfs]] wiki page.
There are two steps to accessing Mu2e code.  The first step is to define all the supporting software packages that are available:
source /cvmfs/mu2e.opensciencegrid.org/setupmu2e-art.sh
The main effect is to define PRODUCTS environmental variable:
<pre>
> printenv PRODUCTS | tr ":" "\n"
/cvmfs/mu2e.opensciencegrid.org/artexternals
/cvmfs/fermilab.opensciencegrid.org/products/common/db
</pre>
all the support code packages are under these directories.


The second step is to setup access to a specific version of the Mu2e code.  This code does all our detector simulation and reconstruction and calls code in the supporting packages.  As the code is developed, we occasionally freeze the code into a fixed version.  The particular version could be from cvmfs:
==Proxies==
<pre>
ls /cvmfs/mu2e.opensciencegrid.org/Offline | tail -3
v7_3_5
v7_4_0
v7_4_1
</pre>


These are versions that you can setup.  To do that you source a setup file like:
You will need proxies to access some utilities provided by Fermilab Computing.  The instructions in this section explain how to get a proxy and how to verify that it is present in your environment.
source /cvmfs/mu2e.opensciencegrid.org/Offline/v7_4_1/SLF6/prof/Offline/setup.sh
You can see the additional directories include a choice of operating system (SL6 or 7, see above) and prof or debug.  '''prof''' refers to the compiled with optimization and '''debug''' refers to the same code compiled with optimization turned off which makes it easier to debug some kinds of errors.


If you are developing Mu2e code, you can setup and build the Mu2e code in your local area - more on that later.  You can only setup a version of the code once in a process. If you try to setup the code again, it will give you a warning and exit.
You can use your kerberos identity to invoke your certificate identity with '''kx509'''.
<pre>
<pre>
  > source /cvmfs/mu2e.opensciencegrid.org/Offline/v7_4_1/SLF6/prof/Offline/setup.sh
  > kx509
mu2egpvm01 ~ > source /cvmfs/mu2e.opensciencegrid.org/Offline/v7_4_1/SLF6/prof/Offline/setup.sh
Authorizing ...... authorized
A base release has already been setup. Hope that's OK.
Fetching certificate ..... fetched
The base release is:  /cvmfs/mu2e.opensciencegrid.org/Offline/v7_4_1/SLF6/prof/Offline
Storing certificate in /tmp/x509up_u1311
Your certificate is valid until: Tue Jun 25 13:18:18 2019
</pre>
</pre>
The original setup is still good.  If you need to run a different setup, you need to do that in a new window.


A simple way to see what is setup is to use the command "proff"
You can see your cert with '''voms-proxy-info --all''' :
<pre>
<pre>
> proff
> voms-proxy-info --all
subject  : /DC=org/DC=cilogon/C=US/O=Fermi National Accelerator Laboratory/OU=People/CN=Raymond Culbertson/CN=UID:rlc
issuer    : /DC=org/DC=cilogon/C=US/O=CILogon/CN=CILogon Basic CA 1
identity  : /DC=org/DC=cilogon/C=US/O=CILogon/CN=CILogon Basic CA 1
type      : unknown
strength  : 2048 bits
path      : /tmp/x509up_u1311
timeleft  : 167:51:31
key usage : Digital Signature, Key Encipherment, Data Encipherment
</pre>


MU2E_BASE_RELEASE:
Your certificate can be extended with information about your experiment and the roles you have access to.  This can be done with a script
  /cvmfs/mu2e.opensciencegrid.org/Offline/v7_4_1/SLF6/prof/Offline
/cvmfs/mu2e.opensciencegrid.org/bin/vomsCert
If you then run '''voms-proxy-info --all''' you will see your cert has gotten longer. 


release buildopts:
build=prof
g4vis=none
g4mt=on
trigger=off
MU2E_SATELLITE_RELEASE:
 
MU2E_SEARCH_PATH:
  /cvmfs/mu2e.opensciencegrid.org/Offline/v7_4_1/SLF6/prof/Offline
  /cvmfs/mu2e.opensciencegrid.org/DataFiles
FHICL_FILE_PATH:
  /cvmfs/mu2e.opensciencegrid.org/Offline/v7_4_1/SLF6/prof/Offline
</pre>
all of this will become clear in following tutorials.


[[Category:Computing]]
[[Category:Computing]]
[[Category:Tutorial]]
[[Category:Tutorial]]

Latest revision as of 18:21, 26 September 2024

Authentication and First Login

This page presumes that you have computer accounts and have completed the the Day 1 Checklist.

  1. If you are sitting at a Fermilab linux computer, you can use your kerberos principal and password login at the login screen.
  2. If you are sitting at your own computer or a university computer that is a unix computer or a Mac, follow the instructions at ComputingLogin#Logging_in_from_Linux_or_Mac.27s.
  3. If you are otherwise on a Windows machine, follow the instructions at ComputingLogin#Logging_in_From_PC.27s. PuTTy allows you to login in a terminal window on the central machines, and xming allows you to display xwindows back on your laptop.

If you have problems the places to look are, ComputingLogin, Authentication, ErrorRecovery or ComputingHelp. You can also ask a colleague or post a question on the Mu2e Slack channel is_it_me_or_a_bug.

On your local machine or on the central machine, you can see the status of your kerberos ticket with the klist command: 

 > klist
Ticket cache: FILE:/tmp/krb5cc_1311_xShHU10541
Default principal: rlc@FNAL.GOV

Valid starting     Expires            Service principal
06/18/19 12:58:54  06/19/19 14:58:50  krbtgt/FNAL.GOV@FNAL.GOV
        renew until 06/25/19 12:58:50

If you stay logged on overnight, it will expire and you will need to renew it with kinit. If you kinit on the central machine it will extend only that session. If you kinit on your local machine, you can start new sessions but it won't extend any existing sessions.

If You Cannot Login

When most people join Mu2e their accounts on the Mu2e computers are created automatically. If you discover that you cannot login, check to see if the following applies to you. There are several groups of people who need to take an extra step and must apply for their accounts on Mu2e computers. These people are those who:

  • are newly hired by Fermilab, including people on the Italian summer student program.
  • already have an active FermiID when they join Mu2e.
  • renewed an expired FermiID when they joined Mu2e and who were on another experiment prior to joining Mu2e. During renewals, the workflow does not have a step to create computer accounts.

If you are in one of these categories, follow the instructions under "New to Mu2e" at ComputingAccounts.

In all other cases, open a ServiceDesk ticket.


Check Your Default Shell

All shell scripts in Mu2e and all of the Mu2e examples are designed to work using the bash shell. Fermilab computer accounts are created with the bash shell as the default.

Some Fermilab accounts created years ago may have a different default shell, possibly tcsh. You can check which shell is active by 

echo $SHELL
/bin/bash

If your shell is not bash, open a ServiceDesk ticket and request that it be changed. The command chsh does not work our machines because the passwd file on each machine is externally managed. You will also want to port any customizations from your .login and .cshrc files to .my_bash_profile and .my_bashrc . See Shells to learn about these the two .my_ files.

Create Your Login Scripts

After you have checked that bash is your default shell, create your login scripts .bash_profile and .bashrc. To do this, follow the instructions in Shells. Be sure to follow the instructions to test the your login scripts correctly setup UPS.

Check setup mu2e

This section is retired but the section header has been retained because there may be links to it. It's replacement is:

  1. Shells#Verify_that_Your_Login_Scripts_Work_Correctly if you are running on a AL9 machine
  2. #Legacy SL7 Environment Check if you are running in an SL7 container

Legacy SL7 Environment Check

This section is only relevant if you are running inside an SL7 container. It shows how to test that your login scripts and mu2einit are working correctly. The analog for AL9 is described in Shells#Verify_that_Your_Login_Scripts_Work_Correctly.

After the login scripts execute, you have a minimal UPS environment, the only UPS product that has been setup is UPS itself. 

> ups active
Active ups products:
ups               v6_0_7          -f Linux64bit+3.10-2.17                    -z /cvmfs/fermilab.opensciencegrid.org/products/common/db

And your environment only knows about one UPS repositories. 

> printenv PRODUCTS | tr ":" "\n"
/cvmfs/fermilab.opensciencegrid.org/products/common/db

The next step is to give the command: 

mu2einit

Now both UPS repositories are available in the environment. You can see this by: 

> printenv PRODUCTS | tr ":" "\n"
/cvmfs/mu2e.opensciencegrid.org/artexternals
/cvmfs/fermilab.opensciencegrid.org/products/common/db

The first UPS repository contains a subset of the software provided by Fermilab Computing. The second UPS repository contains all of the Mu2e software plus additional software provided by Fermilab Computing.

The mu2einit command has also set a few environmental variables for you. For example, 

> printenv MU2E
/cvmfs/mu2e.opensciencegrid.org
> printenv MU2E_DATA_PATH
/cvmfs/mu2e.opensciencegrid.org/DataFiles

These are used by the Mu2e software.

The command has also setup two UPS more products:

  1. a modern version of git
  2. the Mu2e code build scripts, called Muse

you can see this with 

> ups active
Active ups products:
git               v2_40_1         -f Linux64bit+3.10-2.17                    -z /cvmfs/mu2e.opensciencegrid.org/artexternals
muse              v4_06_00        -f NULL                                    -z /cvmfs/mu2e.opensciencegrid.org/artexternals
ups               v6_1_1          -f Linux64bit+3.10-2.17                    -z /cvmfs/mu2e.opensciencegrid.org/artexternals

The exact versions you see will probably be different, since new versions come out fairly regularly.

Your environment is now configured to start work on Mu2e.

Mu2e Machines

There are several dedicated Mu2e interactive machines; follow the link to see the names of the machines and details of their configuration.

You should use these machines for developing and building code, running executables for a short test jobs, making and viewing histograms, and writing notes.

Mu2e Disks

Below is a quick introduction to the disk space available for Mu2e users. When you have time, you can learn more at Disks.

home disk

When you log in, your default directory will be your home area, which is the same for all the Mu2e central machines. It may or may not be the same as a Fermilab linux desktop you are sitting at, depending on how that desktop was set up. 

 > pwd
/nashome/r/rlc

We recommend you use the bash shell, which you should get by default: 

 > echo $SHELL
/bin/bash

to do only minor configuration of your preferences, such and alias for convenience. We have some recommendations

This disk is backed up daily.

app disk

/exp/mu2e/app is where you will put code that you checkout, compile and develop. You can create you own area: 

mkdir -p /exp/mu2e/app/users/$USER

and you should be able to write there 

touch /exp/mu2e/app/users/$USER/test

New Mu2e people will have have a quota of 25 GB on this disk.

This disk is not backed up to tape but it does maintain Disks#Snapshots.

Until the summer of 2023, this area was called, /mu2e/app. If you still have files in the old area, please delete files that are no longer needed and move the remaining files to your space on /exp/mu2e/app. If you had a larger quota on /mu2e/app, you will also have it on /exp/mu2e/app.

data disk

/exp/mu2e/data is where you can put data files and ntuples that you are using now. You can create you own area: 

mkdir -p /exp/mu2e/data/users/$USER

and you should be able to write there 

touch /exp/mu2e/data/users/$USER/test

New Mu2e people will have have a quota of 150 GB on this disk.

This disk is not backed up.

Until the summer of 2023, this area was called, /mu2e/data. If you still have files in the old area, please delete files that are no longer needed and move the remaining files to your space on /exp/mu2e/data. If you had a larger quota on /mu2e/data, you will also have it on /exp/mu2e/data.

dCache

dCache is a system of software, databases, and disk servers that are design to present a huge amount of disk, spread over many machines, as a single simple disk system. We will cover these area in more depth in future tutorials, but as an overview, dCache has three main parts:

  • tape-backed If files are written here, they are copied to tape. These file might disappear off disk if they are not used, and can be restored to disk from tape, as needed. You will only write here using specific tools.
> ls -l /pnfs/mu2e/tape/phy-sim/sim/mu2e/DS-beam/MDC2018a/art/ff/f7/sim.mu2e.DS-beam.MDC2018a.001002_00373598.art 
-rw-r--r-- 1 mu2epro mu2e 34186158 May 24  2018 /pnfs/mu2e/tape/phy-sim/sim/mu2e/DS-beam/MDC2018a/art/ff/f7/sim.mu2e.DS-beam.MDC2018a.001002_00373598.art
  • persistant Files written here are not automatically purged and are not copied to tape, so the space must be managed. Currenlty we only allow writing here in specific cases.
> ls -l /pnfs/mu2e/persistent/datasets/phy-etc/cnf/mu2e/CeEndpoint/MDC2018b/fcl/00/04/cnf.mu2e.CeEndpoint.MDC2018b.001002_00000025.fcl 
-rw-r--r-- 1 mu2epro mu2e 690 Aug 27  2018 /pnfs/mu2e/persistent/datasets/phy-etc/cnf/mu2e/CeEndpoint/MDC2018b/fcl/00/04/cnf.mu2e.CeEndpoint.MDC2018b.001002_00000025.fcl
  • scratch You can write as much data here as you would like. As space is needed, file that have not be recently accessed will be deleted to make room. You can expect untouched files to last a month, but we have seen shorter times, under special circumstances.

Create your scratch directory: 

mkdir -p /pnfs/mu2e/scratch/users/$USER

and you should be able to write there 

touch /pnfs/mu2e/scratch/users/$USER/test

and copy a file there: 

cp /pnfs/mu2e/persistent/datasets/phy-etc/cnf/mu2e/CeEndpoint/MDC2018b/fcl/00/04/cnf.mu2e.CeEndpoint.MDC2018b.001002_00000025.fcl \
 /pnfs/mu2e/scratch/users/$USER

There a are few important notes about dCache you should know right from the beginning.

  • dCache is not a file system - it is an nfs server backed by a database. This means simple disk commands such as "find" can cause millions of database queries and try up resources for hours. Only explore one directory at a time. Even using "ls" and not "ls -l" can be much faster.
  • You cannot modify files on dCache, only write and read them. If you open a dCache file with an editor, it will come up read-only. The system is designed to be a large data cache, not an interactive disk. All your code should be in your home area or /mu2e/app.

cvmfs

We store code on the /cvmfs disk. Software packages from outside Mu2e are stored at: 

ls -l /cvmfs/mu2e.opensciencegrid.org/artexternals

Pre-built Mu2e code versions: 

ls -l /cvmfs/mu2e.opensciencegrid.org/Musings

Our magentic field maps are at: 

ls -l /cvmfs/mu2e.opensciencegrid.org/DataFiles/BFieldMaps

cvmfs acts like a ready-only disk. It is actually a set of clients, servers, and databases. Anywhere the client is run, the identical /cvmfs disk appears on the node and can be read by the users. So this is perfect for distributing code to interactive machines, grid worker nodes, and universities. You can even mount it on your laptop. If you set your your local cache is large enough, you can pre-fill the cache with everything you need to work without a network.

You can read me at the Cvmfs wiki page.

Proxies

You will need proxies to access some utilities provided by Fermilab Computing. The instructions in this section explain how to get a proxy and how to verify that it is present in your environment.

You can use your kerberos identity to invoke your certificate identity with kx509. 

 > kx509
Authorizing ...... authorized
Fetching certificate ..... fetched
Storing certificate in /tmp/x509up_u1311
Your certificate is valid until: Tue Jun 25 13:18:18 2019

You can see your cert with voms-proxy-info --all : 

> voms-proxy-info --all
subject   : /DC=org/DC=cilogon/C=US/O=Fermi National Accelerator Laboratory/OU=People/CN=Raymond Culbertson/CN=UID:rlc
issuer    : /DC=org/DC=cilogon/C=US/O=CILogon/CN=CILogon Basic CA 1
identity  : /DC=org/DC=cilogon/C=US/O=CILogon/CN=CILogon Basic CA 1
type      : unknown
strength  : 2048 bits
path      : /tmp/x509up_u1311
timeleft  : 167:51:31
key usage : Digital Signature, Key Encipherment, Data Encipherment

Your certificate can be extended with information about your experiment and the roles you have access to. This can be done with a script 

/cvmfs/mu2e.opensciencegrid.org/bin/vomsCert

If you then run voms-proxy-info --all you will see your cert has gotten longer.

Retrieved from "http://mu2ewiki.fnal.gov/w/index.php?title=LoginTutorial&oldid=26070"