Docker vs mu2egpvm: Difference between revisions

From Mu2eWiki
Jump to navigation Jump to search
(Created page with "Test")
 
 
(7 intermediate revisions by the same user not shown)
Line 1: Line 1:
Test
==Cheat Sheet==
 
In case you just need the cheat sheet, the following table shows where to find five files directories that are important to the tutorial:
 
{| class="wikitable"
|-
! Mu2e Interactive Machines
! Docker
|-
| /cvmfs/mu2e.opensciencegrid.org/setupmu2e-art.sh
| /setupmu2e-art.sh
|-
| /cvmfs/mu2e.opensciencegrid.org/artexternals
| /artexternals
|-
| /cvmfs/mu2e.opensciencegrid.org/Offline
| /Offline
|-
| /cvmfs/mu2e.opensciencegrid.org/DataFiles
| /DataFiles
|-
| /mu2e/app/Tutorial_2019
| /Tutorial_2019
|-
| /cvmfs/fermilab.opensciencegrid.org/products/common/
| /products
|}
==The Mu2e Interactive Machines==
 
*Fermilab supplies about 8 computers at Fermilab that have
**All of the software needed for Mu2e (many version of it!)
**Lots of disk space
**Lots of tape to store files that do not fit on disk
**Nightly backup of home disks
*We call these the [[ComputingLogin#Machines | Mu2e interactive machines]]
*We can ask for more machines when we demonstrate a need.
*Most of us work by logging into these machines from our laptops/desktops and doing our work there.
*You will also normally use these machines.
 
==Why Docker?==
*If we were running these tutorials at Fermilab you would do the tutorials by logging into the Mu2e interactive machines and working there.
*The network bandwidth is very likely not good enough for us to work this way when we are not on the Fermilab site.
*Our solution is to provide all of the necessary software in a Docker container that you install on your laptop.
*You will do the tutorial exercises from within the Docker container.
 
==Comparison of Environments==
 
The Mu2e environment was designed so that everything is relative to a small number of top level directories.  Everything else is designed to be location independent.  Below is a description of the small number of things that differ between the Mu2e Docker environment and the Mu2e interactive node environment.  In addition that are some resources that are simply absent from the Mu2e Docker environment.
 
 
On the Mu2e interactive machines the disk that contains the Mu2e software, plus all of the underlying tools is:
<pre>
/cvmfs/mu2e.opensciencegrid.org .
</pre>
There are is one important file and three important directories:
{| class="wikitable"
|-
| setupmu2e-art.sh
| script that initializes the Mu2e environment; this is what is run by <code>mu2einit</code>.
|-
| artexternals
| tools on which our software depends (many versions of each); some Mu2e-written some third party
|-
| Offline 
| many version of the Mu2e Offline code
|-
| DataFiles
| auxiliary files files used by Mu2e Offline
|}
 
In the Docker container, these four files/directories are found in the root directory.  Take a look:
<pre>
ls /
</pre>
You should see all four.  An important difference is that the Docker container only contains one version of everything, just the versions needed to run the tutorials.  On the other hand the /cvmfs area has many version of everything.
 
On the Mu2e interactive machines, the tools needed to [https://mu2ewiki.fnal.gov/wiki/Grids run grid jobs] and manage the file catalog, called [https://mu2ewiki.fnal.gov/wiki/SAM SAM], is found in the directory:
<pre>
/cvmfs/fermilab.opensciencegrid.org/products/common/
</pre>
These are kept in a separate place because they are managed by the Fermilab Scientific Computing Division (SCD), not by Mu2e.  In the Docker container a subset of this code is found in
<pre>
/products
</pre>
 
 
On the Mu2e interactive machines, many of the files that have been prepared for the tutorials are found in
<pre>
ls /mu2e/app/Tutorials_2019/
DataExploration  SimpleROOT          setup.sh            testDocker.sh
GeometryBrowsing  TrkAna              setup_container.sh
ModuleWriting    basic_root_tutorial  testDocker.C
RunningArt        data                testDocker.C~
</pre>
 
In the Docker container these files are found in the root directory:
<pre>
/Tutorials_2019/
</pre>

Latest revision as of 21:23, 19 July 2024

Cheat Sheet

In case you just need the cheat sheet, the following table shows where to find five files directories that are important to the tutorial:

Mu2e Interactive Machines Docker
/cvmfs/mu2e.opensciencegrid.org/setupmu2e-art.sh /setupmu2e-art.sh
/cvmfs/mu2e.opensciencegrid.org/artexternals /artexternals
/cvmfs/mu2e.opensciencegrid.org/Offline /Offline
/cvmfs/mu2e.opensciencegrid.org/DataFiles /DataFiles
/mu2e/app/Tutorial_2019 /Tutorial_2019
/cvmfs/fermilab.opensciencegrid.org/products/common/ /products

The Mu2e Interactive Machines

  • Fermilab supplies about 8 computers at Fermilab that have
    • All of the software needed for Mu2e (many version of it!)
    • Lots of disk space
    • Lots of tape to store files that do not fit on disk
    • Nightly backup of home disks
  • We call these the Mu2e interactive machines
  • We can ask for more machines when we demonstrate a need.
  • Most of us work by logging into these machines from our laptops/desktops and doing our work there.
  • You will also normally use these machines.

Why Docker?

  • If we were running these tutorials at Fermilab you would do the tutorials by logging into the Mu2e interactive machines and working there.
  • The network bandwidth is very likely not good enough for us to work this way when we are not on the Fermilab site.
  • Our solution is to provide all of the necessary software in a Docker container that you install on your laptop.
  • You will do the tutorial exercises from within the Docker container.

Comparison of Environments

The Mu2e environment was designed so that everything is relative to a small number of top level directories. Everything else is designed to be location independent. Below is a description of the small number of things that differ between the Mu2e Docker environment and the Mu2e interactive node environment. In addition that are some resources that are simply absent from the Mu2e Docker environment.


On the Mu2e interactive machines the disk that contains the Mu2e software, plus all of the underlying tools is:

 /cvmfs/mu2e.opensciencegrid.org .

There are is one important file and three important directories:

setupmu2e-art.sh script that initializes the Mu2e environment; this is what is run by mu2einit.
artexternals tools on which our software depends (many versions of each); some Mu2e-written some third party
Offline many version of the Mu2e Offline code
DataFiles auxiliary files files used by Mu2e Offline

In the Docker container, these four files/directories are found in the root directory. Take a look:

ls /

You should see all four. An important difference is that the Docker container only contains one version of everything, just the versions needed to run the tutorials. On the other hand the /cvmfs area has many version of everything.

On the Mu2e interactive machines, the tools needed to run grid jobs and manage the file catalog, called SAM, is found in the directory:

/cvmfs/fermilab.opensciencegrid.org/products/common/

These are kept in a separate place because they are managed by the Fermilab Scientific Computing Division (SCD), not by Mu2e. In the Docker container a subset of this code is found in

 /products


On the Mu2e interactive machines, many of the files that have been prepared for the tutorials are found in

ls /mu2e/app/Tutorials_2019/
DataExploration   SimpleROOT           setup.sh            testDocker.sh
GeometryBrowsing  TrkAna               setup_container.sh
ModuleWriting     basic_root_tutorial  testDocker.C
RunningArt        data                 testDocker.C~

In the Docker container these files are found in the root directory:

/Tutorials_2019/