Scisoft: Difference between revisions

From Mu2eWiki
Jump to navigation Jump to search
(Created page with "==Introduction== [http://scisoft.fnal.gov scisoft.fnal.gov] is a server maintained by the art team (currently mostly Lynn Garren). It contains * a set of pre-built products i...")
 
No edit summary
 
(5 intermediate revisions by 2 users not shown)
Line 1: Line 1:
==Introduction==
==Introduction==
[http://scisoft.fnal.gov scisoft.fnal.gov] is a server maintained by the art team (currently mostly Lynn Garren).  It contains
[https://scisoft.fnal.gov scisoft.fnal.gov] is a server maintained by the art team (currently mostly Lynn Garren).  It contains
* a set of pre-built products in ready-install tarballs, many versions and flavors
* a set of pre-built products in ready-install tarballs, many versions and flavors
* a set of manifests whicah are lists of coordinated sets of products.  There are general manifests and experiment-specific manifests.
* a set of manifests whicah are lists of coordinated sets of products.  There are general manifests and experiment-specific manifests.
The products are all stored as [[UPS|relocatable UPS]], so they can be simply unpacked into the products directory.
==Download==
==Download==
mu2e uses the '''mu''' manifest which is maintained by the art team for g-2 and mu2e, and the '''mu2e''' manifest, which adds a few products specific to mu2e only, and is maintained by mu2e offline.  A complete software download would include both manifests.  These manifests are updated as needed, mostly driven by new art versions. They do not include a set of more stable, general purpose products we usually setup from [[Cvmfs|/cvmfs/fermilab.opensciencegrid.org]] or a few products maintained by hand on cvmfs (such as mu2egrid).
mu2e uses the [https://scisoft.fnal.gov/scisoft/bundles/mu/ '''mu''' manifest] which is maintained by the art team for g-2 and mu2e, and the [https://scisoft.fnal.gov/scisoft/bundles/mu2e/ '''mu2e''' manifest], which adds a few products specific to mu2e only, and is maintained by mu2e offline.  A complete software download would include both manifests.  These manifests are updated as needed, mostly driven by new art versions. They do not include a set of more stable, general purpose products we usually setup from [[Cvmfs|/cvmfs/fermilab.opensciencegrid.org]] or a few products maintained by hand on cvmfs (such as mu2egrid).


manifests are labeled by a version and  
manifests are labeled by a version (the art version in our case) and  
[https://cdcvs.fnal.gov/redmine/projects/cet-is-public/wiki/AboutQualifiers qualifiers] for the OS, compiler and art version.
[https://cdcvs.fnal.gov/redmine/projects/cet-is-public/wiki/AboutQualifiers flags] for the OS, compiler, art version, and prof/debug.
Here is a list of the [https://cdcvs.fnal.gov/redmine/projects/cet-is/wiki/Supported_platforms supported platforms] from the [https://cdcvs.fnal.gov/redmine/projects/art/wiki art documentation].
Here is a list of the [https://cdcvs.fnal.gov/redmine/projects/cet-is/wiki/Supported_platforms supported platforms] from the [https://cdcvs.fnal.gov/redmine/projects/art/wiki art documentation].
To download and unpack a complete set of products on a manifest use the [https://scisoft.fnal.gov/scisoft/bundles/tools/pullProducts pullProducts] script.  The available version can be seen from browsing the [https://scisoft.fnal.gov/ manifests]
wget https://scisoft.fnal.gov/scisoft/bundles/tools/pullProducts
chmod a+x pullProducts
  pullProducts <options> <product_topdir> <OS> <bundle-spec> <qual_set> <build-spec>
./pullProducts /path/artexternals slf6 mu-v1_17_02 s20-e9 prof
This would typically be run for prof and debug, on each platform.
To download and install a single product use [https://scisoft.fnal.gov/scisoft/bundles/tools/pullPackage pullPackage].
wget https://scisoft.fnal.gov/scisoft/bundles/tools/pullPackage
chmod a+x pullPackage
pullPackage <options> <product_topdir> <OS> <prod-spec> [<qual_set> [<build-spec>]]
./pullPackage /path/artexternals slf6 ifdhc-v2_1_0 e14-p2713d prof


==Upload==
==Upload==


> https://cdcvs.fnal.gov/redmine/projects/scisoft/wiki/SciSoft
Currently (fall 2017) mu2e can upload these:
* mu2e manifests
* BTrk product
* mu2e_artdaq_core product
 
The current procedure is to take a mu manifest, add the above two products, and create a mu2e manifest.  The mu2e manifest adds a version number to the mu manifest in case it needs to be updated while still based on the same parent mu manifest.  For example here is a mu2e manifest:
mu2e-2.07.03a.0-Linux64bit+2.6-2.12-s50-e14-prof_MANIFEST.txt
It has version 2.07.03a.0, derived from mu 2.07.03a.
 
There are [https://cdcvs.fnal.gov/redmine/projects/scisoft/wiki/SciSoft instructions] on how to use the upload script
[https://scisoft.fnal.gov/scisoft/bundles/tools/copyToSciSoft copyToSciSoft].
 
12/3/20 - Lynn upgraded the upload portal machine, now we have to do:
12/7/20 - this node seems to be made default
 
  ./copyToSciSoft  --node scisoftgpvm01.fnal.gov *.bz2
 
 
[[Category:Computing]]
[[Category:Code]]
[[Category:Infrastructure]]

Latest revision as of 18:38, 7 December 2020

Introduction

scisoft.fnal.gov is a server maintained by the art team (currently mostly Lynn Garren). It contains

  • a set of pre-built products in ready-install tarballs, many versions and flavors
  • a set of manifests whicah are lists of coordinated sets of products. There are general manifests and experiment-specific manifests.

The products are all stored as relocatable UPS, so they can be simply unpacked into the products directory.

Download

mu2e uses the mu manifest which is maintained by the art team for g-2 and mu2e, and the mu2e manifest, which adds a few products specific to mu2e only, and is maintained by mu2e offline. A complete software download would include both manifests. These manifests are updated as needed, mostly driven by new art versions. They do not include a set of more stable, general purpose products we usually setup from /cvmfs/fermilab.opensciencegrid.org or a few products maintained by hand on cvmfs (such as mu2egrid).

manifests are labeled by a version (the art version in our case) and flags for the OS, compiler, art version, and prof/debug. Here is a list of the supported platforms from the art documentation.

To download and unpack a complete set of products on a manifest use the pullProducts script. The available version can be seen from browsing the manifests

wget https://scisoft.fnal.gov/scisoft/bundles/tools/pullProducts
chmod a+x pullProducts
 pullProducts <options> <product_topdir> <OS> <bundle-spec> <qual_set> <build-spec>
./pullProducts /path/artexternals slf6 mu-v1_17_02 s20-e9 prof

This would typically be run for prof and debug, on each platform.

To download and install a single product use pullPackage.

wget https://scisoft.fnal.gov/scisoft/bundles/tools/pullPackage
chmod a+x pullPackage
pullPackage <options> <product_topdir> <OS> <prod-spec> [<qual_set> [<build-spec>]]
./pullPackage /path/artexternals slf6 ifdhc-v2_1_0 e14-p2713d prof

Upload

Currently (fall 2017) mu2e can upload these:

  • mu2e manifests
  • BTrk product
  • mu2e_artdaq_core product

The current procedure is to take a mu manifest, add the above two products, and create a mu2e manifest. The mu2e manifest adds a version number to the mu manifest in case it needs to be updated while still based on the same parent mu manifest. For example here is a mu2e manifest:

mu2e-2.07.03a.0-Linux64bit+2.6-2.12-s50-e14-prof_MANIFEST.txt

It has version 2.07.03a.0, derived from mu 2.07.03a.

There are instructions on how to use the upload script copyToSciSoft.

12/3/20 - Lynn upgraded the upload portal machine, now we have to do: 12/7/20 - this node seems to be made default

 ./copyToSciSoft  --node scisoftgpvm01.fnal.gov *.bz2