Scisoft: Difference between revisions
(Created page with "==Introduction== [http://scisoft.fnal.gov scisoft.fnal.gov] is a server maintained by the art team (currently mostly Lynn Garren). It contains * a set of pre-built products i...") |
No edit summary |
||
(5 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
==Introduction== | ==Introduction== | ||
[ | [https://scisoft.fnal.gov scisoft.fnal.gov] is a server maintained by the art team (currently mostly Lynn Garren). It contains | ||
* a set of pre-built products in ready-install tarballs, many versions and flavors | * a set of pre-built products in ready-install tarballs, many versions and flavors | ||
* a set of manifests whicah are lists of coordinated sets of products. There are general manifests and experiment-specific manifests. | * a set of manifests whicah are lists of coordinated sets of products. There are general manifests and experiment-specific manifests. | ||
The products are all stored as [[UPS|relocatable UPS]], so they can be simply unpacked into the products directory. | |||
==Download== | ==Download== | ||
mu2e uses the '''mu''' manifest which is maintained by the art team for g-2 and mu2e, and the '''mu2e''' manifest, which adds a few products specific to mu2e only, and is maintained by mu2e offline. A complete software download would include both manifests. These manifests are updated as needed, mostly driven by new art versions. They do not include a set of more stable, general purpose products we usually setup from [[Cvmfs|/cvmfs/fermilab.opensciencegrid.org]] or a few products maintained by hand on cvmfs (such as mu2egrid). | mu2e uses the [https://scisoft.fnal.gov/scisoft/bundles/mu/ '''mu''' manifest] which is maintained by the art team for g-2 and mu2e, and the [https://scisoft.fnal.gov/scisoft/bundles/mu2e/ '''mu2e''' manifest], which adds a few products specific to mu2e only, and is maintained by mu2e offline. A complete software download would include both manifests. These manifests are updated as needed, mostly driven by new art versions. They do not include a set of more stable, general purpose products we usually setup from [[Cvmfs|/cvmfs/fermilab.opensciencegrid.org]] or a few products maintained by hand on cvmfs (such as mu2egrid). | ||
manifests are labeled by a version and | manifests are labeled by a version (the art version in our case) and | ||
[https://cdcvs.fnal.gov/redmine/projects/cet-is-public/wiki/AboutQualifiers | [https://cdcvs.fnal.gov/redmine/projects/cet-is-public/wiki/AboutQualifiers flags] for the OS, compiler, art version, and prof/debug. | ||
Here is a list of the [https://cdcvs.fnal.gov/redmine/projects/cet-is/wiki/Supported_platforms supported platforms] from the [https://cdcvs.fnal.gov/redmine/projects/art/wiki art documentation]. | Here is a list of the [https://cdcvs.fnal.gov/redmine/projects/cet-is/wiki/Supported_platforms supported platforms] from the [https://cdcvs.fnal.gov/redmine/projects/art/wiki art documentation]. | ||
To download and unpack a complete set of products on a manifest use the [https://scisoft.fnal.gov/scisoft/bundles/tools/pullProducts pullProducts] script. The available version can be seen from browsing the [https://scisoft.fnal.gov/ manifests] | |||
wget https://scisoft.fnal.gov/scisoft/bundles/tools/pullProducts | |||
chmod a+x pullProducts | |||
pullProducts <options> <product_topdir> <OS> <bundle-spec> <qual_set> <build-spec> | |||
./pullProducts /path/artexternals slf6 mu-v1_17_02 s20-e9 prof | |||
This would typically be run for prof and debug, on each platform. | |||
To download and install a single product use [https://scisoft.fnal.gov/scisoft/bundles/tools/pullPackage pullPackage]. | |||
wget https://scisoft.fnal.gov/scisoft/bundles/tools/pullPackage | |||
chmod a+x pullPackage | |||
pullPackage <options> <product_topdir> <OS> <prod-spec> [<qual_set> [<build-spec>]] | |||
./pullPackage /path/artexternals slf6 ifdhc-v2_1_0 e14-p2713d prof | |||
==Upload== | ==Upload== | ||
Currently (fall 2017) mu2e can upload these: | |||
* mu2e manifests | |||
* BTrk product | |||
* mu2e_artdaq_core product | |||
The current procedure is to take a mu manifest, add the above two products, and create a mu2e manifest. The mu2e manifest adds a version number to the mu manifest in case it needs to be updated while still based on the same parent mu manifest. For example here is a mu2e manifest: | |||
mu2e-2.07.03a.0-Linux64bit+2.6-2.12-s50-e14-prof_MANIFEST.txt | |||
It has version 2.07.03a.0, derived from mu 2.07.03a. | |||
There are [https://cdcvs.fnal.gov/redmine/projects/scisoft/wiki/SciSoft instructions] on how to use the upload script | |||
[https://scisoft.fnal.gov/scisoft/bundles/tools/copyToSciSoft copyToSciSoft]. | |||
12/3/20 - Lynn upgraded the upload portal machine, now we have to do: | |||
12/7/20 - this node seems to be made default | |||
./copyToSciSoft --node scisoftgpvm01.fnal.gov *.bz2 | |||
[[Category:Computing]] | |||
[[Category:Code]] | |||
[[Category:Infrastructure]] |
Latest revision as of 18:38, 7 December 2020
Introduction
scisoft.fnal.gov is a server maintained by the art team (currently mostly Lynn Garren). It contains
- a set of pre-built products in ready-install tarballs, many versions and flavors
- a set of manifests whicah are lists of coordinated sets of products. There are general manifests and experiment-specific manifests.
The products are all stored as relocatable UPS, so they can be simply unpacked into the products directory.
Download
mu2e uses the mu manifest which is maintained by the art team for g-2 and mu2e, and the mu2e manifest, which adds a few products specific to mu2e only, and is maintained by mu2e offline. A complete software download would include both manifests. These manifests are updated as needed, mostly driven by new art versions. They do not include a set of more stable, general purpose products we usually setup from /cvmfs/fermilab.opensciencegrid.org or a few products maintained by hand on cvmfs (such as mu2egrid).
manifests are labeled by a version (the art version in our case) and flags for the OS, compiler, art version, and prof/debug. Here is a list of the supported platforms from the art documentation.
To download and unpack a complete set of products on a manifest use the pullProducts script. The available version can be seen from browsing the manifests
wget https://scisoft.fnal.gov/scisoft/bundles/tools/pullProducts chmod a+x pullProducts pullProducts <options> <product_topdir> <OS> <bundle-spec> <qual_set> <build-spec> ./pullProducts /path/artexternals slf6 mu-v1_17_02 s20-e9 prof
This would typically be run for prof and debug, on each platform.
To download and install a single product use pullPackage.
wget https://scisoft.fnal.gov/scisoft/bundles/tools/pullPackage chmod a+x pullPackage pullPackage <options> <product_topdir> <OS> <prod-spec> [<qual_set> [<build-spec>]] ./pullPackage /path/artexternals slf6 ifdhc-v2_1_0 e14-p2713d prof
Upload
Currently (fall 2017) mu2e can upload these:
- mu2e manifests
- BTrk product
- mu2e_artdaq_core product
The current procedure is to take a mu manifest, add the above two products, and create a mu2e manifest. The mu2e manifest adds a version number to the mu manifest in case it needs to be updated while still based on the same parent mu manifest. For example here is a mu2e manifest:
mu2e-2.07.03a.0-Linux64bit+2.6-2.12-s50-e14-prof_MANIFEST.txt
It has version 2.07.03a.0, derived from mu 2.07.03a.
There are instructions on how to use the upload script copyToSciSoft.
12/3/20 - Lynn upgraded the upload portal machine, now we have to do: 12/7/20 - this node seems to be made default
./copyToSciSoft --node scisoftgpvm01.fnal.gov *.bz2