JustInTimeFcl: Difference between revisions
Jump to navigation
Jump to search
| Line 3: | Line 3: | ||
==Example== | ==Example== | ||
Below is an example of using JustInTimeFcl framework for the cosmic resampling stage. | |||
Setting Up Environment: SimJob, mu2etools and mu2egrid | |||
<pre> | <pre> | ||
source /cvmfs/mu2e.opensciencegrid.org/setupmu2e-art.sh | source /cvmfs/mu2e.opensciencegrid.org/setupmu2e-art.sh | ||
| Line 19: | Line 20: | ||
</pre> | </pre> | ||
Prepare input file list: | Prepare input file list - retrieve a list of files related to the given definition sim.mu2e.CosmicDSStopsCORSIKALow.MDC2020ab.art and append the first 10 filenames into CosmicDSStop.txt: | ||
<pre> | <pre> | ||
samweb list-definition-files sim.mu2e.CosmicDSStopsCORSIKALow.MDC2020ab.art | head -n 10 > CosmicDSStop.txt | samweb list-definition-files sim.mu2e.CosmicDSStopsCORSIKALow.MDC2020ab.art | head -n 10 > CosmicDSStop.txt | ||
</pre> | </pre> | ||
Prepare the tarball | Prepare the job definition file (tarball) | ||
<pre> | <pre> | ||
mu2ejobdef --dsconf MDC2020ab --dsowner mu2e --run-number 1203 --jobdesc CosmicCORSIKALow --embed template.fcl --events-per-job 500000 --auxinput=1:physics.filters.CosmicResampler.fileNames:CosmicDSStop.txt --setup /cvmfs/mu2e.opensciencegrid.org/Musings/SimJob/MDC2020aa/setup.sh | mu2ejobdef --dsconf MDC2020ab --dsowner mu2e --run-number 1203 --jobdesc CosmicCORSIKALow --embed template.fcl --events-per-job 500000 --auxinput=1:physics.filters.CosmicResampler.fileNames:CosmicDSStop.txt --setup /cvmfs/mu2e.opensciencegrid.org/Musings/SimJob/MDC2020aa/setup.sh | ||
</pre> | </pre> | ||
Submit your job | Submit your job - mu2ejobsub submits the job. It specifies the job definition file (--jobdef), the range of job indices to submit (--firstjob, --njobs), and default protocol to access input files and output location location | ||
<pre> | <pre> | ||
mu2ejobsub --jobdef cnf.mu2e.CosmicCORSIKALow.MDC2020ab.0.tar --firstjob=1 --njobs=10 --default-protocol root --default-location | mu2ejobsub --jobdef cnf.mu2e.CosmicCORSIKALow.MDC2020ab.0.tar --firstjob=1 --njobs=10 --default-protocol root --default-location tape --firstjob=0 --njobs=10 | ||
</pre> | </pre> | ||
Inspect fcl file content | |||
<pre> | <pre> | ||
mu2ejobfcl --jobdef cnf.mu2e.CosmicCORSIKALow.MDC2020ab.0.tar --index 1 --default-proto root --default-loc tape | mu2ejobfcl --jobdef cnf.mu2e.CosmicCORSIKALow.MDC2020ab.0.tar --index 1 --default-proto root --default-loc tape | ||
</pre> | </pre> | ||
Inspect input files: | |||
<pre> | <pre> | ||
mu2ejobiodetail --jobdef cnf.mu2e.CosmicCORSIKALow.MDC2020ab.0.tar --index 1 --inputs | mu2ejobiodetail --jobdef cnf.mu2e.CosmicCORSIKALow.MDC2020ab.0.tar --index 1 --inputs | ||
Revision as of 17:22, 18 April 2024
Introduction
Example
Below is an example of using JustInTimeFcl framework for the cosmic resampling stage.
Setting Up Environment: SimJob, mu2etools and mu2egrid
source /cvmfs/mu2e.opensciencegrid.org/setupmu2e-art.sh muse setup SimJob MDC2020aa setup mu2etools setup mu2egrid
Append the content below to the file called "template.fcl"
#include "Production/JobConfig/cosmic/S2Resampler.fcl" #include "Production/JobConfig/cosmic/S2ResamplerLow.fcl" outputs.PrimaryOutput.fileName: "dts.owner.CosmicCORSIKALow.version.sequencer.art"
Prepare input file list - retrieve a list of files related to the given definition sim.mu2e.CosmicDSStopsCORSIKALow.MDC2020ab.art and append the first 10 filenames into CosmicDSStop.txt:
samweb list-definition-files sim.mu2e.CosmicDSStopsCORSIKALow.MDC2020ab.art | head -n 10 > CosmicDSStop.txt
Prepare the job definition file (tarball)
mu2ejobdef --dsconf MDC2020ab --dsowner mu2e --run-number 1203 --jobdesc CosmicCORSIKALow --embed template.fcl --events-per-job 500000 --auxinput=1:physics.filters.CosmicResampler.fileNames:CosmicDSStop.txt --setup /cvmfs/mu2e.opensciencegrid.org/Musings/SimJob/MDC2020aa/setup.sh
Submit your job - mu2ejobsub submits the job. It specifies the job definition file (--jobdef), the range of job indices to submit (--firstjob, --njobs), and default protocol to access input files and output location location
mu2ejobsub --jobdef cnf.mu2e.CosmicCORSIKALow.MDC2020ab.0.tar --firstjob=1 --njobs=10 --default-protocol root --default-location tape --firstjob=0 --njobs=10
Inspect fcl file content
mu2ejobfcl --jobdef cnf.mu2e.CosmicCORSIKALow.MDC2020ab.0.tar --index 1 --default-proto root --default-loc tape
Inspect input files:
mu2ejobiodetail --jobdef cnf.mu2e.CosmicCORSIKALow.MDC2020ab.0.tar --index 1 --inputs