JustInTimeFcl

Examples

Below is an example of using JustInTimeFcl framework for the cosmic resampling stage.

Setting Up Environment: SimJob, mu2etools and mu2egrid

source /cvmfs/mu2e.opensciencegrid.org/setupmu2e-art.sh
muse setup SimJob MDC2020aa
setup sam_web_client
setup mu2egrid

Append the content below to the file called "template.fcl"

#include "Production/JobConfig/cosmic/S2Resampler.fcl"
#include "Production/JobConfig/cosmic/S2ResamplerLow.fcl"
outputs.PrimaryOutput.fileName: "dts.owner.CosmicCORSIKALow.version.sequencer.art"

Prepare input file list - retrieve a list of files related to a given definition and append the first 10 filenames into CosmicDSStop.txt:

samweb list-definition-files sim.mu2e.CosmicDSStopsCORSIKALow.MDC2020ab.art | head -n 10 > CosmicDSStop.txt

Prepare the job definition file (tarball) for resampling.

mu2ejobdef --dsconf MDC2020ab --dsowner mu2e --run-number 1203 --desc CosmicCORSIKALow --embed template.fcl --events-per-job 500000 --auxinput=1:physics.filters.CosmicResampler.fileNames:CosmicDSStop.txt --setup /cvmfs/mu2e.opensciencegrid.org/Musings/SimJob/MDC2020aa/setup.sh

Another example for generation job without any input:

echo '#include "Offline/CRVResponse/test/wideband/wideband4modules.fcl"' > template.fcl
echo 'source.module_type: EmptyEvent' >> template.fcl
mu2ejobdef --jobdesc=CosmicWidebandCRY --dsconf=MDC2020ae_best_v1_3 --dsowner=mu2e --run-number=103001 --events-per-job=10000 --embed template.fcl --setup /cvmfs/mu2e.opensciencegrid.org/Musings/SimJob/MDC2020ae/setup.sh --jobdesc=CosmicWidebandCRY --dsconf=MDC2020ae_best_v1_3 --dsowner=mu2e

Inspect fcl file content based on index

mu2ejobfcl --jobdef cnf.mu2e.CosmicCORSIKALow.MDC2020ab.0.tar --index 1 --default-proto root --default-loc tape

Based on output filename:

mu2ejobfcl --jobdef /pnfs/mu2e/persistent/datasets/phy-etc/cnf/mu2e/CeMLeadingLogMix1BBTriggered/MDC2020am_best_v1_3/tar/ed/d3/cnf.mu2e.CeMLeadingLogMix1BBTriggered.MDC2020am_best_v1_3.0.tar --target mcs.mu2e.CeMLeadingLogMix1BBTriggered.MDC2020am_best_v1_3.001210_00000570.art --default-proto root --default-loc tape

Submit your job - mu2ejobsub submits the job. It specifies the job definition file (--jobdef), the range of job indices to submit (--firstjob, --njobs), and default protocol to access input files and output location location

mu2ejobsub --jobdef cnf.mu2e.CosmicCORSIKALow.MDC2020ab.0.tar --firstjob=0 --njobs=10 --default-protocol root --default-location tape

Extract input files:

mu2ejobiodetail --jobdef cnf.mu2e.CosmicCORSIKALow.MDC2020ab.0.tar --index 1 --inputs

Using a local undeclared input

As an example, generate par file using gen_Reco.sh

gen_Reco.sh --primary CosmicCRYAll --campaign MDC2020 --dver ai --rver ai  --dbpurpose perfect --dbversion v1_3 --merge 10 --digitype OffSpill --stream Triggered --digis /exp/mu2e/app/users/oksuzian/muse_080224/Production/DigisIn2.txt

The file list is required to have basenames only

cat /exp/mu2e/app/users/oksuzian/muse_080224/Production/DigisIn2.txt
dig.oksuzian.CosmicCRYAllOffSpillTriggered.MDC2020ai_perfect_v1_3.001202_00000054.art

When creating fcl you need to specify the directory for the input files:

mu2ejobfcl --jobdef cnf.oksuzian.CosmicCRYAllOffSpillTriggered.MDC2020ai_perfect_v1_3.0.tar --index 0 --default-proto file --default-loc dir:/exp/mu2e/data/users/oksuzian/test3
#include "Production/JobConfig/reco/Reco.fcl"
services.DbService.purpose: MDC2020_perfect
services.DbService.version: v1_3
services.DbService.verbose : 2
#include "Production/JobConfig/reco/MakeSurfaceSteps.fcl"
#----------------------------------------------------------------
# Code added by mu2ejobfcl:
source.fileNames: [
   "/exp/mu2e/data/users/oksuzian/test3/dig.oksuzian.CosmicCRYAllOffSpillTriggered.MDC2020ai_perfect_v1_3.001202_00000054.art"
]
outputs.Output.fileName: "mcs.oksuzian.CosmicCRYAllOffSpillTriggered.MDC2020ai_perfect_v1_3.001202_00000054.art"
# End code added by mu2ejobfcl:
#---------------------------------------------------------------