GeantChecklist
Introduction
So much of the understanding of the performance of mu2e depends on simulation studies that a change in geant version could have far-reaching effects. This page contains a checklist for tests to perform before moving to a new geant. Typically, some tests would be skipped because the expert decides the effort to validate would not be justified for the low level of risk.
Geant4 package switches supported:
- mt - slf7 prof/debug with mt linked, and OGL libraries, OGL not linked by default
- non-mt, with qt, slf7 prof/debug only for viewing geometry
Note on MT build
4/4/18
- The geant MT branch code can be compiled against (and run using) a sequential version of G4 without the '-DG4MULTITHREADED' flag in the SConscript file.
- The MT branch code code can be compiled against an MT version of G4 with the addition of the '-DG4MULTITHREADED' flag in the SConscript file.
- MT version can be run in 3 modes: (1) in sequential mode, (2) in MT mode using a single thread in which there is a master thread that controls a single worker thread, and (3) in MT mode.
v4_10_6_p02a
June 9. 2020. Testing this patch for Krzysztof.
Using this commit:
commit a8757f2b91139
Author: Rob Kutschke <kutschke@fnal.gov>
Date: Wed Jun 3 21:32:08 2020 -0500
Merge pull request #179 from resnegfk/g4106
upgrading geant4 to 10.6.p01
We compare geant4 v4_10_6_p01 to the proposed: v4_10_6_p02a - the only change is the version number in setup.sh.
Results
Memory and file size results from tests, 250K conversion electrons, 25K POT. The CPU result are from repeatedly running old and new version 20min jobs, with various seeds, on the same machine at the same time.
| Sample | g4 CPU (s) | tot CPU (s) | MEM (MB) | file size (MB) |
| old CE | ||||
| new CE | ||||
| old POT | ||||
| new POT |
v4_10_6_p01
issues
- CRV sign-off needed - is neutron physics reasonable?
- file sizes too large - check PS.fcl, the production version of potSim with filtering
- RLC - checked PS.fcl, the file sizes are equivalent in old and new
- memory sizes too large - check if this is neutron cross section tables
- to test: export G4NEUTRONHPDATA=/cvmfs/mu2e.opensciencegrid.org/artexternals/g4neutron/v4_5/NULL/G4NDL4.5
- RLC confirms making the table change brings CE and POT memory back in line with the "old"
- in KLG's small scale test, the electron/gamma/neutron distributions go back to what was seen with v4_10_5_p01a when the above dataset is used, which is consistent with the memory decrease seen/confirmed by RLC
- 4/15/20 - a tentative discussion to accept the memory increase, and deal with it when the next production needs a fix. The solution may b e to split the jobs in parts.
Results
Proposal to evaluate this version 2/19/20. This upgrade is to obtain geant features. It should also fix a geant geometry bug which caused a crash in some particles in the ExtMon.
Comparison done on commit 96105ca37c (head as of 2/20), with the proton generator module name fixed in stoppedMuonsSingleStage.fcl.
Memory and file size results from tests, 250K conversion electrons, 25K POT. The CPU result are from repeatedly running old and new version 20min jobs, with various seeds, on the same machine at the same time. "Old" in comparisons is v4_10_5_p01a.
| Sample | g4 CPU (s) | tot CPU (s) | MEM (MB) | file size (MB) |
| old CE | 0.02527 | 0.2046 | 2331 | 37.4 |
| new CE | 0.02472 (-2%) | 0.2034 (-1%) | 2726 (+17%) | 37.2 (-1%) |
| old POT | 2.664 | 3.190 | 1718 | 2249 |
| new POT | 2.685 (+1%) | 3.289 (+3%) | 2130 (+24%) | 2549 (+13%) |
| Sign off | Responsible | Topic | Notes |
| Krzysztof | release notes presented | talk | |
| Krzysztof | supporting code (new lists, options) | ||
| Ray | high-stats conversion electrons comparison | plots | |
| Ray | high-stats beam stage 1 comparison | plots | |
| Ray/Krzysztof | CPU time and product sizes | ||
| Ray/Andrei | stopped muon rates and shape | rate +1+/-1% plots | |
| Rob | magnetic field transport | ||
| Dave (LBL) | tracker charge deposition/hit resolution | ||
| Dave (LBL) | tracker background hit rates | ||
| Dave (LBL) | tracker integrated charge | ||
| Dave (LBL) | tracker electronics radiation | ||
| Bertrand | calorimeter resolution | ||
| Bertrand | calorimeter electronics radiation | ||
| Ralf | CRV light model | ||
| Yuri | neutron production rates and transport | ||
| Yuri | CRV deadtime | ||
| Dave (LBL) | conversion electron efficiency | ||
| Jim/Yaqian | STM efficiency | ||
| Jim/Yaqian | STM background | ||
| Andrei | ExtMon efficiency | ||
| Andrei | ExtMon background | ||
| Ralf | cosmic background | ||
| Bob | p-bar background | ||
| Andrei | beam electron background | ||
| Dave (Lou) | Pion backgrounds |
v4_10_5_p01a
Moved to this version 01/28/20 PR #124, v08_02_00.
Propose to move to this version 1/25/20. This is the second attempt, after StrawGas was introduced to deal with a higher number of steppoints. Also moved to new stepper and random numbers. This comparison is only on the version number change.
Comparison done on commit 14eeda4, with the proton generator module name fixed in stoppedMuonsSingleStage.fcl.
Memory and file size results from tests, 250K conversion electrons, 25K POT. The CPU result are from repeatedly running old and new version 20min jobs, with various seeds, on the same machine at the same time.
| Sample | CPU (s) | MEM (MB) | file size (MB) |
| old CE | 0.1544 | 2342 | 37.46 |
| new CE | 0.1686 (+9%) | 2349 | 37.51 |
| old POT | 2.585 | 1707 | 1905 |
| new POT | 2.718 (+5%) | 1738 | 2249 (+18%) |
| Sign off | Responsible | Topic | Notes |
| 9/3/19 | Krzysztof | release notes presented | 10.5, 10.5p01, 10.5 TF, |
| 9/3/19 | Krzysztof | selected validation plots | 28336 |
| 9/3/19 | Krzysztof | supporting code (new lists, options) | none needed |
| Ray | high-stats conversion electrons comparison | plots | |
| Ray | high-stats beam stage 1 comparison | plots | |
| Ray/Krzysztof | CPU time and product sizes | ||
| Ray/Andrei | stopped muon rates and shape | increase 7% plots | |
| Rob | magnetic field transport | ||
| 1/27/20 | Dave (LBL) | tracker charge deposition/hit resolution | |
| 1/27/20 | Dave (LBL) | tracker background hit rates | |
| 1/27/20 | Dave (LBL) | tracker integrated charge | |
| 1/27/20 | Dave (LBL) | tracker electronics radiation | |
| Bertrand | calorimeter resolution | ||
| Bertrand | calorimeter electronics radiation | ||
| Ralf | CRV light model | ||
| Yuri | neutron production rates and transport | ||
| Yuri | CRV deadtime | ||
| 1/27/20 | Dave (LBL) | conversion electron efficiency | |
| Jim/Yaqian | STM efficiency | ||
| Jim/Yaqian | STM background | ||
| Andrei | ExtMon efficiency | ||
| Andrei | ExtMon background | ||
| Ralf | cosmic background | ||
| Bob | p-bar background | ||
| Andrei | beam electron background | ||
| Dave (Lou) | Pion backgrounds |
v4_10_5_p01a try 1
Propose to move to this version 8/21/19. The dependencies should be the same as already on the head.
| Sign off | Responsible | Topic | Notes |
| Krzysztof | release notes presented | ||
| 9/3/19 | Krzysztof | selected validation plots | 28336 |
| 9/3/19 | Krzysztof | supporting code (new lists, options) | none needed |
| 8/22/19 | Ray | high-stats conversion electrons comparison | plots |
| 8/22/19 | Ray | high-stats beam stage 1 comparison | plots |
| Ray/Krzysztof | CPU time and product sizes | ||
| 8/22/19 | Ray/Andrei | stopped muon rates and shape | Stops went from 17711 to 18493, a 4 sigma increase. |
| Rob | magnetic field transport | ||
| Dave (LBL) | tracker charge deposition/hit resolution | ||
| Dave (LBL) | tracker background hit rates | ||
| Dave (LBL) | tracker integrated charge | ||
| Dave (LBL) | tracker electronics radiation | ||
| Bertrand | calorimeter resolution | x | |
| Bertrand | calorimeter electronics radiation | ||
| Ralf | CRV light model | ||
| Yuri | neutron production rates and transport | ||
| Yuri | CRV deadtime | ||
| Dave (LBL) | conversion electron efficiency | ||
| Jim/Yaqian | STM efficiency | ||
| Jim/Yaqian | STM background | ||
| Andrei | ExtMon efficiency | ||
| Andrei | ExtMon background | ||
| Ralf | cosmic background | ||
| Bob | p-bar background | ||
| Andrei | beam electron background | ||
| Dave (Lou) | Pion backgrounds |
v4_10_5_p01
5/14/19 built and installed for testing (including comparing Offline and G4Beamline in target studies). SLF6 for qt was not in the build, only SLF7, so that is installed.
v4_10_4_p03
8/21/19 - this version may be not interacting KL in material as much as it should (Yuri Oksuzian, Krzysztof Genser)
Proposal, started 4/16/19, is to move from v4_10_4_p02a to v4_10_4p03.
Fast checks:
v4_10_4_p02
Proposal, started 9/26/18, is to move from v4_10_4 to v4_10_4_p02. Several bugs are fixed, but nothing that is known to be an issue for mu2e.
Closed out this checklist on 11/8/2018. Actually using v4_10_4_p02a ("a" adds e17 compiler), starting in v7_2_0.
To make a release with this change, edit setup.sh and change this line
export MU2E_G4_EXTRA_QUALIFIER=':+cl23'
setup -B geant4 v4_10_4 -q${MU2E_UPS_QUALIFIERS}${MU2E_G4_GRAPHICS_QUALIFIER}${MU2E_G4_MT_QUALIFIER}${MU2E_G4_EXTRA_QUALIFIER}
to
export MU2E_G4_EXTRA_QUALIFIER=
setup -B geant4 v4_10_4_p02 -q${MU2E_UPS_QUALIFIERS}${MU2E_G4_GRAPHICS_QUALIFIER}${MU2E_G4_MT_QUALIFIER}${MU2E_G4_EXTRA_QUALIFIER}
Memory and file size results from tests, 250K conversion electrons, 25K POT. The CPU result are from repeatedly running old and new version 20min jobs, with various seeds, on the same machine at the same time.
| Sample | CPU (s) | MEM (GB) | file size (MB) |
| old CE | 869 | 1.815 | 443 |
| new CE | 865 | 1.822 | 443 |
| old POT | 1238 | 1.735 | 1783 |
| new POT | 1254 | 1.724 | 1780 |
| Accept | Responsible | Topic | Notes |
| 9/26/18 | Krzysztof | release notes presented | 20763 |
| 9/26/18 | Krzysztof | supporting code (new lists, options) | none needed |
| 9/27/18 | Ray | provide high-stats CE comparison | CE validation 21992 |
| 9/27/18 | Ray | provide high-stats POT comparison | POT validation 21992 |
| 9/27/18 | Ray/Krzysztof | provide CPU time and product sizes | see above, 21992 |
| 9/28/18 | Ray/Andrei | provide stopped muon rates and shape | stops 21992 |
| 11/8/18 | Rob | magnetic field transport | no comment, accepted by default |
| 10/31/18 | Dave (LBL) | tracker charge deposition/hit resolution | accept based on notes, generic validation |
| 10/31/18 | Dave (LBL) | tracker background hit rates | accept based on notes, generic validation |
| 10/31/18 | Dave (LBL) | tracker integrated charge | accept based on notes, generic validation |
| 10/31/18 | Dave (LBL) | tracker electronics radiation | accept based on notes, generic validation |
| 11/7/18 | Bertrand | calorimeter resolution | accept based on notes, generic validation |
| 11/7/18 | Bertrand | calorimeter electronics radiation | accept based on notes, generic validation |
| 11/8/18 | Ralf | CRV light model | no comment, accepted by default |
| 11/7/18 | Yuri | neutron production rates and transport | accept based on notes, generic validation |
| 11/7/18 | Yuri | CRV deadtime | accept based on notes, generic validation |
| 10/31/18 | Dave (LBL) | conversion electron efficiency | |
| 11/8/18 | Jim/Yaqian | STM efficiency | no comment, accepted by default |
| 11/8/18 | Jim/Yaqian | STM background | no comment, accepted by default |
| 10/31/18 | Andrei | ExtMon efficiency | accept based on notes, generic validation |
| 10/31/18 | Andrei | ExtMon background | accept based on notes, generic validation |
| 11/8/18 | Ralf | cosmic background | no comment, accepted by default |
| 11/8/18 | Bob | p-bar background | no comment, accepted by default |
| 10/31/18 | Andrei | beam electron background | accept based on notes, generic validation |
| 10/31/18 | Dave (Lou) | Pion backgrounds | accept (by Dave LBL) |
v4_10_4
Started Mar 1, 2018, proposed update to v4_10_4 from v4_10_2_p03e. 3/21/18 - effort paused - now working on v4_10_4 EMZ. On 4/4/18, the decision was taken to restart this effort and delay EMZ, which is more complicated. In the comp/soft meeting on 4/18/18, we decided to make this version the new default.
To make a release with this change, edit setup.sh and change this line
setup -B geant4 v4_10_2_p03e -q${MU2E_UPS_QUALIFIERS}${MU2E_G4_GRAPHICS_QUALIFIER}
to
setup -B geant4 v4_10_4 -qcl23:${MU2E_UPS_QUALIFIERS}${MU2E_G4_GRAPHICS_QUALIFIER}
How to change the stepper to the new one requiring fewer field evaluations. This is off by default and is not part of this checklist.
physics.producers.g4run.physics.stepper : "G4DormandPrince745"
clhep 2.4.0.0 (not used in this checklist evaluation) will enable the switch from HepJamesRandom to MixMax random number engine (art sets the default)
Results from tests, 1M conversion electrons, 300K beam stage1
| Sample | CPU (s) | MEM (GB) | file size (MB) |
| old beam | 9351 | 1.611 | 6.290 |
| new beam | 8483 (-9%) | 1.715 (+6%) | 6.397(+2%) |
| old CE | 6795 | 1.742 | 918.396 |
| new CE | 6180 (-9%) | 1.804 (+4%) | 909.979(-1%) |
beam comparison
C.E. comparison
stopped muons
| Sign off | Responsible | Topic | Notes |
| 3/14/18 | Krzysztof | release notes presented | available |
| 3/14/18 | Krzysztof | supporting code (new lists, options) | changes enabling new features done in February and March |
| 3/14/18 | Ray | high-stats conversion electrons comparison | see links above and talk |
| 3/14/18 | Ray | high-stats beam stage 1 comparison | see links above and talk |
| 3/14/18 | Ray/Krzysztof | CPU time and product sizes | see above and talk |
| 3/14/18 | Ray/Andrei | stopped muon rates and shape | see links above and talk |
| 4/5/18 | Rob | magnetic field transport | the checks done so far are adequate |
| 4/16/18 | Dave (LBL) | tracker charge deposition/hit resolution | roughly 10% more tracker hits than in v4_10_2 talk |
| 4/16/18 | Dave (LBL) | tracker background hit rates | |
| 4/16/18 | Dave (LBL) | tracker integrated charge | |
| 4/16/18 | Dave (LBL) | tracker electronics radiation | A full radiation dose test takes weeks to perform. I validated that the tracker photon interactions are reasonable |
| 4/16/18 | Bertrand | calorimeter resolution | (email) I just looked at the various plots produced for v4_10_4, and they all look fine for the calorimeter. |
| 4/16/18 | Bertrand | calorimeter electronics radiation | |
| Ralf | CRV light model | ||
| Yuri | neutron production rates and transport | ||
| 4/19/18 | Yuri | CRV deadtime | I don’t think new version will make any impact on the CRV rates. |
| 4/16/18 | Dave (LBL) | conversion electron efficiency | Efficiency and quality are slightly degraded |
| 4/14/18 | Jim/Yaqian | STM efficiency | lead flourescence and brem bkg checked |
| 4/14/18 | Jim/Yaqian | STM background | lead flourescence and brem bkg checked |
| 4/10/18 | Andrei | ExtMon efficiency | Skip this time (the baseline has not been checked in years). This is feasible to do in the future. |
| 4/10/18 | Andrei | ExtMon background | Skip. A complete study would require very large amount of CPU and effort. Unlikely to be feasible for GEANT upgrade checks. A simpler proxy like e.g. neutron flux in the ExtMon room might be feasible. |
| Ralf | cosmic background | ||
| Bob | p-bar background | ||
| 4/10/18 | Andrei | beam electron background | Skip. May be feasible in the future. |
| 4/19/18 | Dave (Lou) | Pion backgrounds | It would require a dedicated run to do a "real" test. However, if I look at muon stops as a proxy for pion stops, then I see nothing here that makes me panic. |
v4_10_4 EMZ
Started Mar 21, 2018, proposed update to v4_10_4 EMZ from v4_10_2_p03e. At the code/soft meeting on 4/4/18, we decided to delay moving to EMZ since the very different file sizes caused complications with no obvious benefit. This will be considered at a later date, along with regional minRangeCut's.
How to trigger Mu2e specific electromagnetic option4 (_EMZ) with modified msc model transition energy (committed 3/16/18)
physics.producers.g4run.debug.diagLevel : 1 # the above is to see the ranges printed physics.producers.g4run.physics.modifyEMOption4 : true physics.producers.g4run.physics.physicsListName : "ShieldingM_EMZ"
How to change the stepper to the new one requiring fewer field evaluations (off, except where noted):
physics.producers.g4run.physics.stepper : "G4DormandPrince745"
clhep 2.4.0.0 (not used yet) will enable the switch from HepJamesRandom to MixMax random number engine (art sets the default)
Results from tests, 1M conversion electrons, 300K beam stage1
| Sample | CPU (s) | MEM (GB) | file size (MB) |
| old beam | 7325 | 1.76 | 630 |
| new beam | 7896(+8%) | 1.79(+2%) | 718(+13%) |
| old CE | 6844 | 1.75 | 92088 |
| new CE | 7099(+4%) | 1.84(+5%) | 187456(+100%) |
beam stage 1 comparison
CE comparison
stops
Why are conversion electron files so much larger:
arProductsizes:
old:
Size Size/Entry Fraction Data Product Name
505449709 50545.0 0.548 mu2e::SimParticlemv_g4run__ceSimReco.
240938209 24093.8 0.261 mu2e::StepPointMCs_g4run_calorimeter_ceSimReco.
30774582 3077.5 0.033 mu2e::CaloShowerSteps_CaloShowerStepFromStepPt_calorimeter_ceSimReco.
28011825 2801.2 0.030 mu2e::StepPointMCs_g4run_tracker_ceSimReco.
new:
Size Size/Entry Fraction Data Product Name
514208625 51420.9 0.277 mu2e::SimParticlemv_g4run__ceSimReco.
745787106 74578.7 0.402 mu2e::StepPointMCs_g4run_calorimeter_ceSimReco.
236050707 23605.1 0.127 mu2e::CaloShowerSteps_CaloShowerStepFromStepPt_calorimeter_ceSimReco.
181788056 18178.8 0.098 mu2e::StepPointMCs_g4run_tracker_ceSimReco.
| Sign off | Responsible | Topic | Notes |
| 3/21/18 | Krzysztof | release notes presented | talk |
| 3/14/18 | Krzysztof | supporting code (new lists, options) | changes enabling new features done in February and March |
| Ray | high-stats conversion electrons comparison | see links above | |
| Ray | high-stats beam stage 1 comparison | see links above | |
| Ray/Krzysztof | CPU time and product sizes | see above | |
| Ray/Andrei | stopped muon rates and shape | see link above | |
| Rob | magnetic field transport | ||
| Dave (LBL) | tracker charge deposition/hit resolution | talk | |
| Dave (LBL) | tracker background hit rates | ||
| Dave (LBL) | tracker integrated charge | ||
| Dave (LBL) | tracker electronics radiation | ||
| Bertrand | calorimeter resolution | 4/16/18 probably OK | |
| Bertrand | calorimeter electronics radiation | 4/16/18 this needs more study | |
| Ralf | CRV light model | ||
| Yuri | neutron production rates and transport | ||
| Yuri | CRV deadtime | ||
| Dave (LBL) | conversion electron efficiency | ||
| Jim | STM efficiency | ||
| Jim | STM background | ||
| Andrei | ExtMon efficiency | ||
| Andrei | ExtMon background | ||
| Ralf | cosmic background | ||
| Bob | p-bar background | ||
| Andrei | beam electron background | ||
| Dave (Lou) | Pion backgrounds |
empty example table
| Sign off | Responsible | Topic | Notes |
| Krzysztof | release notes presented | ||
| Krzysztof | supporting code (new lists, options) | ||
| Ray | high-stats conversion electrons comparison | ||
| Ray | high-stats beam stage 1 comparison | ||
| Ray/Krzysztof | CPU time and product sizes | ||
| Ray/Andrei | stopped muon rates and shape | ||
| Rob | magnetic field transport | ||
| Dave (LBL) | tracker charge deposition/hit resolution | ||
| Dave (LBL) | tracker background hit rates | ||
| Dave (LBL) | tracker integrated charge | ||
| Dave (LBL) | tracker electronics radiation | ||
| Bertrand | calorimeter resolution | x | |
| Bertrand | calorimeter electronics radiation | ||
| Ralf | CRV light model | ||
| Yuri | neutron production rates and transport | ||
| Yuri | CRV deadtime | ||
| Dave (LBL) | conversion electron efficiency | ||
| Jim/Yaqian | STM efficiency | ||
| Jim/Yaqian | STM background | ||
| Andrei | ExtMon efficiency | ||
| Andrei | ExtMon background | ||
| Ralf | cosmic background | ||
| Bob | p-bar background | ||
| Andrei | beam electron background | ||
| Dave (Lou) | Pion backgrounds |