RunNumbers: Difference between revisions

From Mu2eWiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(28 intermediate revisions by 3 users not shown)
Line 2: Line 2:


==Introduction==
==Introduction==
On this page we reserve run number ranges to attempt to prevent multiple uses for a single run number.  This is becoming increasingly important to not generate conflicting file names or conditions database entries, and to improve communication.
On this page we reserve run number ranges to attempt to prevent multiple uses for a single run number.  This is important to not generate conflicting file names or conditions database entries, and to improve communication and documentation.


To reserve a range,  
The plan to assign run number ranges, in this period before beam data, was discussed and agreed in the Trigger Group meeting. Any group requesting a run number range must stay within that range until they request another block of run numbers.  An OTS TDAQ instance can have its initial run number set to the beginning of the range, then it can increment the run number automatically with each run start.  The Trigger Group will continue to discuss and develop a plan to coordinate run numbers in a central, database-backed method.
* use the next available numbers
 
Only the Calibration Coordinator or Offline Co-leaders are allowed to reserve run ranges.
* use the next available numbers above the reserved
* be modest in the number reserved (you can always come back for more)
* be modest in the number reserved (you can always come back for more)
* post a note of your reservation to the code/sim group hypernews ''mu2e-hn-Sim@listserv.fnal.gov''
* tests and simulation will have run number below 100,000
* data, including test beam, test stands and cosmics, will have run numbers above 100,000


== Reserved run numbers ==
== Reserved run numbers ==
The "Organizer" of a run number range has the primary responsibility to assure the group using the run numbers stays within the run range, or arrange for another block of numbers.


{|style="width: 90%"
{|style="width: 90%"
|-
|-
|style="width:10%"|'''Start'''
|style="width:5%;text-align:right;"|'''Start'''
|style="width:10%"|'''End'''
|style="width:5%;text-align:right;"|'''End'''
|style="width:15%"|'''Name'''
|style="width:1%"|
|style="width:15%"|'''Organizer'''
|style="width:10%"|'''Name'''
|style="width:10%"|'''Organizer'''
|style="width:50%"|'''Notes'''
|style="width:50%"|'''Notes'''
|-
|-
0 || 999 ||  offline || Rob Kutschke || scratch or temporary experiments
style="text-align:right;" | 1 || style="text-align:right;" | 99 || ||  userSpace || Rob Kutschke || arbitrary user needs (not production)
|-
|  style="text-align:right;" | 100  || style="text-align:right;" | 999 || ||  offline || Rob Kutschke || scratch or temporary experiments
|-
|-
|  1000  || 1099 ||  legacy || Rob Kutschke || numbers commonly used in simulation before this system
style="text-align:right;" | 1000  || style="text-align:right;" | 1099 ||  || legacy || Rob Kutschke || numbers commonly used in simulation before this system
|-
|-
|  1100  || 1199 ||  database tests || Ray Culbertson || run dependence tests
style="text-align:right;" | 1100  || style="text-align:right;" | 1199 || ||  database tests || Ray Culbertson || run dependence tests
|-
|-
|  1200  || 1299 || MDC2020 || Dave Brown (LBL) || production simulation with run dependence
style="text-align:right;" | 1200  || style="text-align:right;" | 1299 || || MDC2020 || Dave Brown (LBL) || MDC2020 production simulation with run dependence
|-
|-
1300 || 4999 || tracker VST || Richie Bonventre || test data from tracker VST 2020-2021
style="text-align:right;" | 1205 || style="text-align:right;" | 1205 || || MDC2020 extracted || Dave Brown (LBL) || MDC2020 production simulation of extracted position
|-
|-
5000  || 7999 || tracker VST || Richie Bonventre || cosmic ray data from tracker VST 2020-2021
style="text-align:right;" | 1300 || style="text-align:right;" | 1399 || || RDM || Ray Culbertson || test data for file movement and processing
|-
|-
1000000 || 1999999 || DAQ || Greg Rakness || data produced at the Mu2e hall using the central TDAQ
style="text-align:right;" | 1400 || style="text-align:right;" | ... || || ... || .... || next non-data reservations
|-
|  style="text-align:right;" | 100000  || style="text-align:right;" | 100999 || || tracker VST || Richie Bonventre || tracker VST 2020-2021
|-
|  style="text-align:right;" | 101000  || style="text-align:right;" | 101999 || || STM test beam || Andy Edmonds || Dresden test beam 4/2022
|-
|  style="text-align:right;" | 102000 || style="text-align:right;" | 102999 || || module-0 || Eleonora Diociaiuti || Calorimeter module-0
|-
| style="text-align:right;" | 103000 || style="text-align:right;" | 103999 || || CRV || Ralf Ehrlich /<br> Simon Corrodi || CRV data from the Wideband teststand
|-
|  style="text-align:right;" | 104000 || style="text-align:right;" | 104999 || || ... || .... || next data reservations
|-
|  style="text-align:right;" | 105000 || style="text-align:right;" | 999999 || || automatic || Antonio Gioiosa || automatic online database
|}
|}


== Simulation run numbers ==
Simulation will eventually need run dependence.  A good example would be if a tracker plane is turned off or recovered during the run, then we will want to run a job which simulated the difference tracker configuration in the proper proportions.  The plan is that when a simulation campaign starts, there will be the following steps
# the collaboration (Offline, Operations, Physics groups) will define periods of stable running called "simulation periods", recorded as run number ranges.  We expect this will be on the order of 10's of periods per year.  (We also expect "run periods" defined by production and analysis operational needs, and may or may not be similar to the simulation periods).
# for each simulation period, a representative set of conditions (typically database calibration tables) will be selected from, or derived from, the calibration data of the data runs in the period.  More conditions data, appropriate only for simulation can be added.
# a sequential set of runs, from the non-data run ranges, will be reserved for this simulation campaign.  The size of the set will essentially be equal to the number of simulation periods.
# The calibration manager will create a new purpose and version in the conditions database for this simulation campaign
# The conditions data prepared for each simulation period will be associated to a run in the reserved run range though the standard conditions database techniques.  The data in these runs will be collected under the new purpose/version.
# simulation jobs will produce data in the reserved run ranges, with each run produced in proportion to the data taken during the corresponding simulation period, and conditions data taken from the new purpose/version.
# reconstruction jobs running on this simulation should be able to use the described conditions data.
This section is an outline.  Additional detailed techniques will need to be defined for staging, mix-in options, and systematic variations of conditions.




==Some History==
==Some History==


[https://mu2e-docdb.fnal.gov/cgi-bin/sso/ShowDocument?docid=33496 talk]<br>
[https://mu2e-docdb.fnal.gov/cgi-bin/sso/ShowDocument?docid=36261 talk]<br>
[https://mu2e-docdb.fnal.gov/cgi-bin/sso/ShowDocument?docid=35960 talk]<br>
[https://mu2e-docdb.fnal.gov/cgi-bin/sso/ShowDocument?docid=17240 Ray talk] page 3 <br>
[https://mu2e-docdb.fnal.gov/cgi-bin/sso/ShowDocument?docid=17240 Ray talk] page 3 <br>
[https://mu2e-docdb.fnal.gov/cgi-bin/sso/ShowDocument?docid=34638 Richie talk] page 7
[https://mu2e-docdb.fnal.gov/cgi-bin/sso/ShowDocument?docid=34638 Richie talk] page 7
Line 76: Line 111:
     115 11911024-19236224
     115 11911024-19236224
</pre>
</pre>
[[Category:Computing]]
[[Category:Workflows]]

Latest revision as of 20:12, 19 April 2024


Introduction

On this page we reserve run number ranges to attempt to prevent multiple uses for a single run number. This is important to not generate conflicting file names or conditions database entries, and to improve communication and documentation.

The plan to assign run number ranges, in this period before beam data, was discussed and agreed in the Trigger Group meeting. Any group requesting a run number range must stay within that range until they request another block of run numbers. An OTS TDAQ instance can have its initial run number set to the beginning of the range, then it can increment the run number automatically with each run start. The Trigger Group will continue to discuss and develop a plan to coordinate run numbers in a central, database-backed method.

Only the Calibration Coordinator or Offline Co-leaders are allowed to reserve run ranges.

  • use the next available numbers above the reserved
  • be modest in the number reserved (you can always come back for more)
  • tests and simulation will have run number below 100,000
  • data, including test beam, test stands and cosmics, will have run numbers above 100,000

Reserved run numbers

The "Organizer" of a run number range has the primary responsibility to assure the group using the run numbers stays within the run range, or arrange for another block of numbers.

Start End Name Organizer Notes
1 99 userSpace Rob Kutschke arbitrary user needs (not production)
100 999 offline Rob Kutschke scratch or temporary experiments
1000 1099 legacy Rob Kutschke numbers commonly used in simulation before this system
1100 1199 database tests Ray Culbertson run dependence tests
1200 1299 MDC2020 Dave Brown (LBL) MDC2020 production simulation with run dependence
1205 1205 MDC2020 extracted Dave Brown (LBL) MDC2020 production simulation of extracted position
1300 1399 RDM Ray Culbertson test data for file movement and processing
1400 ... ... .... next non-data reservations
100000 100999 tracker VST Richie Bonventre tracker VST 2020-2021
101000 101999 STM test beam Andy Edmonds Dresden test beam 4/2022
102000 102999 module-0 Eleonora Diociaiuti Calorimeter module-0
103000 103999 CRV Ralf Ehrlich /
Simon Corrodi
CRV data from the Wideband teststand
104000 104999 ... .... next data reservations
105000 999999 automatic Antonio Gioiosa automatic online database

Simulation run numbers

Simulation will eventually need run dependence. A good example would be if a tracker plane is turned off or recovered during the run, then we will want to run a job which simulated the difference tracker configuration in the proper proportions. The plan is that when a simulation campaign starts, there will be the following steps

  1. the collaboration (Offline, Operations, Physics groups) will define periods of stable running called "simulation periods", recorded as run number ranges. We expect this will be on the order of 10's of periods per year. (We also expect "run periods" defined by production and analysis operational needs, and may or may not be similar to the simulation periods).
  2. for each simulation period, a representative set of conditions (typically database calibration tables) will be selected from, or derived from, the calibration data of the data runs in the period. More conditions data, appropriate only for simulation can be added.
  3. a sequential set of runs, from the non-data run ranges, will be reserved for this simulation campaign. The size of the set will essentially be equal to the number of simulation periods.
  4. The calibration manager will create a new purpose and version in the conditions database for this simulation campaign
  5. The conditions data prepared for each simulation period will be associated to a run in the reserved run range though the standard conditions database techniques. The data in these runs will be collected under the new purpose/version.
  6. simulation jobs will produce data in the reserved run ranges, with each run produced in proportion to the data taken during the corresponding simulation period, and conditions data taken from the new purpose/version.
  7. reconstruction jobs running on this simulation should be able to use the described conditions data.

This section is an outline. Additional detailed techniques will need to be defined for staging, mix-in options, and systematic variations of conditions.


Some History

talk
talk
talk
Ray talk page 3
Richie talk page 7

Approximate survey of existing run numbers. 1000 is current example for sim.

    count runnumber
    19 1
      1 2
      1 3
      5 4
      1 5
      1 7
     23 1000
     10 1001
    234 1002
      6 1052
      3 1053
      1 1357
      6 1500
     11 1600
      5 1700
     10 1800
      1 2000
      1 2001
      9 2700
     20 2701
      2 2705
     57 4001
      1 4002
      3 5001
      1 9002
      1 645440
      1 1792836
      2 2102784
      1 3344213
    115 11911024-19236224