RunNumbers: Difference between revisions
No edit summary |
|||
(22 intermediate revisions by 3 users not shown) | |||
Line 2: | Line 2: | ||
==Introduction== | ==Introduction== | ||
On this page we reserve run number ranges to attempt to prevent multiple uses for a single run number. This is | On this page we reserve run number ranges to attempt to prevent multiple uses for a single run number. This is important to not generate conflicting file names or conditions database entries, and to improve communication and documentation. | ||
The plan to assign run number ranges, in this period before beam data, was discussed and agreed in the Trigger Group meeting. Any group requesting a run number range must stay within that range until they request another block of run numbers. An OTS TDAQ instance can have its initial run number set to the beginning of the range, then it can increment the run number automatically with each run start. The Trigger Group will continue to discuss and develop a plan to coordinate run numbers in a central, database-backed method. | |||
* use the next available numbers above the reserved | |||
Only the Calibration Coordinator or Offline Co-leaders are allowed to reserve run ranges. | |||
* use the next available numbers above the reserved | |||
* be modest in the number reserved (you can always come back for more) | * be modest in the number reserved (you can always come back for more) | ||
* | * tests and simulation will have run number below 100,000 | ||
* data, including test beam, test stands and cosmics, will have run numbers above 100,000 | |||
== Reserved run numbers == | == Reserved run numbers == | ||
The "Organizer" of a run number range has the primary responsibility to assure the group using the run numbers stays within the run range, or arrange for another block of numbers. | |||
{|style="width: 90%" | {|style="width: 90%" | ||
|- | |- | ||
|style="width: | |style="width:5%;text-align:right;"|'''Start''' | ||
|style="width: | |style="width:5%;text-align:right;"|'''End''' | ||
|style="width: | |style="width:1%"| | ||
|style="width: | |style="width:10%"|'''Name''' | ||
|style="width:10%"|'''Organizer''' | |||
|style="width:50%"|'''Notes''' | |style="width:50%"|'''Notes''' | ||
|- | |- | ||
| 1 || 99 || userSpace || Rob Kutschke || arbitrary user needs (not production) | | style="text-align:right;" | 1 || style="text-align:right;" | 99 || || userSpace || Rob Kutschke || arbitrary user needs (not production) | ||
|- | |||
| style="text-align:right;" | 100 || style="text-align:right;" | 999 || || offline || Rob Kutschke || scratch or temporary experiments | |||
|- | |||
| style="text-align:right;" | 1000 || style="text-align:right;" | 1099 || || legacy || Rob Kutschke || numbers commonly used in simulation before this system | |||
|- | |||
| style="text-align:right;" | 1100 || style="text-align:right;" | 1199 || || database tests || Ray Culbertson || run dependence tests | |||
|- | |||
| style="text-align:right;" | 1200 || style="text-align:right;" | 1299 || || MDC2020 || Dave Brown (LBL) || MDC2020 production simulation with run dependence | |||
|- | |- | ||
| | | style="text-align:right;" | 1205 || style="text-align:right;" | 1205 || || MDC2020 extracted || Dave Brown (LBL) || MDC2020 production simulation of extracted position | ||
|- | |- | ||
| | | style="text-align:right;" | 1300 || style="text-align:right;" | 1399 || || RDM || Ray Culbertson || test data for file movement and processing | ||
|- | |- | ||
| | | style="text-align:right;" | 1400 || style="text-align:right;" | ... || || ... || .... || next non-data reservations | ||
|- | |- | ||
| | | style="text-align:right;" | 100000 || style="text-align:right;" | 100999 || || tracker VST || Richie Bonventre || tracker VST 2020-2021 | ||
|- | |- | ||
| | | style="text-align:right;" | 101000 || style="text-align:right;" | 101999 || || STM test beam || Andy Edmonds || Dresden test beam 4/2022 | ||
|- | |- | ||
| | | style="text-align:right;" | 102000 || style="text-align:right;" | 102999 || || module-0 || Eleonora Diociaiuti || Calorimeter module-0 | ||
|- | |- | ||
| | | style="text-align:right;" | 103000 || style="text-align:right;" | 103999 || || CRV || Ralf Ehrlich /<br> Simon Corrodi || CRV data from the Wideband teststand | ||
|- | |- | ||
| | | style="text-align:right;" | 104000 || style="text-align:right;" | 104999 || || ... || .... || next data reservations | ||
|- | |- | ||
| | | style="text-align:right;" | 105000 || style="text-align:right;" | 999999 || || automatic || Antonio Gioiosa || automatic online database | ||
|} | |} | ||
== Simulation run numbers == | |||
Simulation will eventually need run dependence. A good example would be if a tracker plane is turned off or recovered during the run, then we will want to run a job which simulated the difference tracker configuration in the proper proportions. The plan is that when a simulation campaign starts, there will be the following steps | |||
# the collaboration (Offline, Operations, Physics groups) will define periods of stable running called "simulation periods", recorded as run number ranges. We expect this will be on the order of 10's of periods per year. (We also expect "run periods" defined by production and analysis operational needs, and may or may not be similar to the simulation periods). | |||
# for each simulation period, a representative set of conditions (typically database calibration tables) will be selected from, or derived from, the calibration data of the data runs in the period. More conditions data, appropriate only for simulation can be added. | |||
# a sequential set of runs, from the non-data run ranges, will be reserved for this simulation campaign. The size of the set will essentially be equal to the number of simulation periods. | |||
# The calibration manager will create a new purpose and version in the conditions database for this simulation campaign | |||
# The conditions data prepared for each simulation period will be associated to a run in the reserved run range though the standard conditions database techniques. The data in these runs will be collected under the new purpose/version. | |||
# simulation jobs will produce data in the reserved run ranges, with each run produced in proportion to the data taken during the corresponding simulation period, and conditions data taken from the new purpose/version. | |||
# reconstruction jobs running on this simulation should be able to use the described conditions data. | |||
This section is an outline. Additional detailed techniques will need to be defined for staging, mix-in options, and systematic variations of conditions. | |||
==Some History== | ==Some History== | ||
[https://mu2e-docdb.fnal.gov/cgi-bin/sso/ShowDocument?docid=33496 talk]<br> | |||
[https://mu2e-docdb.fnal.gov/cgi-bin/sso/ShowDocument?docid=36261 talk]<br> | |||
[https://mu2e-docdb.fnal.gov/cgi-bin/sso/ShowDocument?docid=35960 talk]<br> | |||
[https://mu2e-docdb.fnal.gov/cgi-bin/sso/ShowDocument?docid=17240 Ray talk] page 3 <br> | [https://mu2e-docdb.fnal.gov/cgi-bin/sso/ShowDocument?docid=17240 Ray talk] page 3 <br> | ||
[https://mu2e-docdb.fnal.gov/cgi-bin/sso/ShowDocument?docid=34638 Richie talk] page 7 | [https://mu2e-docdb.fnal.gov/cgi-bin/sso/ShowDocument?docid=34638 Richie talk] page 7 |
Latest revision as of 20:12, 19 April 2024
Introduction
On this page we reserve run number ranges to attempt to prevent multiple uses for a single run number. This is important to not generate conflicting file names or conditions database entries, and to improve communication and documentation.
The plan to assign run number ranges, in this period before beam data, was discussed and agreed in the Trigger Group meeting. Any group requesting a run number range must stay within that range until they request another block of run numbers. An OTS TDAQ instance can have its initial run number set to the beginning of the range, then it can increment the run number automatically with each run start. The Trigger Group will continue to discuss and develop a plan to coordinate run numbers in a central, database-backed method.
Only the Calibration Coordinator or Offline Co-leaders are allowed to reserve run ranges.
- use the next available numbers above the reserved
- be modest in the number reserved (you can always come back for more)
- tests and simulation will have run number below 100,000
- data, including test beam, test stands and cosmics, will have run numbers above 100,000
Reserved run numbers
The "Organizer" of a run number range has the primary responsibility to assure the group using the run numbers stays within the run range, or arrange for another block of numbers.
Start | End | Name | Organizer | Notes | |
1 | 99 | userSpace | Rob Kutschke | arbitrary user needs (not production) | |
100 | 999 | offline | Rob Kutschke | scratch or temporary experiments | |
1000 | 1099 | legacy | Rob Kutschke | numbers commonly used in simulation before this system | |
1100 | 1199 | database tests | Ray Culbertson | run dependence tests | |
1200 | 1299 | MDC2020 | Dave Brown (LBL) | MDC2020 production simulation with run dependence | |
1205 | 1205 | MDC2020 extracted | Dave Brown (LBL) | MDC2020 production simulation of extracted position | |
1300 | 1399 | RDM | Ray Culbertson | test data for file movement and processing | |
1400 | ... | ... | .... | next non-data reservations | |
100000 | 100999 | tracker VST | Richie Bonventre | tracker VST 2020-2021 | |
101000 | 101999 | STM test beam | Andy Edmonds | Dresden test beam 4/2022 | |
102000 | 102999 | module-0 | Eleonora Diociaiuti | Calorimeter module-0 | |
103000 | 103999 | CRV | Ralf Ehrlich / Simon Corrodi |
CRV data from the Wideband teststand | |
104000 | 104999 | ... | .... | next data reservations | |
105000 | 999999 | automatic | Antonio Gioiosa | automatic online database |
Simulation run numbers
Simulation will eventually need run dependence. A good example would be if a tracker plane is turned off or recovered during the run, then we will want to run a job which simulated the difference tracker configuration in the proper proportions. The plan is that when a simulation campaign starts, there will be the following steps
- the collaboration (Offline, Operations, Physics groups) will define periods of stable running called "simulation periods", recorded as run number ranges. We expect this will be on the order of 10's of periods per year. (We also expect "run periods" defined by production and analysis operational needs, and may or may not be similar to the simulation periods).
- for each simulation period, a representative set of conditions (typically database calibration tables) will be selected from, or derived from, the calibration data of the data runs in the period. More conditions data, appropriate only for simulation can be added.
- a sequential set of runs, from the non-data run ranges, will be reserved for this simulation campaign. The size of the set will essentially be equal to the number of simulation periods.
- The calibration manager will create a new purpose and version in the conditions database for this simulation campaign
- The conditions data prepared for each simulation period will be associated to a run in the reserved run range though the standard conditions database techniques. The data in these runs will be collected under the new purpose/version.
- simulation jobs will produce data in the reserved run ranges, with each run produced in proportion to the data taken during the corresponding simulation period, and conditions data taken from the new purpose/version.
- reconstruction jobs running on this simulation should be able to use the described conditions data.
This section is an outline. Additional detailed techniques will need to be defined for staging, mix-in options, and systematic variations of conditions.
Some History
talk
talk
talk
Ray talk page 3
Richie talk page 7
Approximate survey of existing run numbers. 1000 is current example for sim.
count runnumber 19 1 1 2 1 3 5 4 1 5 1 7 23 1000 10 1001 234 1002 6 1052 3 1053 1 1357 6 1500 11 1600 5 1700 10 1800 1 2000 1 2001 9 2700 20 2701 2 2705 57 4001 1 4002 3 5001 1 9002 1 645440 1 1792836 2 2102784 1 3344213 115 11911024-19236224