CodeDevelopment

Introduction

A page to write some principles and questions for code development.

• 13265 Andrei's recent talk
• 7591 Rob's offline work list
• 6240 Ray's older talk
• 6077 Rob's comments
• 6053 Andrei's older talk
• 5741 Andrei's older talk

Use Cases

Each level incorporates the levels above.

1. user - setup pre-built code and run utilities and standard mu2e fcl. Example: compare filter rates by modifying fcl, make a gdml file, run tutorials. Needs simple and fool-proof procedures.
2. analyzer - checkout, build user code against pre-built offline, commit changes. Example: user makes ntuples from personal code or Stntuple. Needs user code well-incorporated with offline - possibly using the same build system, possibly coordinated directory structures and commands. Fool-proof user-to-offline dependencies.
3. analyzerPlus - checkout, build user code against pre-built offline, commit changes. Also checkout offline code consistent with prebuilt release and make some small modificstion. Do not commit back to offline. Example: user makes ntuples and needs to tweak seom algorithm code to get the right variable out in the ntuple. Needs quick and reliable partial build of arbitrary pieces. Needs to catch dependency mistakes. Needs to switch quickly between prof and debug.
4. minor - make localized modifications to offline head or branch, test and commit them, as in specific product maintenance. Examples: fix a bug in an algorithm, add an accessor. Needs quick and reliable partial build of arbitrary pieces. Needs to catch dependency mistakes. Needs to switch quickly between prof and debug.
5. major - make larger and more widespread changes, test and commit them. Examples: modifying major include files, reco sequences, products. Probably needs full build of offline. Needs quick checkout and build.
6. expert - infrequent complex development - move code between packages, build prototype products with offline, build code releases, build trigger code packages.

In my opinion, the system should serve the first three or four primarily, under the assumption that the others are done less often and are done by people who are less likely to make mistakes.

Some Issues

• is it amenable to a tutorial?
• how will it run on OSG without /mu2e/app?
• will there be a different build time vs run time environment?
• is initial setup different from re-setup?
• what windows does the user have open? How does one know what a window is good for?
• what are likely points of confusion?
• is the directory structure organized and intuitive? Is offline separate from user code?
• is it fast and easy for an analyzer or junior to start a new project?
• what does a user type most often? Can it be shorter and simpler?
• can all dependencies be checked automatically?
• what does switching between prof and debug look like?
• how does an analyzer switch between pre-built releases?
• what build commands are supported for user code, how much are they forced to be coordinated with offline?
• are there clear lines of code responsibility?
• who coordinates pieces and tags and validates the whole (and how)?
• does it support code reviews and pull requests?
• does it support out-of-source builds?
• what parts of the dependencies are tagged and which are outside the tag?
• what do paths look like? do they include "mu2e"?
• what does validation do - continuous and nightly?
• what is the maintenance effort required?
• does it follow KISS?
• is it staged? does it fit into a long-term plan?
• is there a transition plan? How disruptive is it?
• does a major change like introducing the calibration database cause unnecessary disruption?
• is git history preserved?
• have we considered the best ways to speed up compilation?
• split out fcl partially or fully?
• split out Analysis package?
• consider separating sim and reco more sharply

Summary of packages

Many of the longest compile times are in libraries where little or none of the code is needed by 95% of users.

                             files   modules lines  compile time (s)
                  Alignment     14      0    331     14
                   Analyses    162     98  28297    868
                 BFieldGeom     18      0    770     10
                 BFieldTest      5      2    444     22
                   BTrkData      6      0    428     13
                 BTrkHelper     11      0    169     21
               BeamlineGeom     15      0      7      5
                CRVAnalysis      5      0    142     16
                CRVResponse    134     11   6991    123
          CRYEventGenerator      6      1    543     26
                  CalPatRec    108     15  21857    219
                CaloCluster     19      3   1212     46
                     CaloMC     15      4   1909     63
                   CaloReco     13      2   1715     52
            CalorimeterGeom     17      0    708      7
             ConditionsBase      1      0      0      0
          ConditionsService     64      0    379     20
                ConfigTools     10      0   1958     11
        CosmicRayShieldGeom     25      0    485      9
        CutAndCountAnalysis      7      2    495     50
                        DAQ     16      6   2261     64
            DAQDataProducts      6      0    128      8
               DataProducts     28      0    320     10
       DetectorSolenoidGeom      4      0      5      4
               EventDisplay     39      1   3432     54
             EventGenerator     91     25  10075    385
                EventMixing     36      6   1561     68
      ExternalShieldingGeom      7      0     51      9
      ExtinctionMonitorFNAL    168     29   8634    310
                    Filters     33     31   6769    293
                   G4Helper      7      0    169     18
           GeneralUtilities     48      0   1069     22
             GeomPrimitives     19      0    242      6
            GeometryService     86      1  11907    160
     GlobalConstantsService     15      0    639     25
                 HelloWorld      8      3    197     26
                  HitMakers     21      9   4054    102
                  JobConfig    113      0      0      0
                KalmanTests     17      0      0      0
                    MBSGeom      2      0      0      0
             MCDataProducts     99      0   1192     41
MECOStyleProtonAbsorberGeom      4      0     56      4
                   Mu2eBTrk      8      0    267     17
                     Mu2eG4    810      2  39016    536
               Mu2eHallGeom      5      0     58      8
             Mu2eInterfaces      7      0     33      5
                   Mu2eReco      4      0      0      0
              Mu2eUtilities     92      0   3806    122
                 ParticleID     19      4   3867     70
                      Print     75      3   4011    155
     ProductionSolenoidGeom     11      0     98     11
       ProductionTargetGeom      5      0     14      8
                   Projects     71      1    387      0
         ProtonBeamDumpGeom      5      0     47      7
           RecoDataProducts    126      1   2247     44
               ReleaseNotes    188      0      0      0
               RootAnalyzer      5      0      0      0
                 SConstruct      1      0      0      0
                    STMGeom      7      0      0      0
                    Sandbox     83     29   2516    222
                SeedService     20      1    450     17
               ServicesGeom      5      0     35      7
                    Sources      7      0    783     32
         StoppingTargetGeom      6      0     58      4
               TTrackerGeom     14      0    175      7
                  TestTools      3      0     84      3
          TrackCaloMatching     33      6   3250     77
          TrackerConditions      9      0    704     27
                TrackerGeom     12      0    367      7
                  TrackerMC     20      1   1621     32
                    Trigger      2      1     81     15
                 TrkAdapter      4      0     31     16
              TrkChargeReco     19      0    541     18
                    TrkDiag     93     10   5288    130
                     TrkExt     21      1   3143     49
                 TrkFilters     15      5   1012     61
                  TrkPatRec     19      4   2009     71
                    TrkReco     33      0   4407     85
             UtilityModules     11      2    132     22
                 Validation     57      1   2923     83