Difference between revisions of "Tracker Databases"

From Mu2eWiki
Jump to navigation Jump to search
Line 1: Line 1:
 
= Tracker Databases =
 
= Tracker Databases =
There are a few tracker databases hosted at different sites. This wiki page currently focuses on the ones that interact with Fermilab-hosted <code>mu2e_tracker_prd</code> database and use tools available in the [https://github.com/Mu2e/TrackerHardwareDB TrackerHardwareDB] git repository.
+
There are a few tracker databases hosted at different sites. This wiki page currently focuses on the ones that interact with Fermilab-hosted <code>mu2e_tracker_prd</code> database and uses tools available in the [https://github.com/Mu2e/TrackerHardwareDB TrackerHardwareDB] git repository. If you have any questions or comments, contact Andy Edmonds.
  
 
== Setting Up ==
 
== Setting Up ==
Line 14: Line 14:
 
  git pull
 
  git pull
  
== FNAL Planes Database ==
+
= FNAL Planes Database =
 
The database for the plane construction at FNAL. The working version is not hosted on <code>mu2e_tracker_prd</code> but is copied to the <code>imported.fnal_planes</code> database table in <code>mu2e_tracker_prd</code>.
 
The database for the plane construction at FNAL. The working version is not hosted on <code>mu2e_tracker_prd</code> but is copied to the <code>imported.fnal_planes</code> database table in <code>mu2e_tracker_prd</code>.
  
=== Importing ===
+
== Importing ==
 
To import to <code>mu2e_tracker_prd</code>:
 
To import to <code>mu2e_tracker_prd</code>:
  
Line 23: Line 23:
 
  python3 create_imported_fnal_planesdb.py path/to/fnal/planesdb
 
  python3 create_imported_fnal_planesdb.py path/to/fnal/planesdb
  
This will create two .sql files. You will need mu2e_tracker_admin privileges to then do
+
This will create two .sql files. You will need the mu2e_tracker_admin database role to then do
 
  cd ../sql
 
  cd ../sql
 
  psql -h ifdb08 -p 5459 mu2e_tracker_prd < create_imported_fnal_planesdb.sql
 
  psql -h ifdb08 -p 5459 mu2e_tracker_prd < create_imported_fnal_planesdb.sql
Line 30: Line 30:
 
The information can then be viewed using the panel QC viewer below.
 
The information can then be viewed using the panel QC viewer below.
  
== Panel QC Database ==
+
= Panel QC Database =
 
The panel QC database that is stored in mu2e_tracker_prd
 
The panel QC database that is stored in mu2e_tracker_prd
  
=== Viewer ===
+
== Viewing Database ==
 +
The panel QC database viewer can be run on any machine connected to the internet with Node.js installed.
  
==== Installation ====
+
Below are instructions to run the viewer on the mu2egpvm machines but in principle it is runnable from anywhere.
 +
 
 +
=== Installation ===  
 
To install the panel QC viewer:
 
To install the panel QC viewer:
  
Line 41: Line 44:
 
  npm install
 
  npm install
  
==== Running ====
+
=== Running ===
 
To run the panel QC viewer, you need to forward a port when you ssh to the mu2egpvm machine:
 
To run the panel QC viewer, you need to forward a port when you ssh to the mu2egpvm machine:
  
Line 54: Line 57:
 
   PORT=XXXX npm start
 
   PORT=XXXX npm start
  
=== Updating ===
+
== Updating Database ==
 
There are a few python scripts that can be used to update the <code>qc.panels</code> table. To set up to run the python scripts you need to do the following:
 
There are a few python scripts that can be used to update the <code>qc.panels</code> table. To set up to run the python scripts you need to do the following:
  
Line 62: Line 65:
 
  setup pyana
 
  setup pyana
  
==== Adding a new panel ====
+
Then each scripts follows a similar pattern:
 +
 
 +
# Run a python scripts to create a .sql file
 +
# Check the .sql file for any obvious errors
 +
# Run commands contained in the .sql file. This puts the data in the database
 +
 
 +
To put data in the database, you need to have the <code>mu2e_tracker_writer</code> database role
  
  python3 create_insert_new_panel.py --panel_ids X Y Z
+
=== Adding a new panel ===
 +
 
 +
cd TrackerHardwareDB/python
 +
  python3 create_insert_new_panels.py --panel_ids X Y Z
 +
psql -h ifdb08 -p 5459 mu2e_tracker_prd < insert_new_panels.sql
  
 
will create .sql files to insert new rows for panels X, Y, and Z.
 
will create .sql files to insert new rows for panels X, Y, and Z.
  
==== Changing missing_straws, high_current_wires, etc. ====
+
=== Changing missing_straws, high_current_wires, etc. ===
  
 +
cd TrackerHardwareDB/python
 
  python3 create_update_qc_panels_table.py --panel_id XX --{new/add/remove}_{column1} A B C --{new/add/remove}_{column2} D E F
 
  python3 create_update_qc_panels_table.py --panel_id XX --{new/add/remove}_{column1} A B C --{new/add/remove}_{column2} D E F
 +
psql -h ifdb08 -p 5459 mu2e_tracker_prd < update_qc_panels_table.sql
  
 
will create .sql files to update panel XX.
 
will create .sql files to update panel XX.
Line 87: Line 102:
  
 
  python3 create_update_qc_panels_table.py --panel_id 121 --remove_missing_wires 5 --add_loose_omega_pieces 88 89
 
  python3 create_update_qc_panels_table.py --panel_id 121 --remove_missing_wires 5 --add_loose_omega_pieces 88 89
 +
 +
=== Adding Fe55 raw data ===
 +
From the panel QC Fe55 tests, we get csv files named like mn269_ch0_r3_delta.csv and mn269_ch1_r3_delta.csv. The data contained in here can be put into the database like so:
 +
 +
cd TrackerHardwareDB/python
 +
python3 create_update_maxerf_risetime.py ../data/mn269_ch0_r3_delta.csv ../data/mn269_ch1_r3_delta.csv
 +
psql -h ifdb08 -p 5459 mu2e_tracker_prd < update_maxerf_risetime.sql
 +
 +
The python script has some internel checks to make sure there are the correct number of channels and no duplicate channel numbers
 +
 +
=== Adding Fe55 plots ===
 +
These are just stored in the repository so to add these do:
 +
 +
cd TrackerHardwareDB/panel-qc-viewer-public/images/hv_data/
 +
cp /path/to/images/mn*.png .
 +
git add *.png
 +
git commit -m "Adding hv data plots"
 +
git push
 +
 +
Then for other people to see the new images, they will need to pull in the updates.

Revision as of 20:13, 23 May 2023

Tracker Databases

There are a few tracker databases hosted at different sites. This wiki page currently focuses on the ones that interact with Fermilab-hosted mu2e_tracker_prd database and uses tools available in the TrackerHardwareDB git repository. If you have any questions or comments, contact Andy Edmonds.

Setting Up

To use the tools described on this wiki page, you need to clone the TrackerHardwareDB git repository. The following instructions are for running on the mu2egpvms but some tools (e.g. the panel QC viewer) could be run from any machine:

ssh -XY username@mu2egpvmXX.fnal.gov
cd /mu2e/app/users/username/
git clone https://github.com/Mu2e/TrackerHardwareDB

To get any code updates, you should periodically do

cd TrackerHardwareDB
git pull

FNAL Planes Database

The database for the plane construction at FNAL. The working version is not hosted on mu2e_tracker_prd but is copied to the imported.fnal_planes database table in mu2e_tracker_prd.

Importing

To import to mu2e_tracker_prd:

cd TrackerHardwareDB/python
python3 create_imported_fnal_planesdb.py path/to/fnal/planesdb

This will create two .sql files. You will need the mu2e_tracker_admin database role to then do

cd ../sql
psql -h ifdb08 -p 5459 mu2e_tracker_prd < create_imported_fnal_planesdb.sql
psql -h ifdb08 -p 5459 mu2e_tracker_prd < insert_imported_fnal_planesdb.sql

The information can then be viewed using the panel QC viewer below.

Panel QC Database

The panel QC database that is stored in mu2e_tracker_prd

Viewing Database

The panel QC database viewer can be run on any machine connected to the internet with Node.js installed.

Below are instructions to run the viewer on the mu2egpvm machines but in principle it is runnable from anywhere.

Installation

To install the panel QC viewer:

cd TrackerHardwareDB/panel-qc-viewer/
npm install

Running

To run the panel QC viewer, you need to forward a port when you ssh to the mu2egpvm machine:

ssh -L 3000:localhost:3000 -XY username@mu2egpvmXX.fnal.gov
cd TrackerHardwareDB/panel-qc-viewer
npm start

Then you can open a browser and go to localhost:3000 on your local machine

To run on a different port you can type:

 PORT=XXXX npm start

Updating Database

There are a few python scripts that can be used to update the qc.panels table. To set up to run the python scripts you need to do the following:

cd TrackerHardwareDB/python
setup mu2e
muse setup Offline
setup pyana

Then each scripts follows a similar pattern:

  1. Run a python scripts to create a .sql file
  2. Check the .sql file for any obvious errors
  3. Run commands contained in the .sql file. This puts the data in the database

To put data in the database, you need to have the mu2e_tracker_writer database role

Adding a new panel

cd TrackerHardwareDB/python
python3 create_insert_new_panels.py --panel_ids X Y Z
psql -h ifdb08 -p 5459 mu2e_tracker_prd < insert_new_panels.sql

will create .sql files to insert new rows for panels X, Y, and Z.

Changing missing_straws, high_current_wires, etc.

cd TrackerHardwareDB/python
python3 create_update_qc_panels_table.py --panel_id XX --{new/add/remove}_{column1} A B C --{new/add/remove}_{column2} D E F
psql -h ifdb08 -p 5459 mu2e_tracker_prd < update_qc_panels_table.sql

will create .sql files to update panel XX.

Some examples:

  • This will replace the current contents of missing_straws column for panel 53 with straw numbers 1, 2, 3.
python3 create_update_qc_panels_table.py --panel_id 53 --new_missing_straws 1 2 3
  • This will add the straw number 4 to the current list of high_current_wires for panel 76:
python3 create_update_qc_panels_table.py --panel_id 76 --add_high_current_wires 4
  • This will remove the straw number 5 from the current list of missing_wires and add the straw numbers 88 and 89 to the current list of loose_omega_pieces for panel 121:
python3 create_update_qc_panels_table.py --panel_id 121 --remove_missing_wires 5 --add_loose_omega_pieces 88 89

Adding Fe55 raw data

From the panel QC Fe55 tests, we get csv files named like mn269_ch0_r3_delta.csv and mn269_ch1_r3_delta.csv. The data contained in here can be put into the database like so:

cd TrackerHardwareDB/python
python3 create_update_maxerf_risetime.py ../data/mn269_ch0_r3_delta.csv ../data/mn269_ch1_r3_delta.csv
psql -h ifdb08 -p 5459 mu2e_tracker_prd < update_maxerf_risetime.sql

The python script has some internel checks to make sure there are the correct number of channels and no duplicate channel numbers

Adding Fe55 plots

These are just stored in the repository so to add these do:

cd TrackerHardwareDB/panel-qc-viewer-public/images/hv_data/
cp /path/to/images/mn*.png .
git add *.png
git commit -m "Adding hv data plots"
git push

Then for other people to see the new images, they will need to pull in the updates.