Rucio: Difference between revisions

From Mu2eWiki
Jump to navigation Jump to search
Line 28: Line 28:
* [https://indico.cern.ch/event/813752/contributions/4051564/attachments/2122693/3573056/DUNE_Rucio_GDB_2020_Oct_Wenlong.pdf DUNE status]
* [https://indico.cern.ch/event/813752/contributions/4051564/attachments/2122693/3573056/DUNE_Rucio_GDB_2020_Oct_Wenlong.pdf DUNE status]
* Metacat [https://cdcvs.fnal.gov/redmine/projects/metacat redmine] [https://metacat.readthedocs.io/en/latest/ docs][https://docs.google.com/presentation/d/17inkkOch1MyKQmy6gEO6iCVOXI5EgjtaS1bcKeRzINc/edit?usp=sharing CHEP talk] [https://cdcvs.fnal.gov/redmine/attachments/download/64700/MetaCat%20CHEP%202021%20paper%20v5.pdf CHEP paper] [https://github.com/ivmfnal/metacat.git github] [https://metacat.fnal.gov:9443/dune_meta_demo/app/gui/index DUNE GUI]
* Metacat [https://cdcvs.fnal.gov/redmine/projects/metacat redmine] [https://metacat.readthedocs.io/en/latest/ docs][https://docs.google.com/presentation/d/17inkkOch1MyKQmy6gEO6iCVOXI5EgjtaS1bcKeRzINc/edit?usp=sharing CHEP talk] [https://cdcvs.fnal.gov/redmine/attachments/download/64700/MetaCat%20CHEP%202021%20paper%20v5.pdf CHEP paper] [https://github.com/ivmfnal/metacat.git github] [https://metacat.fnal.gov:9443/dune_meta_demo/app/gui/index DUNE GUI]
* DataDispatcher [https://github.com/ivmfnal/data_dispatcher github] [https://data-dispatcher.readthedocs.io/en/latest/ docs]

Revision as of 21:04, 15 June 2023

Introduction

Rucio is a CERN software system for storing file metadata and organizing the delivery of that data to users. Its primary features are scalability, flexibility, adaptive file replication, and built-in monitoring. It can use various backends for databases, various platforms for its servers and daemons, various transfer and storage method plug-ins, and a command line and python interface for users.

The new system would consist of these parts

  • Metacat - a database of file metadata (docs)
  • Rucio - a database of file locations, and servers which can move and track data, responding to user rules (docs)
  • Data Dispatcher - a modern replacement for SAM project file delivery ()
  • mdh - Mu2e data-handling commands added to supplement the above systems (see mdh -h)

Implementation

export METACAT_SERVER_URL=http://dbweb5.fnal.gov:9094/mu2e_meta_prod/app
export METACAT_AUTH_SERVER_URL=https://metacat.fnal.gov:8143/auth/mu2e
export DATA_DISPATCHER_URL=https://metacat.fnal.gov:9443/mu2e_dd_prod/data
export DATA_DISPATCHER_AUTH_URL=https://metacat.fnal.gov:8143/auth/mu2e

References