Distributed data management at LHC scales is a stagering task, accompained by equally challenging pratical management iussues with storage systems and wide-area networks. CMS data transfer management system, PhEDEx, is designed to handle this task with minimum operator effort, automating the workflows from large scale distribution of HEP experiment datasets down to reliable and scalable transfers of individual files over frequentlly unreliable infrastructures. Over the last year PhEDEx has matured to the point of handling virtually all CMS production data transfers. CMS pushes equally its own components to perform and the heavy investment into peer projects at all levels, from technical details to grid standards to world-wide projects, to ensure the end-to-end service is of sufficient quality. We present the throughput and service quality we have reached in the current daily 24/7 production work, the steps taken in LCG service challenges for the next generation transfer service, and the resulting changes in performance. We also report results from out scalability stress tests on PhEDEx alone. We offer an analysis of transfer-related problems we have encountered and how they have been affecting CMS data management.

J.Rhen, T.Barras, D.Bonacorsi, J.Hernandez, I.Semeniouk, L.Tuura, et al. (2006). PhEDEx High-Throughput Data Transfer Management System. NEW DELHI : MACMILLAN INDIA.

PhEDEx High-Throughput Data Transfer Management System

BONACORSI, DANIELE;
2006

Abstract

Distributed data management at LHC scales is a stagering task, accompained by equally challenging pratical management iussues with storage systems and wide-area networks. CMS data transfer management system, PhEDEx, is designed to handle this task with minimum operator effort, automating the workflows from large scale distribution of HEP experiment datasets down to reliable and scalable transfers of individual files over frequentlly unreliable infrastructures. Over the last year PhEDEx has matured to the point of handling virtually all CMS production data transfers. CMS pushes equally its own components to perform and the heavy investment into peer projects at all levels, from technical details to grid standards to world-wide projects, to ensure the end-to-end service is of sufficient quality. We present the throughput and service quality we have reached in the current daily 24/7 production work, the steps taken in LCG service challenges for the next generation transfer service, and the resulting changes in performance. We also report results from out scalability stress tests on PhEDEx alone. We offer an analysis of transfer-related problems we have encountered and how they have been affecting CMS data management.
2006
Computing in High Energy and Nuclear Physics(CHEP-2006)
1026
1029
J.Rhen, T.Barras, D.Bonacorsi, J.Hernandez, I.Semeniouk, L.Tuura, et al. (2006). PhEDEx High-Throughput Data Transfer Management System. NEW DELHI : MACMILLAN INDIA.
J.Rhen; T.Barras; D.Bonacorsi; J.Hernandez; I.Semeniouk; L.Tuura; Y.Wu
File in questo prodotto:
Eventuali allegati, non sono esposti

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11585/74177
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact