Distributed data management at LHC scales is a stagering task, accompained by equally challenging pratical management iussues with storage systems and wide-area networks. CMS data transfer management system, PhEDEx, is designed to handle this task with minimum operator effort, automating the workflows from large scale distribution of HEP experiment datasets down to reliable and scalable transfers of individual files over frequentlly unreliable infrastructures. Over the last year PhEDEx has matured to the point of handling virtually all CMS production data transfers. CMS pushes equally its own components to perform and the heavy investment into peer projects at all levels, from technical details to grid standards to world-wide projects, to ensure the end-to-end service is of sufficient quality. We present the throughput and service quality we have reached in the current daily 24/7 production work, the steps taken in LCG service challenges for the next generation transfer service, and the resulting changes in performance. We also report results from out scalability stress tests on PhEDEx alone. We offer an analysis of transfer-related problems we have encountered and how they have been affecting CMS data management.
PhEDEx High-Throughput Data Transfer Management System / J.Rhen; T.Barras; D.Bonacorsi; J.Hernandez; I.Semeniouk; L.Tuura; Y.Wu. - STAMPA. - II:(2006), pp. 1026-1029. (Intervento presentato al convegno CHEP 2006 Computing in High Energy and Nuclear Physics tenutosi a Mumbai,India nel 13-17 Febbraio 2006).
PhEDEx High-Throughput Data Transfer Management System
BONACORSI, DANIELE;
2006
Abstract
Distributed data management at LHC scales is a stagering task, accompained by equally challenging pratical management iussues with storage systems and wide-area networks. CMS data transfer management system, PhEDEx, is designed to handle this task with minimum operator effort, automating the workflows from large scale distribution of HEP experiment datasets down to reliable and scalable transfers of individual files over frequentlly unreliable infrastructures. Over the last year PhEDEx has matured to the point of handling virtually all CMS production data transfers. CMS pushes equally its own components to perform and the heavy investment into peer projects at all levels, from technical details to grid standards to world-wide projects, to ensure the end-to-end service is of sufficient quality. We present the throughput and service quality we have reached in the current daily 24/7 production work, the steps taken in LCG service challenges for the next generation transfer service, and the resulting changes in performance. We also report results from out scalability stress tests on PhEDEx alone. We offer an analysis of transfer-related problems we have encountered and how they have been affecting CMS data management.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
