Distributed Data Management System at IHEP (Remote presentation)

Mar 23, 2023, 2:00 PM
30m
Auditorium (BHSS, Academia Sinica)

Auditorium

BHSS, Academia Sinica

Oral Presentation Track 6: Data Management & Big Data Data Management & Big Data

Speaker

Xuantong Zhang (IHEP, CAS)

Description

A Distributed Grid-Data Management System serving BES, JUNO, CEPC experiments has been built at IHEP since 2014 based on DIRAC file catalog system. Meanwhile, more experiments, such as HERD or JUNO experiment with different data scales and complicated data management demands in data production, enforce us to make attempts on developing more flexible and experiments scenarios-oriented grid-data management system.
Considering its reliability, scalability and automation, the Rucio system became an option to IHEP grid data management infrastructures. In order to better understand Rucio system ability and performance, we designed its roles in current data production workflow and customized data policies corresponding to different experiments requirements.
This talk will introduce the design of JUNO/HERD experiment data flow and rucio’s integration into present distributed data management system. The progress of Third-party-copy protocol adaption and token-based authentication system (IAM) deployment at IHEP, which work as the grid data base services and infrastructures, will also be included in this presentation.

Primary authors

Xuantong Zhang (IHEP, CAS) Xiaomei Zhang (IHEP, CAS)

Presentation materials