21-25 March 2022
Academia Sinica
Europe/Zurich timezone

Multiple Scenarios Oriented HTC Computing System Based on HTCondor at IHEP

22 Mar 2022, 14:00
30m
Room 1

Room 1

Oral Presentation Track 7: Network, Security, Infrastructure & Operations Network, Security, Infrastructure & Operations

Speaker

Xiaowei Jiang (Institute of High Energy Physics, Chinese Academy of Sciences)

Description

IHEP is a multi-disciplinary comprehensive research institution which is hosting or attending about 15 experiments around high energy physics, including LHAASO, BES, JUNO, HEPS, DYW, ALI, ATLAS, CMS, LHCb etc.
Corresponding to the multiple experiments, in the computing system, the multiple scenarios have to be considered and the proper technologies should be suitable for the different requirements from the multiple experiments and applications. The scenarios and the corresponding tech-architectures are: 1. A big sharing pool for massive offline data processing; 2. A unified sharing pool with a special resource policy for WLCG and JUNO distributed computing grid; 3. dHTC pool for resources sharing between HTC and HPC and between sites; 4. Dedicated pools for on-site data pre-processing; 5. Realtime computing pool for astrophysics streaming data processing; 6. Customized clusters for the cooperated institutions or universities.
HTCondor is a relative popular distributed batch system in HEP computing. This paper will make discussions on the multi-scenarios HTC computing system based on HTCondor at IHEP,including how to consider and design the architecture, how to take advantage of HTCondor’s functions and what we have done on research and development with HTCondor. Currently, the whole computing system is managing about 50,000 CPU cores (including x86-arch CPU and arm-arch CPU) and some GPU cards.

Primary authors

Xiaowei Jiang (Institute of High Energy Physics, Chinese Academy of Sciences) Prof. Jingyan Shi (IHEP)

Co-authors

Ran Du (Institute of High Energy Physics, Chinese Academy of Sciences) Qingbao Hu (IHEP) Prof. Gongxing Sun (IHEP)

Presentation materials