21-25 March 2022
Academia Sinica
Europe/Zurich timezone

Adapting HTCondor fairshare for mixed workloads

Mar 25, 2022, 11:20 AM
20m
Room 1

Room 1

Oral Presentation Track 8: Infrastructure Clouds and Virtualizations Infrastructure Clouds and Virtualisation

Speaker

Stefano Dal Pra (Istituto Nazionale di Fisica Nucleare)

Description

The INFN Tier-1 data centre is the main italian computing site for scientific communities on High Energy Physics and astroparticle research. Access to the resources is arbitrated by a HTCondor batch system which is in charge of balancing the overall usage by several competing user groups according to their agreed quotas. The set of different workloads submitted to the computing cluster is highly heterogeneous and a very rich set of different requirements is to be considered by the batch system in order to provide user groups with a satisfactory fair share over the available resources. To prevent or reduce usage disparities a system to self adjust imbalances has been developed and it is being used with satisfactory results. This work explain how and when fair share implementations can miss optimal performances and describes a general method to improve them. Results of the current solution are presented and possible further developments are discussed.

Primary authors

Mr Carmelo Pellegrino (INFN) Stefano Dal Pra (Istituto Nazionale di Fisica Nucleare)

Presentation materials