13-18 March 2016
Academia Sinica
Asia/Taipei timezone

Visualization of dCache billing files with modern BigData tools.

Mar 15, 2016, 4:00 PM
BHSS, Conf. Room 1 (Academia Sinica)

BHSS, Conf. Room 1

Academia Sinica

No. 128, Sec.2, Academia Road, Taipei, Taiwan
Oral Presentation Data Management Data Management Session II


Mr Tigran Mkrtchyan (DESY)


Running a data center is never a trivial job. In addition to daily routine tasks, service provider teams have to provide a meaningful information to upper management, end users and operators. The dCache production instances at DESY, produce gigabytes of billing files per day. Crunching the millions of numbers into a useful and handy information is unpleasant and boring task. Nevertheless, we can analyse and visualize this huge amount of log information with a modern BigData processing tools. This presentation will show how we built a real-time monitoring system which visualizes dCache billing files and provides intuitive, simple to use web interface using ElasticSearch, Logstash and Kibana.

Primary author


Dr Birgit Lewendel (DESY) Ms Lusine Yakovleva (DESY) Dr Patrick Fuhrmann (DESY/dCache.org)

Presentation materials