13-18 March 2016
Academia Sinica
Asia/Taipei timezone

Visualization of dCache billing files with modern BigData tools.

Mar 15, 2016, 4:00 PM
30m
BHSS, Conf. Room 1 (Academia Sinica)

BHSS, Conf. Room 1

Academia Sinica

No. 128, Sec.2, Academia Road, Taipei, Taiwan
Oral Presentation Data Management Data Management Session II

Speaker

Mr Tigran Mkrtchyan (DESY)

Description

Running a data center is never a trivial job. In addition to daily routine tasks, service provider teams have to provide a meaningful information to upper management, end users and operators. The dCache production instances at DESY, produce gigabytes of billing files per day. Crunching the millions of numbers into a useful and handy information is unpleasant and boring task. Nevertheless, we can analyse and visualize this huge amount of log information with a modern BigData processing tools. This presentation will show how we built a real-time monitoring system which visualizes dCache billing files and provides intuitive, simple to use web interface using ElasticSearch, Logstash and Kibana.

Primary author

Co-authors

Dr Birgit Lewendel (DESY) Ms Lusine Yakovleva (DESY) Dr Patrick Fuhrmann (DESY/dCache.org)

Presentation materials