Next Generation Data Management Services: the eXtreme DataCloud project

20 Mar 2018, 14:00
30m
Conference Room 1, BHSS (Academia Sinica)

Conference Room 1, BHSS

Academia Sinica

Oral Presentation Big Data & Data Management Data Management & Big Data Session

Speaker

Daniele Cesini (INFN-CNAF)

Description

The eXtreme DataCloud (XDC) is a EU H2020 funded project aimed at developing scalable technologies for federating storage resources and managing data in highly distributed computing environments. The services provided will be capable of operating at the unprecedented scale required by the most demanding, data intensive, research experiments in Europe and Worldwide. XDC will be based on existing tools, whose technical maturity is proved and that the project will enrich with new functionalities and plugins already available as prototypes (TRL6+) that will be brought at the production level (TRL8+) at end of XDC. The targeted platforms are the current and next generation e-Infrastructures deployed in Europe, such as the European Open Science Cloud (EOSC), the European Grid Infrastructure (EGI), the Worldwide LHC Computing Grid (WLCG) and the computing infrastructures funded by other public and academic initiatives. The main high-level topics addressed by the project include: i) federation of storage resources with standard protocols, ii) smart caching solutions among remote locations, ii) policy driven data management based on Quality of Service, data lifecycle management, metadata handling and manipulation, data preprocessing and encryption during ingestion, optimized data management based on access patterns. All the developments will be community-driven and tested against real life use cases provided by the consortium partners representing research communities belonging to a variety of scientific domains: Life Science, Astrophysics, High Energy Physics, Photon Science and Clinical Research. The XDC project aims at opening new possibilities to scientific research communities in Europe and worldwide by supporting the evolution of e-Infrastructure services for Exascale data resources. The XDC software will be released as Open Source platforms available for general exploitation.

Primary author

Daniele Cesini (INFN-CNAF)

Co-authors

Dr Alessandro Costantini (INFN-CNAF) Dr Christian Ohmann (ECRIN) Dr Cristina Duma (INFN-CNAF) Dr Fernando Aguilar (University of Cantabria) Dr Giacinto Donvito (INFN-BARI) Dr Lukasz Dutka (AGH-CYFRONET) Dr Patrick Fuhrmann (DESY/dCache.org) Dr Serena Battaglia (ECRIN) Dr Vincent Poireau (CNRS-LAPP) Prof. de Lucas Jesus Marco (University of Cantabria) Dr matthew Viljoen (EGI) Dr oliver keeble (CERN) Dr rachid lemrani (CNRS)

Presentation materials