Speaker
Description
German large scale research facilities like DESY with its PETRA III synchrotron beamlines are strong partners for universities to conduct cutting-edge experiments, which generate vast amounts of data that often exceed the universities’ storage and compute capacities for analysis. Transferring data over wide area networks is becoming ever more impractical and expensive with the growing amount of data produced. In the project PHOTONIC, DESY and its university partners are developing a model for data access tailored to scientists’ (post-)beamtime needs in order to address the aforementioned challenge. At the same time, we hope to enable faster integration of data into analysis workflows and easier access in general by also offering live data access and visualisation possibilities.
The service that will be deployed and integrated with DESY's storage infrastructure is planned to be capable of understanding and interpreting data formats, offer transcoding functionalities and stream partial data by providing API-based access to photon science data. These functionalities will optimize data transfers by delivering only the needed subsets of data, reducing bandwidth and storage demands while accelerating analysis workflows. They also allow for more focused views on data without the need for transferring all available data.
The project is led by DESY (IT & Photon Science) and carried out with partners from universities over 3 years. Its' four positions to implement the functionalities described above are funded by the German Federal Ministry of Research, Technology and Space (BMFTR) and will take care of the backend and client integration tasks as well as training, dissemination and communication.
In this talk we will present a brief overview of the current data-access methods and their limitations followed by an outline of the project’s goals, methods & deployment architecture as well as expected outcomes.
