Speaker
Description
The European Centre for Medium-Range Weather Forecasts (ECMWF) operates as both a world-leading provider of global numerical weather predictions and a major research organization advancing Earth system science.
Within its mission to deliver reliable data, forecasts, and insights for the benefit of society, ECMWF is actively involved in several European initiatives such as EO4EU [1], MediTwin [2], CLIMRES [3], and BUILDSPACE [4]. These projects aim to create advanced services and applications that simplify access to Earth Observation (EO) data and leverage AI/ML techniques to deliver high-level analytics, decision-support tools, and user-oriented services across multiple domains.
To support these projects, we have developed a fully automated, cloud-agnostic infrastructure capable of managing multiple Kubernetes clusters deployed across geographically distributed cloud environments. This setup ensures that the deployment of services and applications is secure, reproducible and scalable. Automation tools such as Git, Terraform, Ansible and Fleet are integrated into a GitOps-driven workflow, enabling consistent cluster management and streamlined application delivery.
The infrastructure also supports AI/ML-oriented workloads, including the automated deployment of JupyterHub environments with GPU-enabled nodes managed through the NVIDIA GPU Operator, allowing for efficient resource partitioning and dynamic scaling of GPU resources.
Building on this foundation, we are now exploring the integration of Cluster API, Crossplane, and Sveltos to enable declarative cluster provisioning and dynamic add-on management, with the goal of making the platform even more automated, adaptive, and interoperable through GitOps-based workflows.
The oral contribution will provide implementation details of this approach and discuss how these technologies are being leveraged to support AI-driven scientific workflows across distributed cloud infrastructures.
References:
[1] https://www.eo4eu.eu/
[2] https://meditwin-project.eu/
[3] https://climres.eu/
[4] https://buildspaceproject.eu/
