Parallel Computation in Clouds for Ecological Data Linking
"Embark on a journey of parallel computation in clouds, linking ecological data sources for standardized access to climate algorithms. Explore sustainable solutions for confidential data sharing, data subscription, delivery, and access control. Discover how to manage compute, storage, and access efficiently across multiple servers. Join a community of 150 researchers to build an interdisciplinary data science infrastructure with a focus on scalability and reproducibility."
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
PhenoMeNal (Enol) EuroArgo,SeaDataNet (Giuseppe) IndexMed (Jan) BirdHouse (Sy/Roberta) ICOS (Diego) Embarrassingly parallel computation on clouds Data subscription and delivery cloud Linking data from various ecological sources Standardised access to climate algorithms on the Internet Computing with data from the ICOS carbon portal Cloud, AARC, parallel computing EGI cloud, EUDAT storage, IdFederation EGI portfolio, Data/metadata registries, EUDAT? EGI portfolio, cloud, scheduling, access control Cloud, parallel computing, sustainability Confidential data (deployable in hospitals, data sharing?) Subscribe to data, system delivers it into personal cloud account Graph data structure to model data and links among data elements Managing compute, Storage, Access in a single server on top of 3rdparty systems Relatively early use cases two use cases IT-unaware users (e.g. doctors) Permanent storage in EU, US, Australia, Japan for continuously growing data For a growing community, of 150 interested researchers Deploy server/apps as Docker containers? Make observation data freely accessible on the Carbon Portal done with EUDAT Microservices orchestrated on IaaS clouds VMs for indexing data files (ElasticSearch) Collaborative building of graphs, from references extracted from DBs Which EGI service can be relevant? Facilitate usage of data in modelling activities 1st: Web service for data visualisation 2nd: Upload/download (with Docker?) MANTL by Cisco Cloud integrating all three XaaS layers Build an infrastructure of interdisciplinary data Science will come later Running on a few clouds, want to scale Need for PIDs/DOIs for reproducibility Sustainability? How does this work with EGI/EUDAT? 3/17/2025 out to EGI FedCloud 1
Design your e-infrastructure! https://indico.egi.eu/indico/event/2895/ Use case: Break out group coordinator: www.egi.eu EGI-Engage is co-funded by the Horizon 2020 Framework Programme of the European Union under grant number 654142
Group members 3/17/2025 3
First break-out Background and Users www.egi.eu EGI-Engage is co-funded by the Horizon 2020 Framework Programme of the European Union under grant number 654142
Who will be the user? Can the users be characterised? How many are they? 3/17/2025 5
What will be the value of the infrastructure for them? What will the system exactly deliver to them? 3/17/2025 6
How should they use the system? 3/17/2025 7
What's the timeline for development, testing and large-scale operation? (Consecutive releases can/should be considered.) 3/17/2025 8
Second break-out Design and implementation plan www.egi.eu EGI-Engage is co-funded by the Horizon 2020 Framework Programme of the European Union under grant number 654142
What should the first version include? - The most basic product prototype imaginable already bringing value to the users (the so-called Minimal Viable Product - MVP) 3/17/2025 10
Which components/services already exist in this architecture? 3/17/2025 11
Which components/services are under development (and by who)? 3/17/2025 12
Which components/services should be still brought into the system? Can EGI/EUDAT partners do it? 3/17/2025 13
Are there gaps in the EGI/EUDAT service catalogues that should be filled to realise the use case? Which service provider could fill the gap? 3/17/2025 14
3/17/2025 15
3/17/2025 16
3/17/2025 17
3/17/2025 18
