Euro-Argo and Copernicus Marine Service Integration for Ocean Observation Infrastructure
Explore the integration of Euro-Argo and Copernicus Marine Service within the ocean observation infrastructure supported by the ENVRIPLUS EU project. Learn about the timeline, user interactions, data subscription processes, and identity federation schemes involved in this comprehensive community-driven initiative.
Uploaded on | 0 Views
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Background, Background, ocean ocean observation e observation e- -infrastructure infrastructure Who is the community/project the use case belongs to? ENVRIPLUS EU project Ocean observations : Euro-Argo, SeaDataNet, Copernicus Marine service What's the timeline for development, tests and large-scale operation? First implementation in 2016 Large scale operations from 2017 What's your role in the use case? Any experience/link to EGI? Implement Euro-Argo and Copernicus use case Involve SeaDataNet, SOCAT and ESONET in a next step Link with EGI through ENVRIPLUS EGI workshop for E-infrastructure, April 2016 Amsterdam
Users Users Who will be the users of the planned community-specific e-infrastructure? How many of them? Environmental monitoring and forecasting : EU ocean-atmosphere models Calibration and validation observations : SMOS, Sentinel3 satellite missions Trans-disciplinary communities (ocean - atmosphere - biology - solid earth) Convenient space to associate storage and CPU for advanced services (such as geo-spatial analysis) Input stream or digital collaborative space for researchers or data providers How will the users interact with the system - Show this on a system architecture diagram Who and how would validate the system? Who : ENVRIPLUS project, Euro-Argo ERIC, SeaDataNet infrastructure How : use case review EGI workshop for E-infrastructure, April 2016 Amsterdam
ENVRIPLUS data ENVRIPLUS data subscription subscription use case use case The user provides his criteria time, spatial, parameter, data type update period for delivery (daily, monthly, yearly, on the spot) The relevant data are extracted from ENVRIPLUS cloud Data may be converted/transformed on ENVRIPLUS grid WPS ? The user s cloud account is updated regularly with the new data provided above An accounting of data delivery is performed MDC ? A citation scheme is attached to the delivered data DOI bibliographic surveys can track the use of these data in publications reproducibility is possible A users identification scheme is implemented Federation of identities : Marine-ID, Shibboleth, OpenID EGI workshop for E-infrastructure, April 2016 Amsterdam
ENVRIPLUS ENVRIPLUS cloud cloud service service Euro-Argo, SeaDataNet, Copernicus datasets 5 billion ocean observations 300 parameters 15 000 observing platforms from 1900 to today. This cloud of observations is pushed and continuously updated on ENVRIPLUS cloud (EGI, EUDAT, ) Copies are replicated in different places, close to users location (EU, US, Australia, Japan) EGI workshop for E-infrastructure, April 2016 Amsterdam
Argo Argo floats floats observations in 2015 (& observations in 2015 (& others others) ) Vertical profiles, 2015 observations, vertical profiles Argo floats - sea-mammals - XBT and CTDs from vessels - gliders EGI workshop for E-infrastructure, April 2016 Amsterdam
Ocean Ocean observation cloud data model observation cloud data model Observation data model : a flat table of 5 billion records ID platformCode dataType x y z t parameter value value_qc Observation metadata JSON collections of metadata (platform codes, parameter codes, data types) Observations are hosted in a workplace such as Hadoop, NoSQL files or PostgreSQL The use of in-memory features would provide the best reactivity (instant answers) Metadata are indexed with ElasticSearch The workplace is activated in a virtual server, replicated on the cloud EGI workshop for E-infrastructure, April 2016 Amsterdam
Federated Federated cloud cloud data distribution data distribution Research Infrastructure Infrastructure Infrastructure Research Research data VM my cloud data VM my cloud data EGI workshop for E-infrastructure, April 2016 Amsterdam
ENVRIPLUS e ENVRIPLUS e- -infrastructure architecture infrastructure architecture Logs management accounting Kibana Data a metadata workplace Drill, HBASE Data subscription service OwnCloud CSV & JSON data upload Drill Data extraction Drill, SQL, PIG Data discovery and visualization OpenLayer3, AngularJS, BootStrap, html5/css3, Material design G ographique index and facets ElasticSearch, Drill EGI workshop for E-infrastructure, April 2016 Amsterdam
Ocean Ocean observation e observation e- -services services Data services to be developed around Euro-Argo cloud Ocean observations API : profiles, time-series, trajectories Metadata visualization : map wms services Data visualization : graphics Data products such as mixed-layer depths maps Agile and incremental implementation Step 1 : Euro-Argo data file on the cloud (1to file daily updated) Step 2 : VM for indexation of data file (ElasticSearch) Step 3 : data file generation service (CSV then NetCDF) Step 4 : data subscription/distribution service to OwnCloud accounts Step 5 : replicate data and VM in mirror sites (EU, US, AU, JP) Next steps : promote the development of various services around an ENVRIPLUS cloud EGI workshop for E-infrastructure, April 2016 Amsterdam
Current status Current status Which components/services already exist in your architecture? The data and metadata files are available The ElasticSearch index is available Which components/services are under development (and by who)? The web interface for discovery and data access (Ifremer) The data processing services (Ifremer) Which components/services do you expect to get from EGI? Data storage and distribution Data processing EGI workshop for E-infrastructure, April 2016 Amsterdam
Plans for today Plans for today What questions would you like to get answered today? Can we replicate data, metadata and index on EGI infrastructure Do you have equivalent services based on these technologies What issues you would like to solve today? How do we push our data and metadata files Do you have a federation of identity Other outcomes that you would like to get out from the workshop? Meet groups interested in interdisciplinary studies EGI workshop for E-infrastructure, April 2016 Amsterdam
