Data Delivery
This content discusses the Ocean Observatories Initiative's data delivery process, addressing science, operator, education, and public use cases. Topics covered include user requests, data products generation, and command and control aspects. It emphasizes the importance of data management plans in serving user needs and identifies areas for improvement. The presentation highlights the significance of delivering quality data to users effectively, with a focus on user experiences and needs. It evaluates how the Ocean Observatories Initiative is judged based on initial and matured use cases, involving science users, marine operators, educators, and community testing. The content also provides insights into science use case summaries, user requests, and data product generation processes.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Ocean Observatories Initiative Data Delivery
Agenda- Data Delivery Topics covered: Science/Operator/Education/Public Use Cases , User requests and Data Product Generation , Operator Use Cases (mobile, uncabled and cabled), Command & Control Addresses: C?#2= How well will the data management plans and processes serve the OOI users as defined by the presented use cases ? Are there any gaps identified, critical weaknesses or recommendations for improvement? 1
Bottom Line of Data Delivery Get good data to people good = quality (covered below); Get data to people = this section Physical and functional/logical design just presented We followed the basic engineering V approach. Now that we have all the front-end items, it s the user and their needs that determine success or failure How Users Get the Data- OOI is, to the science user, a website How do we judge? 2
How Is OOI Judged? Initial; then Matured Use Cases Science user Marine Operator Educator Requirements Verification Beta Testing Validation Testing User Community 3
Science use case summary Search, Locate and download OOI data Search, Locate and download OOI documents, reports Add or retrieve ancillary data related to an OOI deployment/recovery cruise Subscribe to alerts based on user-defined criteria Examine a data product for consistency and quality External PI wishes to alter the configuration of an instrument or platform 4
User Request and Data Product Generation User Interface Database- Postgres Product Catalog MODIFI ED 2014-04- 08 22:11:24 Z 2014-04- 08 22:11:43 Z NAME Fluorometric CDOM Concentration L1 Upstream Inshore Fluorometric Chlorophyll-a Concentration L1 Product Generation ??cos??? + ??sin??? ? ? = ?0+ ? ? ?=1 5
Search, Locate and download OOI data Most fundamental task Pick area, find product TYP E LIFECYCLE STATE DEPLOYED NAME Fluorometric CDOM Concentration L1 FLORT Wire-Following Profiler Mooring - Coastal Pioneer Upstream Inshore MODIFIED 2014-04-08 22:11:24Z Fluorometric Chlorophyll-a Concentration L1 FLORT Wire-Following Profiler Mooring - Coastal Pioneer Upstream Inshore 2014-04-08 22:11:43Z DEPLOYED Optical Backscatter (Red Wavelengths) L1 FLORT Wire-Following Profiler Mooring - Coastal Pioneer Upstream Inshore 2014-04-08 22:12:04Z DEPLOYED Photosynthetically Active Radiation (400- 700 nm) L1 PARAD Wire-Following Profiler Mooring - Coastal Pioneer Upstream Inshore 2014-04-08 22:04:01Z DEPLOYED 6
Search, Locate and download OOI data Most fundamental task Pick time, examine/look at, download Download formats include NetCDF, CSV, Matlab, HTML, PNG (map plot), PDF Download URL accessible from external tools, e.g. Matlab using DAP protocol 7
Search, Locate and download OOI documents, reports Data is only part of the need Pick array- find document, read/download Coastal Glider, June 2011 Glider specifications Open Ocean Glider, October 2011 Glider specifications Global Hybrid Profiler Mooring, May 2012 Top-level drawings Global Mesoscale Flanking Mooring, May 2012 Top-level drawings Global Surface Mooring, July 2012 Top-level drawings Coastal Surface Mooring, July 2012 Top-level drawings Coastal Profiler Mooring, July 2012 Top-level drawings Endurance Array Benthic Experiment Package, May 2012 Top-level drawing Operations and Management Component, January 2013 AUV/AUV Dock System, February 2013 Uncabled Coastal Surface Piercing Profiler, Sept 2013 Cabled Coastal Surface Piercing Profiler, Mar 2014 8
Add or retrieve ancillary data related to an OOI deployment/recovery cruise All sites have pages For a summary of the July 2013 Station Papa Deployment, click here Click to download a PDF of all Infrastructure Tables Click the below links for individual Instrument Tables (A) Hybrid Profiler Mooring (B) Flanking Mooring A (C) Flanking Mooring B Mobile Assets Gliders Technical Drawings Click here for the online Technical Data Package OOI Station Papa Labels Click the below links for High-Level Technical Drawings (A) Hybrid Profiler Mooring (B, C) Flanking Mooring 9
Subscribe to alerts 12 predefined events can be subscribed 10
Examine a data product for consistency and quality Check the Metadata 11
External PI wishes to alter the configuration of an instrument or platform This is largely a management level use case with a triad of actors- NSF, OOI and an external PI It then becomes part of routine operations Steps: 1. The PI and the O&M Lead for the Endurance Array conduct planning meetings to confirm the configuration(s) required to address the approved science objectives of the PI s work, and the planned frequency of configuration changes to OOI instruments and gliders within the scope of the approved work. 2. The Endurance O&M Lead provides the PI with the information and possibly structure required for submitting configuration changes (sampling rates, glider mission plans). 3. PI edits the instrument configuration and glider mission plan templates and submits the changes and their likely frequency within the approved 2-year study interval, to the Endurance O&M Lead. 4. The Endurance O&M Lead reviews those requests with the PI to assure that OOI safety and performance guidelines will not be compromised. 5. Observatory Director submits approved change(s) to UNOLS for scheduling of implementation. 6. UNOLS evaluates, with iteration with Observatory Director and external PI, when approved configuration should be implemented (other approved projects may be in the queue). Work is scheduled for implementation within the next calendar year. 7. Observatory Director includes this approved work in the next Annual Work Plan and schedule for the next calendar year. 8. External PI begins interactions with Endurance O&M Lead to implement his/her specific instrument and glider configurations upon the approved change dates (as dictated by the work approved by NSF and scheduled via UNOLS). 9. The Endurance O&M Lead sustains the PI configuration(s) per the approved award. 12
Marine Operator use case summary PERFORM COMMAND AND CONTROL OF DEPLOYED ASSETS MONITOR SYSTEM HEALTH AND STATUS AND ACT PER PROCEDURES SUBMIT DATA TO NATIONAL ARCHIVE(S) PERFORM HW AND/OR SW UPDATES OPERATOR SYNCHRONIZES OBSERVATORY CONFIGURATION ACROSS DATA SYSTEM AND DOCUMENT SYSTEM MANAGE OOI HOMEPAGE (SYSTEM INFORMATION PORTAL) AND TROUBLE REPORTS ESTABLISH / ALTER / MAINTAIN THE CONFIGURATION OF OOI PROVIDE EXTERNAL PI SUPPORT 13
Educational Community use case summary 14
Output Data Product Variables Each L1 and L2 product has the following variables (i.e., columns in the time series): Time <measurement>_L1a (e.g., Conductivity_L1a) <measurement>_L1b_Post_Deployment_Cal <measurement>_L1b_Post_Recovery_Cal <measurement>_L1c QC_Flag_GlobalRange QC_Flag_LocalRange <additional QC flags> Single Parsed (Combined) product per instrument, with all variables for applicable L1 and L2 products, additional time stamps TIME L1a L1b-pd L1b-pr L1c QC-gr QC-lr
Output Data Product Metadata In the metadata (i.e., Metadata link from ERDDAP page, AND metadata on Data Product facepage on OOINet UI): Calibration coefficients (as a comma separated list) QC Look Up Table (as a url) Data Product Algorithm code (as a url) DPS for Data Product Algorithm (as a url) QC Algorithms (as urls) DPS s for QC Algorithms (as urls)
Data Delivery Summary DM is not a noun, verb, thing nor separate effort But rather a perspective that includes all the other groups and actions Fundamentally it is the timely implementation of the necessary requirements to deliver data to users Each piece, say UI, metadata or algorithms or archive policies, is just one piece of the chain that has to be defined, built, integrated and supported 17
