Research to Operations Transition: Importance and Implementation
Process of transitioning scientific research to operational environments, maximizing ROI, improving NOAA systems, and achieving forecast skill enhancements. Learn about R2O, R2X, O2R, and their significance in advancing science and decision-making.
Uploaded on Mar 01, 2025 | 0 Views
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Research To Operations (R2O, R2X and O2R) Sid-Ahmed Boukabara NOAA/NESDIS/STAR & JCSDA The JCSDA Summer Colloquium, August 7th, 2015, Fort Collins, CO
Contents What Is R2O or R2X in General? 1 What is an O2R and why is it important? 2 3 What is needed for a successful R2O/O2R? How do we measure Success in R2O? 4 5 Specific Example of the JIBB and S4 Conclusions and Lessons Learned 6 2
What Is R2O or R2X in General? R2O is the process by which a scientific project led by a researcher, proven to provide added value, is transitioned to the Operational environment R2X refers to the transition of a research project outcome to operations, applications, and services (for example to decision makers). R2O is an example of R2X 3
Why is R2O Important for NOAA? (and other operational partners) R2O is an efficient way to leverage research done in NOAA and in the outside research community, to improve NOAA operational systems R2O allows NOAA to maximize the return on investment (ROI) of its outsourced research R2O allows NOAA to remain competitive, and at the edge of science in its products. 4
R2O ? R2O means often different things to different people NESDIS STAR R2O R2O OSPO R2O Researcher in Academia NWS EMC R2O NCO 5
JCSDA R2O Concept Goal: Accelerates the use of satellite data in NWP centers Scientific efforts in satellite DA in research community Diverse R&D activities Scientific efforts in satellite DA in academia, CIs Scientific efforts in satellite DA in NOAA/OAR Ensures resources are in place to achieve successful transition: supercomputer, a software integration team, etc. Scientific efforts in satellite DA in NOAA/NESDIS (funded by GOES-R PGs, JPSS PGs, etc) Ensures consistency and demonstrates benefits with operational systems like: Hybrid GDAS system (global), HWRF (Hurricane forecast). JCSDA s own DA Activities Products, techniques, improvements, with direct and immediate relevance to NWP Operational Models (both global and regional) Select projects for R2O Transition Made consistent with Operational Systems NWP Operational Centers Objective: Improvements in Forecast skills Ongoing Baseline Improvement in NWP centers
Contents What Is R2O or R2X in General? 1 What is an O2R and why is it important? 2 3 What is needed for a successful R2O/O2R? How do we measure Success in R2O? 4 5 Specific Example of the JIBB and S4 Conclusions and Lessons Learned 6 7
What is an O2R and why is it important? O2R infrastructure brings the Operational environment to researchers It allows scientists to perform their research in a system that is compatible with operational needs When a successful research project is performed in an O2R environment, it increases its chances to transition to Operations (R2O) Without an O2R environment, R2O is at best difficult, (often does not happen). 8
O2R Is the Bridge Between R and O Research Operations O2R Valley of Death 9
Contents What Is R2O or R2X in General? 1 What is an O2R and why is it important? 2 3 What is needed for a successful R2O/O2R? How do we measure Success in R2O? 4 5 Specific Example of the JIBB and S4 Conclusions and Lessons Learned 6 11
Components of an O2R/R2O Infrastructure(1) The infrastructure is not just the supercomputer: What Money can Buy . Supercomputer (Hardware, basic software, and IT) (almost the easiest part!) Scientific Software Integration (this is the hardest part: keep in synch with operational models) User Support & Documentation Management of resources to allow R2O and O2R to be sustained Documented Utilities to help scientists (CRTM, assessment, formatting tools) Constant engagement between Research and Operations Rigorous software configuration system to track changes Rigorous (and independent) testing mechanism to demonstrate added value 12
Components of an O2R/R2O Infrastructure (2) What Money can NOT Buy . And perhaps more importantly: the right culture. Transparency of R2O protocols and willingness to accept research performed by researchers & work collaboratively (on the operational partner) Willingness to work/start with operational system and follow protocols and accept constructive feedback (on the research partner) Sense of Mutual respect between Researchers and Operational partners: respect for innovation in research and pragmatism in the operational implementation Accept that R2O is not for everyone: careful choice of research and operations partners is important 13
Contents What Is R2O or R2X in General? 1 What is an O2R and why is it important? 2 3 What is needed for a successful R2O/O2R? How do we measure Success in R2O? 4 5 Specific Example of the JIBB and S4 Conclusions and Lessons Learned 6 14
Metrics for Assessing O2R Success When R2O transition has occurred: assess specific scientific metric(s) that was(ere) improved upon transition to operational environment. Example: Anomaly correlation, hurricane track improvement, etc. When R2O transition has not occurred yet, assess progress in the maturity of the project, as a result of the O2R infrastructure 15
Example of a project that led to an R2O transition (Improved QC) Southern hemisphere 500-hPa height ACC for 10 September 24 October 2012 for the control (black) and LNVD (red). Bottom: Variation between LNVD (red) and the control (operational QC of MODIS winds: black) based on forecast hour. Significant All ACC differences outside of the bars are significant at the 95% confidence level. The circle represents a statistically significant improvement for the Day-4 and Day-5 forecasts. Plot courtesy of D. Santek, UW (S4 project) 16
How do we assess R2O-maturity? (Case of JCSDA/NESDIS) JCSDA Modes of Operation NOAA Technical Readiness Level (TRL) Basic principles observed and reported TRL 1 TRL 2 Technology concept and/or application formulated TRL 3 Analytical and experimental critical function and/or characteristic proof-of-concept Directed Research FFO TRL 4 Component/subsystem validation in laboratory environment Visiting Scientist Program R2O Path TRL 5 System/subsystem/component validation in relevant environment TRL 6 System/subsystem model or prototyping demonstration in a relevant end-to-end environment System prototyping demonstration in an operational environment TRL 7 TRL 8 Actual system completed and mission qualified through test and demonstration in an operational environment TRL 9 Actual system mission proven through successful mission operations TRLs 1-2 are nominally considered Research, TRLs 3-5 are Development, TRLs 6-8 are Demonstration, and TRL 9 is Deployment, Implementation, or Operational Transition. 17
Contents What Is R2O or R2X in General? 1 What is an O2R and why is it important? 2 3 What is needed for a successful R2O/O2R? How do we measure Success in R2O? 4 5 Specific Example of the JIBB and S4 Conclusions and Lessons Learned 6 18
Supercomputer for Satellite Simulations and data assimilation Studies (S4) Overview: NESDIS, in collaboration with UW is making S4 available to scientists involved in satellite data assimilation and other activities related to it. One of the purposes is to consolidate data assimilation activities performed in the research community and to encourage the usage of NOAA operational models/tools/codes installed on it. Open policy for users (since hosted in academic institution). Only open source/non-restricted data allowed on S4. Status: Upgraded in 2014. The S4 system, hosted by University of Wisconsin. 19
S4 technical Description Brief Technical Description The S4 system is a Linux cluster (Dell/Intel hardware), 4700 CPU cores (60 Tflops) with 18TB of total RAM 1700TB in storage nodes Hosted in the UW/SSEC Data Center (with UPS) Able to run 4 GSI/GFS experiments simultaneously at T1534 System Administration and Resource Allocation SSEC provides system administration services and security protocols for the system, NESDIS/STAR provides overall guidance on system resource allocation and selection of users as well as scientific support (code management, software integration, etc) 20
S4 Intented Activities This system is intended to allow primarily (but not exclusively) the following major activities: (1) Undertake satellite data assimilation experiments at global and/or regional scales and the assessment of their impacts on forecast models skills, using currently flying satellite sensors and allowing scientists to test new science/methodology and (2) In support of the activity above, undertake all necessary satellite data simulations, calibration, algorithms development/improvement, radiative transfer modeling and validation, quality control (QC) procedures, etc (3) Perform Observing System Simulation Experiments (OSSEs) for new sensors (such as GOES-R and JPSS). This component is expected to address key questions that NESDIS constantly faces: (a) what is the impact and the added value of flying new sensors, (b) what is the optimal orbital configuration, (c) what are the best mitigation strategies possible, from an NWP perspective, given the budget and other constraints, (d) how to best coordinate with other national and international partners to achieve the most optimal Earth satellite observing system 21
S4 Governance & Management S4 is governed by a NESDIS/STAR policy memo identifying roles and responsibilities IT support is handled by the host institution UW. Software Integration is handled by NESDIS (the same team handles the software integration on the JCSDA sister machine JIBB) To request an account, a simple email to STAR POC will suffice. Accounts usually set up the same day. The S4 committee includes NESDIS/STAR director, deputy, and three senior program scientists (JCSDA, JPSS and GOES-R), makes recommendations concerning 90% of the machine's resources. The remaining 10% is managed by UW. 22
Policies and Code management on S4 Code management is the hardest part The software integration team is dedicated to making sure the NOAA codes installed on S4 are in synch with the operational models (including scripts, codes, datasets, libraries, etc) SVN is extensively used when appropriate. The following is a list of S4 Policies: User account request Managing resources Account expiration Tools and softwares guidance User guide Compiler & Software Availability o Intel Fortran / C++ o Gfortran /gcc/g++ o Netcdf 3 & 4 o Matlab o IDL o NCO 4 o GRADS o HDF 4 & 5 o NCL 23
How much did Projects benefit from S4? Project Title Project Leads Operational Partner Point of Contact at Oper. Partner Expected Date (best guess) for R2O Dec. 2015 R2O Current Stage (TRL) 8 TRL before using S4 1. A New Quality Control Method for Satellite-derived Polar Winds in the NCEP GDAS/GFS David Santek, and Brett Hoover NCEP Andrew Collard, John Derber 6 Daisuke Hotta,Tse- Chun Chen and Eugenia Kalnay 2. Proactive Quality Control (PQC) based on Ensemble Forecast Sensitivity to Observations (EFSO) NCEP Daryl Kleist, Mark Iredell January 2016 7 3 3. Development of Community Satellite Data Thinning and Representation Optimization Tool (CSTROT) 4. A near real time regional satellite data assimilation system for high impact weather research and forecasts Tong Zhu and Sid Boukabara NCEP Andrew Collard June 2016 6 2 Jun Li, Jinlong Li, and Pei Wang NHC and EMC NHC: Mark DeMaria, John L. Beven EMC: Vijay Tallapragada October 2014 to NHC June 2016 to EMC 3 1 was Steve Lord 5. Tropical cyclone data assimilation of MSG SEVIRI all- sky infrared satellite radiances Milija Zupanski and Man Zhang NCEP/EMC begin in 2016 6 2 6. Studying tropical cyclone genesis and rapid intensification with the NCEP GSI data assimilation system Zhaoxia Pu NCEP/EMC Vijay Tallapragada Dec. 2015 6 3 7. Assessment of Temperature and Humidity Prediction with Microwave and Hyperinfrared Satellite Data Assimilation over United States Jianjun Xu, and Alfred Powell,Jr STAR Mike Kalb & Sid Boukabara June 2016 3 1 24
Maturity Assessment of S4 projects Added value of S4 O2R Environment measured by the maturity-index (before and after using the S4) 25
Examples of S-NPP Projects Benefiting from O2R (Not directly funded by JCSDA, but supported through the O2R: HPC, user support, Porting of codes, etc) PI/Scientist Project Title Main Sensor used # -CrIS, ATMS 1 Jinlong Li Near real-time assimilation system development for improving tropical cyclone forecast with NPP/JPSS sounder data -ATMS 2 Banglin Zhang -Improve Hurricane Structure Monitoring and Intensity Forecast Using NPP ATMS and GCOM-W AMSR2 -CrIS 3 Jianjun Xu Application of hyper-spectral satellite data in severe weather forecasts with GSI/WRF regional data assimilation system -ATMS 4 Lin Lin -Evaluation of BG-Remapped ATMS on Hurricane Sandy Forecast & Test of Mie and DDA Scattering in the Simulation of ATMS -ATMS 5 Marouane Temimi Assessment of Assimilating NPP/JPSS ATMS Ground Surface Observations - ATMS, VIIRS 6 Li Fang, et al. Dual Assimilation of Microwave and thermal-infrared observations of soil moisture into NLDAS for improved drought monitoring -ATMS 7 Marouane Temimi Assessment of Assimilating NPP/JPSS ATMS Ground Surface Observations -S-NPP Sensors (ATMS) 8 Hu Yang Scientific Support for JPSS Instrument Calibration -All S-NPP assimilated sensors (ATMS) 9 Daisuke Hotta Proactive-QC based on Ensemble Forecast Sensitivity to Observations (EFSO) -VIIRS 10 Xinjia Zhou BTs validation of CRTM simulations against AVHRR, MODIS and VIIRS BTs in SST bands Johnathan W. Smith -IASI (potential application to CrIS) 11 WRF-Chem data assimilation of NOAA-Unique CO retrievals: Impact study on tropospheric ozone enhancement observed during the 2010 AEROSE Campaign -CrIS 12 Agnes Lim Huei Ni Impact analysis of LEO Hyperspectral Sensor IFOV size on the next generation NWP model forecast performance
Resources installed for Global and Regional DA and OSSE Activities The following models & tools are installed on S4: Global Data Assimilation System (GDAS) (including the new Hybrid Ensemble system) Grid Statistical Interpolation (GSI) Weather Research & Forecast (WRF) (WRF-ARW and WRF-NMM) Hurricane Weather Research & Forecast (HWRF) Microwave Integrated Retrieval System (MIRS) Community Radiative Transfer Model (CRTM) Verification Statistics Data Base (VSDB) Ocean Model HYCOM Etc The following models & tools will be installed in the future: NOAA Environmental Modeling System (NEMS) Unified Post Processor (UPP) Land Information System (LIS) Etc..
Major projects on jibb General data impact and OSSE experiments for Several Sensors Next generation NOAA hybrid data assimilation system port Hyperspectral IR methodology Hyperspectral IR cloudy radiances Satellite wind impact experiments Lightning assimilation methodology FFO projects of the JCSDA NASA Roses Projects of the JCSDA 28
JIBB and S4 Benchamarking Efforts have been devoted to compare satellite data experiments results on S4 and JIBB, to those performed on the NOAA R&D Benchmarking performed for currently operational GDAS (GSI+GFS) The results show that the operational version of GDAS is running well on S4 and JIBB (both T382 and T574) Anomaly correlation performed using GDAS system on NOAA R&D machine (Vapor), JCSDA JIBB machine and the S4 system. Benchmarking performed by J. Jung and the STAR/JCSDA SI team. Courtesy of J. Jung. Efforts included the benchmark of the hybrid data assimilation system. 29
Contents What Is R2O or R2X in General? 1 What is an O2R and why is it important? 2 3 What is needed for a successful R2O/O2R? How do we measure Success in R2O? 4 5 Specific Example of the JIBB and S4 Conclusions and Lessons Learned 6 30
Lessons learned for an effective R2O Importance of the O2R environment Importance of availability of an accessible HPC Importance of a transparent R2O protocol Importance of code portability & synchonization Importance of enforcing the exclusive usage (by researchers) of the operational systems Importance of a rigorous traceability of changes Importance of the S4 and JIBB approaches Importance of nurturing a culture of mutual respect between research and operations 31
Coordination & Synchronization: JCSDA Accelerated R2O coordination for SNPP/ATMS SNPP Was launched Nov 2011 Dec 2011, the JCSDA coordinated the signature of the R2O Action Plan for ATMS R2O between NESDIS and NWS The main points were: Who, does what and when and where (between NESDIS and NWS) Strong backing from STAR and NCEP Directors (present at kick off meeting) 7 months after launch, ATMS was assimilated operationally in NOAA Important of coordinate & synch 32
Conclusions JCSDA and NESDIS are involved in R2O by making significant resources available to scientists and collaborators (HPC, support, ..) The two major activities targeted are: (1) AdvancedSatellite data assimilation and (2) OSSE activities. The O2R infrastructure is more than the supercomputer: components include software management, integration and user support, etc The JIBB and S4 offer an O2R that is both operationally relevant as well as open and inclusive S4 and JIBB access offers an O2R environment conducive of innovation, but with a clear R2O objective. 33
Questions? 34
