Hydrologic Ensemble Forecast Service (HEFS) Overview

Hydrologic Ensemble Forecast Service (HEFS) Revisited Erick Boehmler Northeast River Forecast Center National Oceanic and Atmospheric Administration s National Weather Service National Weather Service National Oceanic and Atmospheric Administration s Northeast River Forecast Center Northeast River Forecast Center Taunton, MA Taunton, MA

HEFS Revisited HEFS Objective Meteorological Ensemble Forecast Processor (MEFP) >Capabilities >Methodology Parameter estimation Schaake Shuffle (Clark et al., 2004) Hydrologic Ensemble Processor Ensemble Postprocessor Ensemble Verification Service Validation results Short Long-range Products National Oceanic and Atmospheric Administration s National Weather Service Northeast River Forecast Center 2 Taunton, MA

HEFS Objective Improve NWS hydrologic services Feature ESP HEFS Forecast time horizon Weeks to seasons Hours to years, depending on the input forecasts Input forecasts ( forcing ) Historical climate data with some variations between RFCs Short-, medium- and long- range weather forecasts Uncertainty modeling Climate-based. No accounting for hydrologic uncertainty or bias. Suitable for long-range forecasting only Captures total uncertainty and corrects for biases in forcing and flow at all forecast lead times Products Limited number of graphical products (focused on long- range) and verification A wide array of data and user- tailored products are planned, including standard verification National Oceanic and Atmospheric Administration s National Weather Service Northeast River Forecast Center 3 Taunton, MA

HEFS Purpose Quantify forecast uncertainty for Short-range (hours to days) Flood watch and warning program. Local emergency management . Flood control system management. Reservoir management. Medium-range (days to weeks) Local emergency management preparedness. Reservoir management. Snowmelt runoff management. Long-range (weeks to months) Water supply planning. Reservoir management. National Oceanic and Atmospheric Administration s National Weather Service Office of Hydrologic Development Silver Spring, MD

HEFS Components National Oceanic and Atmospheric Administration s National Weather Service Northeast River Forecast Center 5 Taunton, MA

Meteorological Ensemble Forecast Processor Capabilities Forecast variables: precipitation and temperature. Forecast temporal horizon: up to about a year. Forecast spatial scale: basin. Forecast sources: WPC/RFC single-valued forecasts >QPF for days 1-5 and QTF 1-7 GEFS (days 1 -15) CFSv2 (days1- 270). Climatology (1 day 1 year). Ensemble quality: bias-corrected. Multiple temporal scales: 6 hours to 3 months to capture forecast skill at various temporal scales. Configurable Configurable Configurable Configurable National Oceanic and Atmospheric Administration s National Weather Service Office of Hydrologic Development Silver Spring, MD

MEFP Capabilities (Continued) Seasonality: accounted for by using moving window of user- specified size to pool data points in calibration. Temperature diurnal cycle: accounted for through equations relating 6-hr values and daily max and min values. Space-time coherence: preserved among upstream and downstream basins in forecast ensembles using Schaake Shuffle. Ensemble blending: Correlation-based. Ensemble forecasts are generated iteratively for all time scales and forecast sources from low correlation to high correlation. Operation modes: forecasting and hindcasting. Diagnostic tools: MEFPPE, GraphGen. Configurable National Oceanic and Atmospheric Administration s National Weather Service Office of Hydrologic Development Silver Spring, MD

MEFP Component Function WPC (2-day Planned) MEFP Correct forcing bias Merge in time Downscale (basin) Forecast Ensembles GEFS (Day 1 -15) MEFPPE CFSv2 Parameter Estimation (Days 16 270) Climatology (Days 271-365) NWS and external user applications National Oceanic and Atmospheric Administration s National Weather Service Northeast River Forecast Center 8 Taunton, MA

MEFP General Methodology Objective: Produce reliable ensemble forcing variables that capture the skill and quantify the uncertainty in the source forecasts. Key Idea: Condition the joint distribution of single-valued forecasts and the corresponding observations using the forecast. Select source forecasts from multiple models to cover short- to long-range. Define durations useful for MEFP application. Use a common modeling framework (the meta-Gaussian model) for both precipitation and temperature. National Oceanic and Atmospheric Administration s National Weather Service Office of Hydrologic Development Silver Spring, MD

MEFP General Methodology For a given forecast source, forecast start time, and duration, model the joint probability distribution between the single- valued forecast and the corresponding observation from historical records. Sample the conditional probability distribution of the joint distribution given the single-valued forecast. Rank short, medium, and long range ensembles according to the magnitude of the correlation coefficients between forecast and observation for the duration and associated forecast sources. Generate blended ensembles (with Schaake Shuffle applied) iteratively for all durations from low correlation to high correlation National Oceanic and Atmospheric Administration s National Weather Service Office of Hydrologic Development Silver Spring, MD

HEFS Components National Oceanic and Atmospheric Administration s National Weather Service Northeast River Forecast Center 11 Taunton, MA

Ensemble Verification Service Supports verification of HEFS including for precipitation, temperature and streamflow Verification of all forecasts or subsets based on prescribed conditions (e.g. seasons, thresholds, aggregations) Provides a wide range of verification metrics, including measures of bias and skill Requires a long archive of forecasts or hindcasts GUI or command-line operation National Oceanic and Atmospheric Administration s National Weather Service Northeast River Forecast Center 12 Taunton, MA

Forecast quality: validation results MEFP forcing Skill of the MEFP with GEFS forcing inputs Skill (fractional gain over climatology) Positive values mean fractional gain vs. climatology (e.g. 50% better on day 1 at FTSC1) 50% better than climatology MEFP temperature generally skillful, even after 14 days MEFP precipitation skillful during first week, but skill varies between basins Forecast lead time (days) National Oceanic and Atmospheric Administration s National Weather Service Northeast River Forecast Center 13 Taunton, MA

Forecast quality: validation results Long-range forecasts Walton, NY WALN6 (MARFC) MEFP precipitation forecast Example of MEFP precipitation forecasts from Walton, NY Skill (fractional gain over climatology) Beyond one week of GEFS, there is little skill vs. climatology No skill after ~one week In other words, the CFSv2 adds little skill for the long-range (but forcing skill may last >2 weeks in flow) GEFS CFSv2 CLIM If climate models improve in future, HEFS can be updated Forecast lead time (days) National Oceanic and Atmospheric Administration s National Weather Service Northeast River Forecast Center 14 Taunton, MA

HEFS Product Examples AHPS short-range probabilistic product National Oceanic and Atmospheric Administration s National Weather Service Office of Hydrologic Development Silver Spring, MD

HEFS Product Examples AHPS medium-range probabilistic products National Oceanic and Atmospheric Administration s National Weather Service Northeast River Forecast Center 16 Taunton, MA

An early application of long-range HEFS forecasts Managing NYC water supply Croton; Catskill; and Delaware Includes 19 reservoirs, 3 lakes; 2000 square miles Serves 9 million people (50% of NY State population) Delivers 1.1 billion gallons/day Operational Support Tool (OST) to optimize infrastructure, and avoid unnecessary ($10B+) water filtration costs HEFS forecasts are central to OST. The OST program has cost NYC under $10M National Oceanic and Atmospheric Administration s National Weather Service Office of Hydrologic Development Silver Spring, MD

Summary and conclusions Ensemble forecasts are the future Forecasts incomplete unless uncertainty captured Ensemble forecasts are becoming standard practice HEFS implementation, products, and validation is ongoing and expanding Initial validation results are promising HEFS will evolve and improve Science and software will improve through feedback Guidance will improve through experience We are looking forward to supporting end users! National Oceanic and Atmospheric Administration s National Weather Service Northeast River Forecast Center 18 Taunton, MA

References Clark, M., Gangopadhyay, S., Hay, L., Rajagopalan, B., Wilby, R., 2004. The Schaake Shuffle: a method for reconstructing space time variability in forecasted precipitation and temperature fields. Journal of Hydrometeorology 5 (1), 243 262. Demargne, J., Wu, L., Regonda, S.K., Brown, J.D., Lee, H., He, M., Seo, D.-J., Hartman, R., Herr, H.D., Fresch, M., Schaake, J. and Zhu, Y. (2014) The Science of NOAA's Operational Hydrologic Ensemble Forecast Service. Bulletin of the American Meteorological Society, 95, 79 98. Brown, J.D. (2014) Verification of temperature, precipitation and streamflow forecasts from the Hydrologic Ensemble Forecast Service (HEFS) of the U.S. National Weather Service: an evaluation of the medium-range forecasts with forcing inputs from NCEP's Global Ensemble Forecast System (GEFS) and a comparison to the frozen version of NCEP's Global Forecast System (GFS). Technical Report prepared by Hydrologic Solutions Limited for the U.S. National Weather Service, Office of Hydrologic Development, 139pp. Brown, J.D. (2013) Verification of long-range temperature, precipitation and streamflow forecasts from the Hydrologic Ensemble Forecast Service (HEFS) of the U.S. National Weather Service. Technical Report prepared by Hydrologic Solutions Limited for the U.S. National Weather Service, Office of Hydrologic Development, 128pp. National Oceanic and Atmospheric Administration s National Weather Service Northeast River Forecast Center 19 Taunton, MA

National Oceanic and Atmospheric Administrations National Weather Service Northeast River Forecast Center 20 Taunton, MA

Model the Forecast / Observed Joint Distribution PDF of STD Normal PDF of Observed Joint distribution Sample Space Y NQT Observed Joint distribution Model Space Y Correlation(X,Y) X 0 Forecast Observed PDF of Forecast PDF of STD Normal NQT X Forecast National Oceanic and Atmospheric Administration s National Weather Service Office of Hydrologic Development Silver Spring, MD

General Methodology For a given forecast source, forecast start time, and duration, model the joint probability distribution between the single- valued forecast and the corresponding observation from historical records. Sample the conditional probability distribution of the joint distribution given the single-valued forecast. Rank short, medium, and long range ensembles according to the magnitude of the correlation coefficients between forecast and observation for the selected duration and associated forecast sources. Generate blended ensembles (with Schaake Shuffle applied) iteratively for all durations from low correlation to high correlation National Oceanic and Atmospheric Administration s National Weather Service Office of Hydrologic Development Silver Spring, MD

Sample Conditional Joint Distribution Conditional distribution given xfcst Joint distribution Model Space Y 1 Observed x1 Probability Ensemble members xn xfcst X Forecast 0 x1 xi xn Ensemble forecast Obtain conditional distribution given a single-value forecast xfcst National Oceanic and Atmospheric Administration s National Weather Service Office of Hydrologic Development 3.23 Silver Spring, MD

General Methodology For a given forecast source, forecast start time, and duration, model the joint probability distribution between the single- valued forecast and the corresponding observation from historical records. Sample the conditional probability distribution of the joint distribution given the single-valued forecast. Rank short, medium, and long range ensembles according to the magnitude of the correlation coefficients between forecast and observation for the selected duration and associated forecast sources. Generate blended ensembles (with Schaake Shuffle applied) iteratively for all durations from low correlation to high correlation National Oceanic and Atmospheric Administration s National Weather Service Office of Hydrologic Development Silver Spring, MD

Blend Ensembles with Schaake Shuffle Ensemble members Blended ensemble members National Oceanic and Atmospheric Administration s National Weather Service Office of Hydrologic Development 3.25 Silver Spring, MD