Optimized Inverse Method for SST Retrieval

A Deterministic Inverse Method for SST Retrieval from VIIRS: Incorporating Aerosol in the Retrieval Vector Research: Prabhat Koner PI: Andy Harris CICS, ESSIC, University of Maryland 1 NASA MODIS-VIIRS ST Meeting, June 6 10, 2016

Physical Retrieval - Recap Reduces the problem to a local linearization Dependent on ancillary data (NWP) for an initial guess More compute-intensive than regression not an issue nowadays Especially with fast RTM (e.g. CRTM) Widely used for satellite sounding More channels, generally fewer (larger) footprints Initially, started with a simple reduced state vector x = [SST, TCWV]T N.B. Implicitly assumes NWP profile shape is more or less correct Selection of an appropriate inverse method Ensure that satellite measurements are contributing to signal Avoid excessive error propagation from measurement space to parameter space If problem is ill-conditioned 2 NASA MODIS-VIIRS ST Meeting, June 6 10, 2016

History of Inverse Model Y=KX Forward model: Simple Inverse: X=K-1Y (measurement error) Legendre (1805) Least Squares: X=Xig+(KTK)-1KT(Yd-Yig) X=Xig+(KTK+lR)-1KT(Yd-Yig) MTLS: OEM: X=Xa+(KT Se -1K+Sa -1)-1KTSe -1 (Yd-Ya) NASA MODIS-VIIRS ST Meeting, June 6 10, 2016

Uncertainty Estimation Physical retrieval Normal LSQ Eqn: MTLS modifies gain: Regularization strength: ( 2end = lowest singular value of [K y]) x = (KTK)-1KT y [= G y] G = (KTK + I)-1KT = (2 log( )/|| y||) 2end Total Error ||e|| = ||(MRM I) x|| + ||G || ||( y - K x)|| N.B. Includes TCWV as well as SST NASA MODIS-VIIRS ST Meeting, June 6 10, 2016

DFS/DFR and Retrieval error May2011 1 MTLS OEM OEM2 LS IG DFR DFS DFS2 0.9 0.8 RMSE & DFR/DFS 0.7 0.6 0.5 0.4 3 4 5 Percentage of total mathches (using DFR) 6 7 8 9 10 11 12 13 Retrieval error of OEM higher than LS More than 75% OEM retrievals are degraded w.r.t. a priori error DFR of MTLS is high when a priori error is high NASA MODIS-VIIRS ST Meeting, June 6 10, 2016

Optimized OE is an overestimate [Se], Sa = 0 or an underestimate 0 Perform experiment insert true SST error into Sa-1 Can only be done when truth is known, e.g. with matchup data 6 NASA MODIS-VIIRS ST Meeting, June 6 10, 2016

DFS/DFR and Retrieval error May2011 1 MTLS OEM OEM2 LS IG DFR DFS DFS2 0.9 0.8 RMSE & DFR/DFS 0.7 0.6 0.5 0.4 3 4 5 Percentage of total mathches (using DFR) The retrieval error of OEM is good when a priori SST is perfectly known, but DFS of OEM is much lower than for MTLS 6 7 8 9 10 11 12 13 Retrieval error of OEM higher than LS More than 75% OEM retrievals are degraded w.r.t. a priori error DFR of MTLS is high when a priori error is high NASA MODIS-VIIRS ST Meeting, June 6 10, 2016

Improved cloud detection Use a combination of spectral differences and RT Envelope of physically reasonable clear-sky conditions Spatial coherence (3 3) Also check consistency of single-channel retrievals Flag excessive TCWV adjustment & large MTLS error Increased coverage w.r.t. GHRSST QL3+, but with reduced cloud leakage Prabhat stalk in yesterday s Oceans Breakout ~50% increase in coverage & ~50% reduction in error 8 NASA MODIS-VIIRS ST Meeting, June 6 10, 2016

VIIRS Initial Results July 2014 (Night) June 2014 (Night) 1.8 1.6 1.7 1.5 1.6 MTLS(NC) OEM(NC+CH) IG(NC) OSPO(NC) OSISAF(NC) SST4(NC) NAVO(NvC) OSPO(OsC) 1.4 1.5 MTLS(NC) OEM(NC+CH) IG(NC) OSPO(NC) OSISAF(NC) SST4(NC) NAVO(NvC) OSPO(OsC) 1.3 1.4 1.2 1.3 1.1 1.2 SD & RMSE SD & RMSE 1 1.1 0.9 1 0.9 0.8 0.8 0.7 0.7 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 4 6 8 10 12 14 16 18 20 22 4 6 8 10 12 14 16 18 20 22 24 % of total matches (under Quality analysis) % of total matches (under Quality analysis) Data are ordered according to MTLS error Reliable guide for regression as well as MTLS Trend of initial guess error is expected 9 NASA MODIS-VIIRS ST Meeting, June 6 10, 2016

MODIS experiments Channel selection Test various combinations and look at accuracy of retrieval (a) (b) 0.5 3 4 11 12 1 3 4 11 12 3 4 5 11 12 3 4 6 11 12 3 4 8 11 12 3 4 9 11 12 3 4 10 11 12 3 4 11 12 13 3 4 11 12 15 3 4 11 12 3 4 11 13 1 3 4 11 13 1 3 4 6 11 13 1 3 4 6 11 13 15 1 3 4 6 11 12 13 15 1 3 4 6 11 12 1 3 4 6 12 13 15 0.55 0.45 0.5 0.45 0.4 0.4 SD & RMSE (K) SD & RMSE (K) 0.35 0.35 0.3 0.3 0.25 0.25 0.2 0.2 0 5 10 15 20 25 0 5 10 15 20 25 % of total matches (using TTLS QI) % of total matches (using TTLS QI) RTM may be inadequate for some channels Channels 1, 3, & 13 are particularly useful bias 10 NASA MODIS-VIIRS ST Meeting, June 6 10, 2016

Addition of aerosol Put aerosol information in the CRTM NGAC profiles, multiple species (dust, salt, sulfate, soot) Improve match of RTM to observation Does this improve retrieval? Put aerosol in the retrieval vector Allow Total Column Aerosol to vary x = [SST, WV, TCA]T Jacobian now includes T/ TCA for each channel Does this improve retrieval? MTLS developed for 2-parameter retrieval Try different regularization operator since problem is now more ill- conditioned: Truncated Total Least Squares (TTLS) | y| 1: = ( end-1)2 | y| > 1: = ( end-1/log(| y|))2 11 NASA MODIS-VIIRS ST Meeting, June 6 10, 2016

Inclusion of aerosol Accuracy with TTLS & joint [SST, WV, TCA] ~0.2 K Algorithm sensitivity is also improved cf. MTLS 12 NASA MODIS-VIIRS ST Meeting, June 6 10, 2016

Summary Addition of aerosol has significant benefit Most of all when included in retrieval vector as well as CRTM Better partitioning of brightness temperature residuals No longer forcing delta-BTs caused by aerosol into the SST and/or WV retrieval space Also improves algorithm sensitivity to SST (better overall fit to model) TTLS better choice for 3-parameter retrieval Initial tuning with MODIS works well Adaptation to VIIRS channels underway Validation results are approaching buoy accuracy limit Best ~50% of retrievals at 0.2 K Implies actual retrieval accuracy is better than this Need to consider what might be needed @SIPS Full aerosol profiles as well as NWP 13 NASA MODIS-VIIRS ST Meeting, June 6 10, 2016

Backup slides 14 NASA MODIS-VIIRS ST Meeting, June 6 10, 2016

Improvements It seems obvious that a sensitivity of 1 is desirable E.g. if there is diurnal warming of 5 K, it will be observed in the data, and strong upwellings will be accurately observed, etc. However, there is a penalty to be paid Ill-conditioned problem noise propagates from measurement space to parameter space Compromise is usually struck (e.g. minimum least squares result for training data in a regression algorithm) Regression algorithms may have sensitivity <1 for large regions E.g. daytime algorithms in the tropics (diurnal warming!) Causes bias if local atmospheric conditions are different from the ensemble mean for the training data 15 NASA MODIS-VIIRS ST Meeting, June 6 10, 2016

MODIS Initial Results Fig. 1a Fig. 1b 1.45 1.45 1.4 1.4 1.35 1.35 MTLS(NC) OEM(NC) SST4(NC) MTLS(MC) OEM(MC) SST4(MC) IG(NC) IG(MC) SST4(MQI) MTLS(NC) OEM(NC) cSST4(NC) MTLS(MC) OEM(MC) cSST4(MC) IG(NC) IG(MC) SST4(MQI) 1.3 1.3 1.25 1.25 1.2 1.2 1.15 1.15 1.1 1.1 1.05 1.05 1 1 0.95 0.95 0.9 0.9 0.85 0.85 0.8 0.8 0.75 0.75 0.7 0.7 SD & RMSE SD & RMSE 0.65 0.65 0.6 0.6 0.55 0.55 0.5 0.5 0.45 0.45 0.4 0.4 0.35 0.35 0.3 0.3 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 % of total matches (under Quality analysis) % of total matches (under Quality analysis) Note improvement from discarding MTLS error last bin Irrespective, MTLS is quite tolerant of cloud scheme Recalculated SST4 coefficients produce quite good results 16 NASA MODIS-VIIRS ST Meeting, June 6 10, 2016