Understanding Generalized Inverses and Resolution Matrices

Lecture 6 Resolution and Generalized Inverses

Syllabus Lecture 01 Lecture 02 Lecture 03 Lecture 04 Lecture 05 Lecture 06 Lecture 07 Lecture 08 Lecture 09 Lecture 10 Lecture 11 Lecture 12 Lecture 13 Lecture 14 Lecture 15 Lecture 16 Lecture 17 Lecture 18 Lecture 19 Lecture 20 Lecture 21 Lecture 22 Lecture 23 Lecture 24 Lecture 25 Lecture 26 Describing Inverse Problems Probability and Measurement Error, Part 1 Probability and Measurement Error, Part 2 The L2 Norm and Simple Least Squares A Priori Information and Weighted Least Squared Resolution and Generalized Inverses Backus-Gilbert Inverse and the Trade Off of Resolution and Variance The Principle of Maximum Likelihood Inexact Theories Prior Covariance and Gaussian Process Regression Non-uniqueness and Localized Averages Vector Spaces and Singular Value Decomposition Equality and Inequality Constraints L1 , L Norm Problems and Linear Programming Nonlinear Problems: Grid and Monte Carlo Searches Nonlinear Problems: Newton s Method Nonlinear Problems: MCMC and Bootstrap Confidence Intervals Factor Analysis Varimax Factors, Empirical Orthogonal Functions Backus-Gilbert Theory for Continuous Problems; Radon s Problem Linear Operators and Their Adjoints Fr chet Derivatives Estimating a Parameter in a Differential Equation Exemplary Inverse Problems, incl. Filter Design Exemplary Inverse Problems, incl. Earthquake Location Exemplary Inverse Problems, incl. Vibrational Problems

Purpose of the Lecture Introduce the idea of a Generalized Inverse, the Data and Model Resolution Matrices and the Unit Covariance Matrix Quantify the spread of resolution and the size of the covariance Use the maximization of resolution and/or covariance as the guiding principle for solving inverse problems

Part 1 The Generalized Inverse, the Data and Model Resolution Matrices and the Unit Covariance Matrix

all of the solutions of the form m mest = Md Md + v v

m mest = Md Md + v v let s focus on this matrix

m mest = G G- -g gd d + v v rename it the generalized inverse and use the symbol G G-g

(lets ignore the vector v v for a moment) Generalized Inverse G G-g operates on the data to give an estimate of the model parameters if d dpre = Gm then m mest = G G- -g gd dobs Gmest

Generalized Inverse G G-g if d dpre = Gm Gmest then m mest = G G- -g gd dobs sort of looks like a matrix inverse except M N, not square and GG GG- -g g I I and G G- -g gG G I I

so actually the generalized inverse is not a matrix inverse at all

plug one equation into the other Gmest and m mest = G G- -g gd dobs d dpre pre = Gm obs obs with N N= GG d dpre pre = Nd Ndobs GG- -g g data resolution matrix

Data Resolution Matrix, N N d dpre pre = Nd Ndobs obs How much does diobs contribute to its own prediction?

if N=I N=I d dpre pre = d dobs obs dipre = diobs diobs completely controls its own prediction

(A) d dpre d dobs = The closer N N is to I, the more diobs controls its own prediction

straight line problem 15 15 10 10 d d 5 5 0 0 0 5 z 10 0 5 z 10

d dpre pre = N d N dobs obs j = = i i j only the data at the ends control their own prediction

plug one equation into the other Gmtrue and m mest = G G- -g gd dobs d dobs obs = Gm obs true with R R= G G- -g gG G m mest est = Rm Rmtrue model resolution matrix

Model Resolution Matrix, R R m mest est = Rm Rmtrue true How much does mitrue contribute to its own estimated value?

if R=I R=I m mest est = m mtrue true miest = mitrue miestreflects mitrueonly

else if R R I I miest = + Ri,i-1mi-1true + Ri,imitrue + Ri,i+1mi+1true+ miest is a weighted average of all the elements of m mtrue

m mest m mtrue = The closer R R is to I, the more miest reflects only mitrue

Discrete version of Laplace Transform large c: d is shallow average of m(z) small c: d is deep average of m(z)

m(z) integrate e-c loz dlo z integrate e-c hiz dhi z z

m mest est = R R m mtrue true j = = i i j the shallowest model parameters are best resolved

Covariance associated with the Generalized Inverse unit covariance matrix divide by 2 to remove effect of the overall magnitude of the measurement error

unit covariance for straight line problem model parameters uncorrelated when this term zero happens when data are centered about the origin

Part 2 The spread of resolution and the size of the covariance

a resolution matrix has small spread if only its main diagonal has large elements it is close to the identity matrix

Dirichlet Spread Functions

a unit covariance matrix has small size if its diagonal elements are small error in the data corresponds to only small error in the model parameters (ignore correlations)

Part 3 minimization of spread of resolution and/or size of covariance as the guiding principle for creating a generalized inverse

over-determined case note that for simple least squares G G-g = [G GTG G]-1G GT model resolution R R=G G-gG G = [G GTG G]-1G GTG G=I I always the identify matrix

suggests that we try to minimize the spread of the data resolution matrix, N N find G G-g that minimizes spread(N)

spread of the k-th row of N N now compute

first term

second term third term is zero

putting it all together which is just simple least squares G G-g = [G GTG G]-1G GT

the simple least squares solution minimizes the spread of data resolution and has zero spread of the model resolution

under-determined case note that for minimum length solution G G-g = G GT [GG GGT]-1 data resolution N N=G GG G-g = G G G GT [GG always the identify matrix GGT]-1 =I I

suggests that we try to minimize the spread of the model resolution matrix, R R find G G-g that minimizes spread(R)

minimization leads to [GG GGT]G G-g = G GT which is just minimum length solution G G-g = G GT [GG GGT]-1

the minimum length solution minimizes the spread of model resolution and has zero spread of the data resolution

general case leads to

general case leads to a Sylvester Equation, so explicit solution in terms of matrices

special case #1 1 0 2 I I [G GTG G+ 2I I]G G-g=G GT G G-g=[G GTG G+ 2I I]-1G GT damped least squares

special case #2 0 1 2 I I G G-g[GG GGT+ 2I I] =G GT G G-g=G GT [GG GGT+ 2I I]-1 damped minimum length

so no new solutions have arisen just a reinterpretation of previously- derived solutions

reinterpretation instead of solving for estimates of the model parameters We are solving for estimates of weighted averages of the model parameters, where the weights are given by the model resolution matrix

criticism of Direchlet spread() functions when m m represents m(x) is that they don t capture the sense of being localized very well