Environmental Data Analysis: Linear Approximations and Nonlinear Least Squares

Environmental Data Analysis with MatLab 2ndEdition Lecture 22: Linear Approximations and Non Linear Least Squares

SYLLABUS Lecture 01 Lecture 02 Lecture 03 Lecture 04 Lecture 05 Lecture 06 Lecture 07 Lecture 08 Lecture 09 Lecture 10 Lecture 11 Lecture 12 Lecture 13 Lecture 14 Lecture 15 Lecture 16 Lecture 17 Lecture 18 Lecture 19 Lecture 20 Lecture 21 Lecture 22 Lecture 23 Lecture 22 Lecture 23 Lecture 24 Using MatLab Looking At Data Probability and Measurement Error Multivariate Distributions Linear Models The Principle of Least Squares Prior Information Solving Generalized Least Squares Problems Fourier Series Complex Fourier Series Lessons Learned from the Fourier Transform Power Spectra Filter Theory Applications of Filters Factor Analysis Orthogonal functions Covariance and Autocorrelation Cross-correlation Smoothing, Correlation and Spectra Coherence; Tapering and Spectral Analysis Interpolation Linear Approximations and Non Linear Least Squares Adaptable Approximations with Neural Networks Hypothesis testing Hypothesis Testing continued; F-Tests Confidence Limits of Spectra, Bootstraps

Goals of the lecture learn how to make linear approximations of non-linear functions apply liner approximations to error estimation apply liner approximations to least squares

Taylor Series and Linear Approximations

polynomial approximation to a function y(t) in the neighborhood of a point t0

polynomial approximation to a function y(t) in the neighborhood of a point t0 find coefficients by taking derivatives

polynomial approximation to a function y(t) in the neighborhood of a point t0 evaluate at t0 0 find coefficients by taking deriatives 0 0

polynomial approximation to a function y(t) in the neighborhood of a point t0

polynomial approximation to a function y(t) in the neighborhood of a point t0 Taylor series

Taylor Series Linear approximation

example

example

example

example Linear approximation

example: distances on a sphere ( 1,L1) ( 2,L2) r measured in terms of central angle, r

exact formula: 6 trig functions approximate formula: 1 trig function and 1 square root

(2,L2=0) ( 1=0,L1=0)

application to estimates of variance

spectral analysis scenario measure angular frequency, m want confidence bounds on corresponding period, T

exact (but difficult) method assume m is Normally-distributed, p(m) work out the distribution p(T) compute its mean and variance by integration

approximate (and easy) method assume m is Normally-distributed with mean mest work out a linear approximation of T in neighborhood of mest use formula for error propagation for a linear functions

consider small fluctuations about the estimated angular frequency Test so

application to least squares

Goal Solve non-linear problems of the form by generalized least squares

Taylor series of predicted data

Taylor expansion of predicted data with and

Taylor expansion of predicted data linearized equation with and

Taylor expansion of total error

Taylor expansion of total error

Taylor expansion of total error gradient vector curvature matrix

linearized least squares

linearized least squares minimize error

linearized least squares minimize error

linearized least squares minimize error linear theory

linearized least squares minimize error linear theory

linearized least squares minimize error linear theory

linearized least squares minimize error linear theory

linearized least squares guess for the solution

linearized least squares trial solution deviation of data from prediction of trial solution

linearized least squares trial solution deviation of data from prediction of trial solution linearized data kernel

linearized least squares trial solution deviation of data from prediction of trial solution linearized data kernel correction to solution

linearized least squares trial solution deviation of data from prediction of trial solution linearized data kernel correction to solution updated solution

linearized least squares repeat

prior information written in terms of the unknown =

modification of generalized least squares

example of generalized least squares

example of generalized least squares sinusoid of unknown amplitude & frequency superimposed on a constant background level

example of generalized least squares sinusoid of unknown amplitude & frequency superimposed on a constant background level normalized unknowns, so mi 1 level background frequency amplitude