Environmental Data Analysis with MATLAB or Python - Lecture Insights
Explore the diverse topics covered in the Environmental Data Analysis with MATLAB or Python 3rd Edition lecture series, including Factor Analysis, Cluster Analysis, Fourier Series, Hypothesis Testing, and more. Gain valuable insights into directed and undirected data analysis approaches, as well as practical applications such as plotting data points on ternary diagrams.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Environmental Data Analysis with MATLAB or Python 3rdEdition Lecture 15
SYLLABUS Lecture 01 Lecture 02 Lecture 03 Lecture 04 Lecture 05 Lecture 06 Lecture 07 Lecture 08 Lecture 09 Lecture 10 Lecture 11 Lecture 12 Lecture 13 Lecture 14 Lecture 15 Lecture 16 Lecture 17 Lecture 18 Lecture 19 Lecture 20 Lecture 21 Lecture 22 Lecture 23 Lecture 24 Lecture 25 Lecture 26 Intro; Using MTLAB or Python Looking At Data Probability and Measurement Error Multivariate Distributions Linear Models The Principle of Least Squares Prior Information Solving Generalized Least Squares Problems Fourier Series Complex Fourier Series Lessons Learned from the Fourier Transform Power Spectra Filter Theory Applications of Filters Factor Analysis and Cluster Analysis Empirical Orthogonal functions and Clusters Covariance and Autocorrelation Cross-correlation Smoothing, Correlation and Spectra Coherence; Tapering and Spectral Analysis Interpolation and Gaussian Process Regression Linear Approximations and Non Linear Least Squares Adaptable Approximations with Neural Networks Hypothesis testing Hypothesis Testing continued; F-Tests Confidence Limits of Spectra, Bootstraps
Goals of the lecture introduce Factor Analysis and Cluster Analysis as methods of detecting patterns in data
Goals of the lecturec introduce Factor Analysis and Cluster Analysis as methods of detecting patterns in data
Directed data analysis The practitioner proposed a theory, as embodied in the ? in ?? = ?, and then focuses on estimating ?.
Undirected data analysis The practitioner does not have a specific theory in mind, but focuses on discovering patterns of variation within the data ?.
C data points plotted on a ternary diagram B A
C data points plotted on a ternary diagram B A
C C B A A B data that are close to one or another cluster centers data that are a mix of two end-members or factors
C C B A A B data that are close to one or another cluster centers data that are a mix of two end-members or factors
C C B A A B Factor Analysis Cluster Analysis
Part 1 Factor Analysis
Example sediment samples are a mix of several sources source A source B ocean sediment s1 s2 s3 s4 s5
what does the composition of the samples tell you about the composition of the sources? s1 s2 e1 e2 e3 e4 e5 e1 e2 e3 e4 e5 ocean sediment
another example rocks are composed of minerals
Rocks are a mix of minerals, and rock 3 rock 1 rock 2 rock 4 rock 6 rock 7 rock 5 mineral 1 mineral 2 mineral 3 minerals have a well-defined composition
Which simpler? rocks have a chemical composition or rocks contain minerals and minerals have chemical compositions
answer will depend on how many minerals are involved and how many elements are in each mineral
the sample matrix, S S N samplesby M elements e.g. sediment samples rock samples word element is used in the abstract sense and may not refer to actual chemical elements
the factor matrix, F F P factors by M elements e.g. sediment sources minerals note that there are P factors a simplification if P<M
the loading matrix, C C N samplesby P factors specifies the mix of factors for each sample
summary samples contain factors factors contain elements
an important issue how many factors are needed to represent the samples? need at most P=M but is P < M ?
element samples element B element
element line of samples implies only 2 factors, so P=2 samples element B element
element factors samples ?1 ?2 element B element
data do not uniquely determine factors A) B) factor, f1 factor, f 2 factor, f 1 factor, f2 two bracketing factors most typical factor and deviation from it
mathematically S S = CF CF = C C F F with F F = M F M F and C C = C M C M-1 where M M is any P P matrix with an inverse must rely on prior information to choose M M
a method to determine the minimum number of factors, P and one possible set of factors
a digression, but an important one suppose that we have an N N square matrix, M M and we experiment with it by multiplying input vectors, v v, by it to create output vectors, w w w w = Mv Mv
surprisingly, the answer to the question when is the output parallel to the input ? tells us everything about the matrix
if w w is parallel to v v then w w = v v where is a proportionality factor the equation w w = Mv is then Mv or (M Mv v v = Mv (M - - I)v=0 I)v=0
but if (M (M - - I)v=0 I)v=0 then it would seem that v = (M v = (M - - I) I)-10 = 0 0 = 0 which is not a very interesting solution w w is parallel to v when v is zero
to make an interesting solution you must choose so that (M (M - - I) I)-1 doesn t exist which is equivalent to choosing so that I)=0 det det(M (M - - I)=0
to make an interesting solution you must choose so that (M (M - - I) I)-1 doesn t exist which is equivalent to choosing so that since a matrix with zero determinant has no inverse I)=0 det det(M (M - - I)=0
in the 22case this is a quadratic equation in and so has two solutions 1 and 2
in the NN case I)=0 det det(M (M - - I)=0 is an N-order polynomial equation and so has N solutions 1, 2, N each corresponds to a different v v v v(1),v v(2), v v(N)
in the NN case I)=0 det det(M (M - - I)=0 is an N-order polynomial equation and so has N solutions 1, 2, N eigenvalues each corresponds to a different v v v v(1),v v(2), v v(N) eigenvectors
NN matrix, M w w = Mv Mv when is the output parallel to the input ? N different cases Mv Mv(1) = Mv Mv(2) = = 1v v(1) = 2v v(2) = Nv v(N) Mv Mv(N) =
Mv Mv(1) = Mv Mv(2) = = 1v v(1) = 2v v(2) = Nv v(N) Mv Mv(N) = simplify notation MV = MV = V V
In the text its shown that if M M is symmetric then all s are real v v s are orthonormal 1 if i=j v v(i)T v v(j) = 0 if i j
In the text its shown that if M M is symmetric then all s are real v v s are orthonormal 1 if i=j v v(i)T v v(j) = 0 if i j implies V VTV V = VV VVT= = I I
MV = MV = V V post-multiply by V VT M = M = V V V VT M M can be constructed from V V and so when is the output parallel to the input ? tells you everything about M M
suppose S S is square and symmetric then S S= CF CF = V V V VT
suppose S S is square and symmetric then S S= CF CF = V V V VT C C F F
suppose S S is square and symmetric then S S= CF CF = V V V VT C C F F S S can be represented by M mutually-perpendicular factors, F
furthermore, suppose that only P eigvenvalues are nonzero the eigenvectors with zero eigenvalues can be thrown out of the equation
