Importance of Normal Distributions in Engineering
Normal distributions, also known as Gaussian distributions, are crucial in engineering due to their analytical tractability and modeling capabilities. They are central to statistical concepts like mean, covariance, entropy, and the Central Limit Theorem, making them essential for various engineering applications.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Normal Distributions Recall the definition of a normal distribution (Gaussian): = x x d 1 / 1 x 1 t ( ) exp[ ( ) ( )] p / 1 2 2 2 2 ( ) Why is this distribution so important in engineering? x ( = - x ) x - x ) p( x ( )d E[ Mean: Covariance: t t = - ) - ) x x x x x ( ( ] p( )d Statistical independence? Higher-order moments? Occam s Razor? Entropy? Linear combinations of normal random variables? Central Limit Theorem? ECE 8527: Lecture 02, Slide 1
Univariate Normal Distribution A normal or Gaussian density is a powerful model for modeling continuous- valued feature vectors corrupted by noise due to its analytical tractability. Univariate normal distribution: 2 x 1 1 = ( ) exp[ ] p x 2 2 where the mean and covariance are defined by: = [ ] ( ) E x xp x dx 2 2 2 = [( ) ( ) ( ) E x x p x dx The entropy of a univariate normal distribution is given by: 1 2 = = e ( ( )) ( ) ln ( ) log( 2 ) H p x p x p x dx 2 ECE 8527: Lecture 02, Slide 2
Mean and Variance A normal distribution is completely specified by its mean and variance: The peak is at: 1 ( = ) p 2 66% of the area is within one ; 95% is within two ; 99% is within three . A normal distribution achieves the maximum entropy of all distributions having a given mean and variance. Central Limit Theorem: The sum of a large number of small, independent random variables will lead to a Gaussian distribution. ECE 8527: Lecture 02, Slide 3
Multivariate Normal Distributions A multivariate distribution is defined as: 1 / 1 x 1 t = x x ( ) exp[ ( ) ( )] p / 1 2 2 d 2 2 ( ) where represents the mean (vector) and represents the covariance (matrix). Note the exponent term is really a dot product or weighted Euclidean distance. The covariance is always symmetric and positive semidefinite. How does the shape vary as a function of the covariance? ECE 8527: Lecture 02, Slide 4
Support Regions A support region is the obtained by the intersection of a Gaussian distribution with a plane. For a horizontal plane, this generates an ellipse whose points are of equal probability density. The shape of the support region is defined by the covariance matrix. ECE 8527: Lecture 02, Slide 5
Derivation ECE 8527: Lecture 02, Slide 6
Identity Covariance ECE 8527: Lecture 02, Slide 7
Unequal Variances ECE 8527: Lecture 02, Slide 8
Nonzero Off-Diagonal Elements ECE 8527: Lecture 02, Slide 9
Unconstrained or Full Covariance ECE 8527: Lecture 02, Slide 10
