Introduction to Machine Learning - Regression Readings and Topics
The content covers various topics on regression in machine learning, discussing readings by Barber, linear regression, Naive Bayes, logistic regression, MLE for Gaussian, Bayesian learning, and more. The slides provide insights into learning algorithms, parameter estimation, and implementations for different models.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
ECE 5424: Introduction to Machine Learning Topics: Regression Readings: Barber 17.1, 17.2 Stefan Lee Virginia Tech
Administrativia HW1 39 Submissions 58 Students Enrolled 19 MIA (?????) Project Proposal Due: 09/21, 11:55 pm <= 2pages, NIPS format (LESS THAN A WEEK!) HW2 Out today Due on Wednesday 09/28, 11:55pm Please please please please please start early Implement Linear Regression, Na ve Bayes, Logistic Regression (C) Dhruv Batra 2
Recap of last time (C) Dhruv Batra 3
Learning a Gaussian Collect a bunch of data Hopefully, i.i.d. samples e.g., exam scores Learn parameters Mean Variance (C) Dhruv Batra 4
MLE for Gaussian Prob. of i.i.d. samples D={x1, ,xN}: Log-likelihood of data: (C) Dhruv Batra Slide Credit: Carlos Guestrin 5
Your second learning algorithm: MLE for mean of a Gaussian What s MLE for mean? (C) Dhruv Batra Slide Credit: Carlos Guestrin 6
Learning Gaussian parameters MLE: (C) Dhruv Batra 7
Bayesian learning of Gaussian parameters Conjugate priors Mean: Gaussian prior Variance: Inverse Gamma or Wishart Distribution Prior for mean: (C) Dhruv Batra Slide Credit: Carlos Guestrin 8
MAP for mean of Gaussian (C) Dhruv Batra Slide Credit: Carlos Guestrin 9
New Topic: Regression (C) Dhruv Batra 10
1-NN for Regression Often bumpy (overfits) (C) Dhruv Batra Figure Credit: Andrew Moore 11
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 12
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 13
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 14
Linear Regression Demo http://hspm.sph.sc.edu/courses/J716/demos/LeastSquares/L eastSquaresDemo.html (C) Dhruv Batra 15
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 16
Plan for Today Regression Linear Regression Recap Some matrix calculus review Connections with Gaussians The outlier problem Robust Least Squares (C) Dhruv Batra 17
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 18
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 19
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 20
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 21
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 22
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 23
But, why? Why sum squared error??? Gaussians, Watson, Gaussians (C) Dhruv Batra 24
(C) Dhruv Batra Slide Credit: Greg Shakhnarovich 25
MLE Under Gaussian Model On board (C) Dhruv Batra 26
Is OLS Robust? Demo http://www.calpoly.edu/~srein/StatDemo/All.html Bad things happen when the data does not come from your model! How do we fix this? (C) Dhruv Batra 27
Robust Linear Regression y ~ Lap(w x, b) On board (C) Dhruv Batra 28
