Logistic Regression for Classification and Probability Estimation
Explore logistic regression, a linear model effective for classification and probability estimation. Learn about its structure, loss function, optimization methods, and gradient descent with fitting parameters. Discover how to implement the logistic regression algorithm with gradient descent steps to find optimal weights for effective predictions.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Logistic Regression Geoff Hulten
Overview of Logistic Regression A linear model for classification and probability estimation. 1 0.9 0.8 0.7 0.6 Can be very effective when: The problem is linearly separable Or there are a lot of relevant features (10s - 100s of thousands can work) You need something simple and efficient as a baseline X2 0.5 0.4 0.3 0.2 0.1 0 0 0.2 0.4 0.6 0.8 1 X1 Real implementations can learn from a lot of data. Logistic regression will generally not be the most accurate option.
Reminder: Components of Learning Algorithm Model Structure Loss Function Optimization Method
Structure of Logistic Regression ??? ??) ?^= ???????(?0+ ? Threshold Predict 0 1 ??????? ? = Predict 1 1+ ? ? Example: X1 X2 W0 W1 W2 1 1 .25 -1 1 1 0 0 1 0 0
Not Not the Loss Function the Loss Function for Logistic Regression ???? ?^,? = (?^ ?)2 1 0.9 0.8 0.7 0.6 X2 0.5 0.4 0.3 0.2 0.1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 X1
Loss function for Logistic Regression ???? ?^,? = log ?^ ???? ?^,? = log 1 ?^ ?? ? = 0 ?? ? = 1 ???? ?^,? = ? ??? ?^ 1 ? ???(1 ?^) y^ y .95 1 .1 0 .1 1 .95 0 ? ???? ??????? =1 ^,??) ? ????(?? ?
Fitting Parameters for Logistic Regression with Gradient Descent ??= ?? ?????(????????) ??? ?? 1 y^ y .25 1 .25 0 1 ? ?????(????????) ??? =1 ^ ?? ??? ? ?? ? Note: ?0= 1.0 for all samples
Logistic Regression Algorithm Do numIterations steps of gradient descent Find the gradient for each weight by averaging across the training set Update each weight by taking a step of size opposite the gradient Parameters size of the step to take in each iteration numIterations number of iterations of gradient descent to perform Threshold value between 0-1 to convert ?^into a classification
