Introduction to SVM and Logistic Regression in Machine Learning

ece 5984 introduction to machine learning n.w
1 / 17
Embed
Share

Explore the fundamental concepts of Support Vector Machines (SVM) and Logistic Regression in machine learning through topics such as soft and hard margin comparisons, generative versus discriminative approaches, linear classifiers, margin optimization, and more. Dive into the differences between SVMs and logistic regression for effective classification tasks and understand how to handle non-linearly separable data.

  • Machine Learning
  • SVM
  • Logistic Regression
  • Classification
  • Support Vector Machines
rayaanacharya
jveron Follow

Uploaded on | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.

Download Presentation

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.

E N D

Presentation Transcript

  1. ECE 5984: Introduction to Machine Learning Topics: SVM soft & hard margin comparison to Logistic Regresion Readings: Barber 17.5 Dhruv Batra Virginia Tech

  2. New Topic (C) Dhruv Batra 2

  3. Generative vs. Discriminative Generative Approach (Na ve Bayes) Estimate p(x|y) and p(y) Use Bayes Rule to predict y Discriminative Approach Estimate p(y|x) directly (Logistic Regression) Learn discriminant function f(x) (Support Vector Machine) (C) Dhruv Batra 3

  4. Linear classifiers Which line is better? w.x = j w(j) x(j) 4

  5. Margin x+ x- (C) Dhruv Batra Slide Credit: Carlos Guestrin 5

  6. Support vector machines (SVMs) Solve efficiently by quadratic programming (QP) Well-studied solution algorithms Hyperplane defined by support vectors (C) Dhruv Batra Slide Credit: Carlos Guestrin 6

  7. What if the data is not linearly separable? 7 Slide Credit: Carlos Guestrin

  8. What if the data is not linearly separable? Minimize w.w and number of training mistakes 0/1 loss Slack penalty C Not QP anymore Also doesn t distinguish near misses and really bad mistakes (C) Dhruv Batra Slide Credit: Carlos Guestrin 8

  9. Slack variables Hinge loss If margin >=1, don t care If margin < 1, pay linear penalty (C) Dhruv Batra Slide Credit: Carlos Guestrin 9

  10. Soft Margin SVM Effect of C Matlab demo by Andrea Vedaldi (C) Dhruv Batra 10

  11. Side note: Whats the difference between SVMs and logistic regression? SVM: Logistic regression: Log loss: SVM: Hinge Loss LR: Logistic Loss (C) Dhruv Batra Slide Credit: Carlos Guestrin 11

  12. (C) Dhruv Batra Slide Credit: Andrew Moore 12

  13. (C) Dhruv Batra Slide Credit: Andrew Moore 13

  14. (C) Dhruv Batra Slide Credit: Andrew Moore 14

  15. Does this always work? In a way, yes (C) Dhruv Batra Slide Credit: Blaschko & Lampert 15

  16. Caveat (C) Dhruv Batra Slide Credit: Blaschko & Lampert 16

  17. Kernel Trick One of the most interesting and exciting advancement in the last 2 decades of machine learning The kernel trick High dimensional feature spaces at no extra cost! But first, a detour Constrained optimization! (C) Dhruv Batra Slide Credit: Carlos Guestrin 17

Related


Rescue Drone: Increasing Autonomy and Implementing Computer Vision

Rescue Drone: Increasing Autonomy and Implementing Computer Vision

conan
Sports Volunteers Movement in Botswana: Challenges and Strategies

Sports Volunteers Movement in Botswana: Challenges and Strategies

perseus
Counterfeit Detection Techniques in Currency to Combat Financial Fraud

Counterfeit Detection Techniques in Currency to Combat Financial Fraud

tova
Machine Learning Algorithms and Models Overview

Machine Learning Algorithms and Models Overview

arianell
Kernel Tricks in Machine Learning

Kernel Tricks in Machine Learning

bria
Efficient Anomaly Detection for Batch Systems Using Machine Learning

Efficient Anomaly Detection for Batch Systems Using Machine Learning

ellianna
NLDB 2020 Pattern Learning for Detecting Defect Reports and Improvement Requests

NLDB 2020 Pattern Learning for Detecting Defect Reports and Improvement Requests

zkeev
Functional Approximation Using Gaussian Basis Functions for Dimensionality Reduction

Functional Approximation Using Gaussian Basis Functions for Dimensionality Reduction

abdalrah
Evolution of Sentiment Analysis in Tweets and Aspect-Based Sentiment Analysis

Evolution of Sentiment Analysis in Tweets and Aspect-Based Sentiment Analysis

smalc
Gender Identification in SMS Texts: An Exploration of Authorship Characteristics

Gender Identification in SMS Texts: An Exploration of Authorship Characteristics

tkal
Stream Management and Online Learning in Data Mining

Stream Management and Online Learning in Data Mining

bentlie
Implicit Citations for Sentiment Detection: Methods and Results

Implicit Citations for Sentiment Detection: Methods and Results

dis_van
Overview of Linear Classifiers and Perceptron in Classification Models

Overview of Linear Classifiers and Perceptron in Classification Models

shin577
Object Detection Techniques Overview

Object Detection Techniques Overview

zmode
Exploring Iris Dataset: Attribute Analysis and Prediction

Exploring Iris Dataset: Attribute Analysis and Prediction

sartori_e
Aspect-Based Sentiment Analysis Study

Aspect-Based Sentiment Analysis Study

nobriga_r
Predicting Treatment Outcome in Cervical Cancer Using Texture Analysis and SVM

Predicting Treatment Outcome in Cervical Cancer Using Texture Analysis and SVM

kard786
Support Vector Machines

Support Vector Machines

tuft_ara
Machine Learning for Public Health in R

Machine Learning for Public Health in R

kera671
Black-box Image Extraction Attacks on RBF SVM Model

Black-box Image Extraction Attacks on RBF SVM Model

dsaub
Support Vector Machines for Classification and Regression

Support Vector Machines for Classification and Regression

luhr633
Advanced Protein Prediction Methods: Deep Learning Approach

Advanced Protein Prediction Methods: Deep Learning Approach

jermiahbi

More Related Content