Linear Classifiers in Computer Vision & Deep Learning
Explore the intricacies of linear classifiers in the realm of computer vision and deep learning, including disadvantages of k-NN, parameterized score functions, bias tricks, image data preprocessing, and more. Learn how linear classifiers map raw data to class scores and interpret results effectively.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
2019 MVL Lab Training Course Computer Vision & Deep Learning - 2 Presenter: Hao-Ting Li ( )
Disadvantages of k-NN The classifier must remember all of the training data and store it for future comparisons with the test data. Classifying a test image is expensive since it requires a comparison to all training images.
Linear Classification Score function maps the raw data to class scores Loss function quantifies the agreement between the predicted scores and the ground truth labels Classifier SVM Softmax classifier
Parameterized Score Function Formulation: ?? ??training dataset of images, each associated with a label ?? where ? = 1 ? and ?? 1 ? ? examples ? dimension for each example ? classes Linear classifier: ? ??;?,? = ???+ ? Parameters: ? ?? ?weights ? ?? 1bias vector
Bias Trick ? and ? can be combined into single matrix ? ? ??;? = ???
Linear Classifier Matrix multiplication ???is effectively. Our goal will be to set these parameters in such way that the computed scores match the ground truth labels across the whole training set. Once the learning is complete we can discard the entire training set and only keep the learned parameters.
Image Data Preprocessing Raw pixel values range from [0 255] In Machine Learning, it is a very common practice to always perform normalization of your input features Center subtracting the mean from every feature the pixels range from approximately [-127 127] Scale so that its values range from [-1, 1] Normalize ? ? ?
Loss Function Loss function (cost function, or objective) quantifies the agreement between the predicted scores and the ground truth labels Intuitively, the loss will be high if we re doing a poor job of classifying the training data, and it will be low if we re doing well.
Multiclass Support Vector Machine Loss (SVM) The SVM loss is set up so that the SVM wants the correct class for each image to a have a score higher than the incorrect classes by some fixed margin . margin classifier
Multiclass Support Vector Machine Loss (SVM) The score for the j-th class is the j-th element ??= ? ??;?? The Multiclass SVM loss (also called hinge loss ) for the i-th example ??= max 0,?? ???+ ? ??
Example ??= max 0,?? ???+ ? ?? The scores ? = 5,2,6 , margin = 1, and the first class is the true class. The loss is ??= max 0,2 5 + 1 + max 0,6 5 + 1 = 0 + 2 = 2
Regularization Suppose that we have a dataset and a set of parameters ? that correctly classify every example (i.e. all scores are so that all the margins are met, and ??= 0 for all ?). ??= ? ?? = max 0,??? ????+ max 0,?? ???+ ? ?? If some parameters ? correctly classify all examples (so loss is zero for each example), then any multiple of these parameters ?? where ? > 1 will also give zero loss.
Regularization L2-regularization 2 ? ? = ??,? ? ? The full Multiclass SVM loss becomes ? =1 ? ??+ ?? ? ?
Softmax Classifier Softmax function
Softmax Classifier Score function ? ??;? = ??? Softmax function ??? ? ??= ???? Loss function: cross-entropy loss ???? ???? ??= log
Softmax Classifier Score function 1 2 -5 exp(.) 3 ? ??;? = ??? sum Softmax function 30.196 2.71 7.39 0.006 20.09 ??? (.)/sum ? ??= 0.089 0.244 0.0... 0.665 ???? Loss function: cross-entropy loss ???? ???? ??= log
Information Theory View The cross-entropy between a true distribution ? and an estimated distribution ? is defined as: ? ?,? = ?(?)log? ? ? The Softmax classifier is minimizing the cross-entropy between the estimated class probabilities ? and the true distribution ? ? = [0, ,1, ,0] ???? ???? ? = log ? ? =
Summary Parametric approach Score function Loss function SVM Softmax
Reference CS231n: Convolutional Neural Networks for Visual Recognition http://cs231n.github.io/linear-classify/
