Learning Using Privileged Information: SVM and Weighted SVM
Explore SVM and Weighted SVM techniques in this study, which reveal the uniqueness of SVM solutions, the relation between SVM and WSVM, and the direct weight learning from data. Discover how these methods optimize risk estimation and enhance classifier performance across various experimental settings.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Learning using privileged information: SVM+ and weighted SVM (Neural Networks 2014) Maksim Lapin, Matthias Hein, Bernt Schiele
SVM+ and WSVM primal form SVM+ SVM with additional information available only at training time WSVM SVM with instance weighting
SVM+ and WSVM dual form SVM+ SVM with additional information available only at training time WSVM SVM with instance weighting
Contributions of this work The authors show that any non-trivial SVM+ solution is unique. (in the primal form) any SVM+ dual solution can be used to construct weights for the weighted SVM (WSVM) that will yield the same primal solution up to the non-uniqueness of b. the SVM+ solutions are a strict subset of the WSVM solutions. the weights can be learned directly from data by minimizing an estimate of risk similar to standard procedures of hyper-parameter tuning.
SVM+ WSVM
SVM+ WSVM
Experiments toy data The setting with the enough size of validation set The setting with the limited size of validation set
Experiments digit recognition (5s v. 8s) The extended setting where each digit is translated by 1 pixel in each of the 8 directions The original setting
Conclusion The authors considered the weight learning method which allows one to learn weights directly from data (using a validation set). The latter approach is not limited to SVMs and can be extended to other classifiers. Experimental results confirmed the intuition that importance weighting is a powerful method of incorporating prior knowledge.
