Foundations of Algorithms & Machine Learning: Overview and Logistics
Explore the world of probabilistic machine learning and its foundations with a senior scientist from TCS Labs. Learn about the prerequisites, recommended books, and the reasons why machine learning has gained momentum. Dive into different types of machine learning problems like supervised learning and understand the significance of learning in machines. Uncover the impact of data, computational resources, business interest, and sound mathematical principles on the growth of machine learning.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Who am I? Work in Probabilistic Machine Learning Like to teach Feb 2017 current: Senior Scientist, TCS Labs Previously at IBM Research India, IISc Bangalore 2006: Phd in CS, U. Maryland, College Park 1999: Btech in CSE, IIT KGP 1 Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya
What about you? And what would you like to get out of this course? 2 Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya
Logistics Pre-requisites Probability Linear algebra, optimization, high school calculus, Recommended Books Machine Learning: A Probabilistic Perspective , Kevin Murphy Pattern Recognition and Machine Learning , Christopher Bishop An Introduction to Probabilistic Graphical Models , Michael Jordan Evaluation Assignments Exam 3 Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya
Machine Learning: Why and What? 4 Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya
How are ML problems different? Learning What does it mean? Abductive and inductive reasoning vs deductive reasoning Why do machines / programs need to learn? Does it make sense to learn to sort or to add? 5 Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya
Why has ML taken off? Data Computational resources, storage Business interest Toolkits Sound mathematical principles 6 Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya
Types of ML problems: Supervised Learning Predict response / output / label given input Input: d-dimensional vector. Features / attributes Output: Categorical: Binary, Multi-class, Multi-label Real valued: Simple regression, multiple regression Ordinal: Ordinal regression Ranking: Learning to rank Structured: Sequence, Tree, Graph Training set: both inputs and outputs available Test set: for evaluation; only inputs available 7 Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya
Types of ML problems: Unsupervised Learning Also called Knowledge discovery No labels available No clear notion of training or test Discover patterns e.g. clusters in the data Density estimation Evaluation? 8 Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya
Types of ML Problems: Reinforcement Learning 1. A robot vehicle on Mars 2. Learning to play chess Direct feedback only at the end Possibly closer to human learning Not covered in this course 9 Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya
Concept Learning: Example Hidden set of instances called concept Need to predict if a test instance is in that set Given training examples from that set 10 Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya
Probabilistic Concept Learning What are possible concepts? (Hypothesis space) How likely are the training examples to come from a particular concept? (Training likelihood) How likely is the test example to belong to a particular concept? (Test Likelihood) Bayesian Concept Learning How likely is a particular concept apriori? (Prior probability) How is likely is a particular concept after seeing the training data? (posterior probability) How likely is the test example considering all these concepts and their posterior probabilities? 11 Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya
Notions in ML Overfitting, generalization, regularization Validation of learning: whether future test instances are correctly predicted Curse of dimensionality Lots of features is not always a good idea Model selection Need to know many different ML models No free lunch theorem 12 Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya
Course Plan Module 1 Introduction Machine Learning: What and why Probability review for Machine Learning Estimation and decision theory Simple probabilistic models for classification and regression Module 2 Latent variable models Graphical Models: Models for complex (non-iid) data Module 3 Non parametric models, Kernel methods Ensemble methods 13 Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya
