Expectation Maximization

Discover the Expectation Maximization (EM) Algorithm, a powerful statistical method for estimating unknown parameters in a model. Explore its application in Maximum Likelihood estimation, clustering, and more. Learn about the EM Algorithm's iterative process of alternating between the expectation and maximization steps until convergence is reached. Delve into examples using Scikit-learn and gain insights into how EM Algorithm is utilized in various fields such as population estimation, coin tossing, and clustering using the Old Faithful geyser data. Uncover the importance of EM Algorithm in finding local optima and its potential limitations. Access valuable resources for further reading and sample code for implementing EM Algorithm in clustering tasks.

• EM Algorithm
• Expectation Maximization
• Maximum Likelihood
• Clustering
• Scikit-learn

sulami Follow

Uploaded on Mar 08, 2025 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.

Presentation Transcript

1. Expectation Maximization Heymo Kou Jan 9th, 2017

2. Contents Maximum Likelihood EM Algorithm Coin Toss What is EM Algorithm? Example using Scikit-learn 2/9

3. Maximum likelihood [Ronald Fisher, 1912] Estimate population from a given sample Find probability of head for each coin ? = ?+? ? Are you sure it s the best estimation? 3/9

4. EM Algorithm [Dempster et. al., 1997] Repeat guessing and calculating Until local optima has found 2 Steps Expectation Step Guess unknown parameters Maximization Step Observe result of guessed parameters Reuse the new result in Expectation Step 4/9

5. EM Algorithm in nutshell initial guess E step Guess of unknown hidden structure (tags, parses, weather) Guess of unknown parameters (probabilities) Observed structure (words, ice cream) M step 5/9

6. Coin toss [Chuong B Do & Serafim Batzoglou, 2008] Published in Nature What is the EM algorithm? Probability of k heads 6/9

7. EM Clustering Using Old Faithful geyser data https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm 7/9

8. Sample Code Clustering using EM Algorithm http://scikit-learn.org/stable/auto_examples/mixture/plot_gmm_sin.html 8/9

9. Summary Maximize expectation by repeating Expectation step Maximization step Guaranteed to find local optima Cons Iteration may greatly slow If dataset gets big Cf) Monte Carlo method (Randomized) Used in Alpha Go 9/9

10. Reference http://ai.stanford.edu/~chuongdo/papers/em_tutorial.pdf JHU CS 600.465 - Intro to NLP - J. Eisner 10/9