Introduction to Video Background Subtraction Methods
Explore popular methods like Gaussian Mixture Model and Statistics for extracting the foreground in video action analysis. Learn about the process, advantages, and applications of background subtraction techniques.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Introduction to Video Background Subtraction 1
Motivation 2 In video action analysis, there are many popular applications like surveillance for security, home care and athletes action analysis to predict and develop the strategy. The first step is to capture the human or moving parts which we call foreground. And then use the information to recognize the actions. In this presentation, I will introduce the methods to extract the foreground.
Overall structure 3 3D model Estimation Human Joints Estimation Model learning Activity Recognition Background Subtraction Feature points
Methods of Background Subtraction 4 At early time, the widely used method is the Gaussian Mixture Model (GMM) [1] and statistics [2]. The main advantage of GMM is that the it can reach real time processing. But it is sensitive to small noise like changing the luminance. In recent years, robust principle component analysis (RPCA) model [3][4][5] is found good performance than other state-of- the-art means. But instead, it costs more iterative time.
GMM (Method 1) 5 GMM uses 3~5 Gaussian functions to each color channel pixel of the video frame. New input pixels the algorithm would check whether the value is smaller than the deviation of Gaussian which is determined by the past pixel values of the same pixel location to determine this pixel is foreground or not.
GMM (contd) 6 Traffic sequence This picture is result showed in [1] Left is original frame, and the right is the overlapped frames
GMM (contd) 7 We use the code provided by [1]. And the clip was shot by ourselves There are holes and noise in the left image
GMM (contd) 8 We use closing morphology to do post processing in order to get more clean images.
Foreground classification & Statistics (Method 2) 9 The work [2] tears foreground into 4 parts, including moving visual object (MVO), ghost due to deinterlacing of TV, moving visual object shadow (MVO shadow), and ghost shadow. Judge whether the pixel belongs to foreground. If not, use several previous frames to estimate the current value. Examples from [6] to show the effect of ghost
Foreground classification & Statistics 10 ??? : ghost shadow ??? moving visual object ? : ghost ????? : moving visual object shadow The result from [2]
RPCA (Method 3) 11 The method is to transform video frames to vectors. And then combine the vectors into a big matrix. It follows the conditions that B is background matrix which is low rank and F represent foreground matrix which is sparse. And the combination of the two matrix is the original frames. argmin ?,? ????(?) + ? ? 1 = B F + . . st D ? : constant ? : original frame matrix ? ?????????? ?????? 1 : ?1 norm ? ?????????? ??????
RPCA (contd) 12 The objective function in P.9 is an ill-posed problem, so the solution is to iteratively optimize B and F This is captured from [3].
RPCA (contd) 13 We also run the code provided by [3] with our own video clips. Use the code from [3] by our clips Comparing with GMM, obviously RPCA is more robust.
Generalized Fused Lasso (GFL) (Method 4) 14 The latest research about background subtraction is [4]. With the same low-ranked objective function. The zero norm term is modeled by generalized fused lasso function. + min ( ) rank B F , B F gfl = B F + . . st D ?, ??? : weighting ? : constant ? : original frame matrix ? ?????????? ?????? ?,? : spatial neighborhood pixels ?? ?? ?????????? ?????? ? ?????????? ??????
Generalized Fused Lasso (GFL) (contd) 15 Results comparison of GFL and RCPA from [4] Result from [4] Original image, ground truth, GFL
Conclusion 16 GMM only cost 0.03 second per frame, but it always cause holes in the foreground parts. Also the low-ranked matrix is fancy, but the cost time is high. (Almost 30 seconds for 10 frames) Also the shadow parts of images are not considered as foreground. There may need some constraints to remove the shadow. All of the methods discussed above are only applied in static cameras.
References 17 [1] Zoran Zivkovic Improved Adaptive Gaussian Mixture Model for Background Subtraction In Proc. ICPR, 2004 Rita Cucchiara, Costantino Grana, Massimo Piccardi, Andrea Prati Detecting Moving Objects, Ghosts, and Shadows in Video Streams IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 25, NO. 10, OCTOBER 2003 John Wright , Yigang Peng, Yi Ma, Arvind Ganesh, Shankar Rao Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices by Convex Optimization NIPS 2009 Xiaowei Zhou, Can Yang and Weichuan Yu Moving Object Detection by Detecting Contiguous Outliers in the Low-Rank Representation Pattern Analysis and Machine Intelligence, IEEE Transactions 2012 Bo Xin Yuan Tian Yizhou Wang Wen Gao Background Subtraction via Generalized Fused Lasso Foreground Modeling CVPR 2015 https://en.wikipedia.org/wiki/Interlaced_video [2] [3] [4] [5] [6]
