Shedding Light on Internet Video Quality Problems
This study delves into the structure of internet video quality issues, focusing on factors impacting user engagement and revenue. It examines recent efforts and critical aspects, backed by a rich dataset of 300 million video sessions. Methodologies for problem identification are outlined, emphasizing session clustering and key metrics like buffering ratio and average bitrate.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Shedding Light on the Structure of Internet Video Quality Problems in the Wild Junchen Jiang (CMU) Vyas Sekar (Stony Brook U.) Ion Stoica (UC Berkeley/Conviva Inc.) Hui Zhang (CMU/Conviva Inc.) 1
Video Quality Matters Quality impacts user engagement Subscription- and advertisement-driven revenue [X. Liu, et al. Sigcomm 2012] A lot of video sessions suffer bad quality 2
A Lot of Recent Efforts Video Source Global control plane [SIGCOMM 12] Screen Better video encoding services Video Player Encoders & Video Servers Better interaction with ISPs. e.g., [Akamai] ISP & Home Net CMS and Hosting Content Delivery Networks (CDN) Server-side optimization. e.g., [CoNEXT 12, SIGCOMM 11] Client-side techniques Better bitrate adaptation. e.g., [CoNEXT 12, Mobicom 13] 3
A Critical Missing Piece Structure of the quality problems Types of problem: e.g., CDN or AS problems? Values of problem: e.g., a particular bad CDN, or AS? Potential applications Focus efforts on certain subsystem (e.g., CDN) Why reactive approach does work or does not 4
Outline Motivation Structure analysis What-if improvement analysis Conclusion 5
Dataset 300 million video sessions over two weeks Panoramic view across video providers 4 Browsers (e.g., MSIE) 379 Sites (e.g., HBO) 5 Connection Types (e.g., Cellular wireless) VodOrLive 4 Player types (e.g., Flash) 19 CDNs (e.g., Akamai) 15K ASNs 6
Group Sessions into Clusters CDN1 ASN2 ASN1, CDN1 ASN1 ASN2, CDN1 ASN2, CDN2 ASN1, CDN2 CDN2 Sessions are grouped into clusters by attribute values. 7
Methodology of Structure Analysis Session cluster Problem sessions Problem session (Bad quality) 8
Identifying Problem Sessions Join failure: Joining started or not Buffering ratio: % of buffering Join time: Length of joining Average bitrate: Average of played bitrate Identifying problem sessions: a. Join failure: joining failed b. Join time: > 10 sec c. Bitrate: < 700Kbps d. Buffering ratio: > 5% 9
Methodology of Structure Analysis Session cluster Problem sessions Problem cluster 10
Identifying Problem Clusters Problem Clusters CDN1 Not statistically significant ASN2 ASN1, CDN1 ASN1 ASN2, CDN1 ASN2, CDN2 ASN1, CDN2 Not high enough CDN2 A problem cluster is a. High problem ratio w.r.t global average b. Statistically significant 11
Identify Critical Clusters Root& ProbRa/o& & =& 0.1& Critical cluster ASN1& ASN2& CDN1& CDN2& ProbRa/o& & =& 0.2& ProbRa/o& & =& 0.1& ProbRa/o& & =& 0.5& ProbRa/o& & =& 0.05& ASN1,& CDN2& ProbRa/o& & =& 0.1& ASN2,& CDN1& ProbRa/o& & =& 0.4& ASN1,& CDN1& ProbRa/o& & =& 0.55& Problem clusters may be related to same underlying root cause. 12
Methodology of Structure Analysis Session cluster Problem sessions Problem cluster Critical clusters 13
Outline Motivation Structure analysis Methodology Analysis results Prevalence and persistence Types and values of critical clusters What-if improvement analysis Conclusion 14
Measuring Persistence & Prevalence CDN1 ASN1, CDN1 ASN1, CDN1 ASN1 ASN1 Epoch1 Epoch2 Epoch3 Cluster ASN1, CDN1 ASN1 CDN1 Prevalence 2/3 2/3 1/3 Persistence 1 2 1 Prevalence: % of hours in which cluster C is a problem cluster Persistence: streak of epochs where C is a problem cluster 15
Prevalence of problem clusters 10% clusters: prevalence 10+%. Observations: Many clusters have high prevalence 16
Persistence of problem clusters 1% clusters: peak duration 24+ hrs Observations: Many clusters last several hours 17
Types of Critical Clusters 41% 48% 6% 14% 9% 11% Join Time Join Failure Top two types of critical clusters = Site, CDN Types of critical clusters are similar across quality metrics. 18
Where are these Critical Clusters? (Interesting anecdotes) ASN CDN Site Connection Type BufRatio Asian ISPs In-house single bitrate Single bitrate Mobile wireless JoinTime Chinese ISPs accessing CDNs in China, but loading player modules from US In-house CDNs of UGC providers High bitrate JoinFailure Same set as buffering ratio Same single global CDN, maybe low priority providers Bitrate Wireless providers UGC Site Different instances of critical clusters across metrics. 19
Outline Motivation Structure analysis What-if improvement analysis Conclusion 20
Proactive Approach Inspired by prevalence analysis Pick top-k critical clusters from previous week Metric Inter-week Proactive 0.19 After-the-fact 0.31 BufRatio Bitrate 0.09 0.14 JoinTime 0.42 0.49 JoinFailure 0.54 0.63 A proactive strategy can yield huge improvement. 21
Reactive Approach Inspired by persistence analysis Wait for cluster to become critical, then fix # of problem sessions (k) Original Reactive After-the- fact A simple reactive strategy can yield huge improvement. 22
Conclusions Video quality is critical to user engagement Lots of solutions, but ... Understanding of structure of quality issues is limited This work is an initial attempt to bridge a critical gap Key observations Many quality problems are prevalent and persistent Proactive or reactive strategies give huge improvement Future work Cost-benefit analysis, root cause diagnosis, .... 23
Questions 24
