FlashP: Analytical Pipeline for Real-time Series Relational Data Forecasting
"Explore FlashP, an analytical pipeline for real-time forecasting of time-series relational data. Learn about tasks like decision-making on online advertising platforms, bottleneck speedup via sampling, and the use of generalized weighted samplers for accurate estimation. Dive into forecasting with noisy input and the application of ARMA and LSTM models for prediction. Discover how the Generalized Weighted Sampler ensures high accuracy metrics in an easy-to-implement manner."
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
FlashP: An Analytical Pipeline for Real FlashP: An Analytical Pipeline for Real- -time Forecasting of Time Forecasting of Time- -Series Relational Data Series Relational Data time Shuyuan Yan1, Bolin Ding1, Wei Guo1, Jingren Zhou1, Zhewei Wei2, Xiaowei Jiang1, and Sheng Xu1 1Alibaba Group 2Renmin University of China
Outline Real-time Forecasting Task Forecasting with Noisy Input Generalized Weighted Sampler Experiments
Real-time Forecasting Task Decision making on online advertising platform Decide bidding strategy based on the forecasting of Impression 1. 2. 3. Rewriting it into OLAP aggregation queries Fitting a forecasting model Forecasting for a future time stamp
Real-time Forecasting Task Bottleneck: speedup via sampling? ?: data source ?: the metric to be forecasted C: the predicate (??,??): historical data (used to train the model) t_future: future time point when the metric is forecasted
Outline Real-time Forecasting Task Forecasting with Noisy Input Generalized Weighted Sampler Experiments
Forecasting with Noisy Input A simple ARMA model with noisy data points Model noise Historical data points: from aggregation queries Noisy data points: from aggregation queries on samples Sampling noise Prediction error is proportional to a linear combination of model noise and sampling noise
Forecasting with Noisy Input A more complex forecasting model based on LSTM Sampling noise Prediction error is proportional to a linear combination of model noise and sampling noise
Outline Real-time Forecasting Task Forecasting with Noisy Input Generalized Weighted Sampler Experiments
Generalized Weighted Sampler (GSW) Estimating metric with high accuracy Easy to be implemented/parallelized/maintained Handling a large number of metrics with small space Sampling weight: ? = ?? ? ? Drawing each ? ? into sample ? with probability ?? +?? Estimating ? = ? ??? as ? = ? ? ??, where: ??=?? +?? ??
Generalized Weighted Sampler (GSW) Estimating ? = ? ??? as ? = ? ? ??, where: ??=?? +?? ?? Unbiasedness: Error is bounded ( ): as long as ?? and ?? are close
Generalized Weighted Sampler (GSW) Error is bounded as long as ?? and ?? are close Metric to be aggregated: ? = 100,100,200,400 Sampling weights: ? = 10,10,20,50 Min ratio = 400/50 = 8 Max ratio = 100/10 = 10 Equivalent to priority sampling [Duffield et al. 2007] when ? = ?
Generalized Weighted Sampler (GSW) Error is bounded as long as ?? and ?? are close => No need to maintain a sample for every metric ?? (1), ?? (2), , ?? (?) with Maintain one sample for ? metrics ?? sampling weights ?? + or ?? Geometric compressed GSW Arithmetic compressed GSW
Outline Real-time Forecasting Task Forecasting with Noisy Input Generalized Weighted Sampler Experiments
Experiments 100 days of data, 15M rows per day 15 dimensions (4 of them are metrics to be predicted: Favorite, Impression, Click, and Cart)
