Efficient Neural Network for Time Series Forecasting
This study introduces EGNN, an efficient graph-enhanced neural network for multivariate time series forecasting. It explores the intricate spatiotemporal dependencies in real-world time series data and leverages deep learning techniques such as MLP, Transformer variants, and GNNs. The paper presents a spectral similarity-based graph construction method and a two-stage low-cost prediction strategy to enhance temporal modeling. Various methods such as RevIN Instance Normalization, Residual Connection, and Wavelet Convolution are discussed in detail to optimize forecasting accuracy.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
EGENN: An Efficient Graph-Enhanced Neural Network for Multivariate Time Series Forecasting Haoxuan Xu, Haiqi Zhu, Chunzhi Yi, Baichun Wei, Feng Jiang Harbin Institute of Technology, Harbin, China Yifan Chen, University of Nottingham Malaysia Semenyih, Malaysia
Introduction Realworld time series data often characterized by intricate spatiotemporal dependencies and multifaceted relationships Deep learning excels in multivariate time series forecasting MLP Emphasizing intra-series correlations using temporal decomposition techniques TimesNet converts time seies into 2D representations Transformer variants harness attention mechanisms to capture both temporal and spatial intersequence dependencies Transformer and GNN have fundamental equivalence in encoding spatial dependencies. GNNs are widely used in spatio-temporal prediction, their expressive power in temporal modeling is usually underutilized This paper spectral similarity-based graph construction method an efficient inter-layer attention graph convolutional network a two-stage low-cost prediction strategy
Adjacency Matrix Generator Method ??,??= 1, ?? ? ??,?,??,? ? ??? ? ? 0, ?? ?????? spectral similarity 2 ??,? 2+ ??,? ??,? S ??,?,??,? = 1 2 ??,? GNN Module ? 1 ??= ? ???? ??+ ?????(??) ?=1 graph convolutional layers ?(?)= ?(?? ??? 1??) ReLU ?(?) ?,? > 0 0,? 0 LeakyReLU ?(?) ?,? > 0 ??,? 0 Projection Module ??= ??? ???1(??????????? ??) ??= ??? ???2(?????? ??) * ??? ???1 ?? ? (?/?) * ?????? ?? ? (?/?)
Method - RevIN Instance Normalization ????? (?)= 1 ?? ?=1 1 ?? ?=1 ??? (?)= (?) ????? ??(??? ? ????? ?)2 ??? ??? (?) ????? (?) ?= ?? ??? + ?? (?)+? ??? ??? DeNormalization ? ?? ?? ??? (?)+ ? (?) ?= ??? ??? ??? + ?????
Method Residual conneaction When the input and output are of the same dimensions ? = ? ?, ?? + ? When the dimensions increase Extra zero entries padded for increasing dimensions The projection shortcut is used to match dimensions (done by 1 1 convolutions) ? = ? ?, ?? + ???
Method Wavelet Convolution (Haar) If the downsampling operator has a factor of 2 ???is a Low-pass filter ???=1 ???is a Horizontal High-Pass Filter filter ???=1 ???is a Vertical High-Pass Filter filter ???=1 ???is a Diagonal High-Pass Filter filter ???=1 1 1 1 1 2 1 1 1 1 1 1 1 2 1 1 2 1 1 1 2
Method Wavelet Convolution 1. ??() ???,???,???,??? = ???? ???,???,???,???,? 2. ???() ? = ???? ?????????? ???,???,???,???, ???,???,???,???
Dataset Dataset Flight Electricity ETTh1 ETTm1 Exchange Train 70% Valid 10% Test 20% Baseline Model iTransformer Dlinear TimesNet MSGNet FourierGNN Prediction lengths H {96, 192, 336, 720} Performance evaluation Mean Squared Error (MSE) and Mean Absolute Error (MAE) TimesNet ? = ?1,?2,?3,?4,?5,?6,?7,?8,?9,?10,?11,?12 If this data has periodicity, with a period of p=4 ?1 ?2 ?3 ?4 ?5 ?6 ?7 ?8 ?9 ?10 ?11 ?12 ? =
