Few-Shot Link Prediction Using Variational Heterogeneous Attention Networks
Explore a novel approach for few-shot link prediction leveraging variational heterogeneous attention networks to enhance representations of high-frequency and few-shot entities. Addressing challenges of long-tailed distributions, tail entity importance variations, and information bottlenecks, the model demonstrates improved generalizability and performance against existing methods.
Uploaded on | 0 Views
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Few-Shot Link Prediction using Variational Heterogeneous Attention Networks SEKE 2023 Xi Tao, Hao Wang, Xiangfeng Luo and Pinpin Zhu School of Shanghai University
Introduction Motivation Existing models can effectively learn the representation of high-frequency entities and high- frequency relations in the process of obtaining node features, but their ability to represent few-shot entities and few-shot relations is insufficient. The method based on graph attention mechanism focuses on measuring the weight of neighboring entities, but ignoring the fact that neighboring relations and neighboring entities can interact each other and collectively enhance the representation of central nodes. 2
Introduction Challenge 1 1. The entities and relations presents a long-tailed distribution, which makes the model is easily disturbed by irrelevant features because of lack of sufficient data 3
Introduction Challenge 2-3 2. The relation cooperate links to different head entities, and it may be more important when linking to Bill Gates instead of David , because the former is a influential figure, which illustrates that the contribution of the same relation may be different due to different head entities. 3. The tail entity ChatGPT may be more important when linking to develop instead of user of , because develop needs more effort, which illustrates that the contribution of tail entity may be affected by the relation. 4
Model Our Model We dynamically adjust the weights of relations and tail entities when aggregating information about neighboring nodes and let them optimize each other when updating node embeddings. We introduce the principle of information bottlenecks to control the information from the encoder to the decoder, which helps the model to better capture potentially useful features and improve its generalizability to few-shot entities. Experimental results show the strength of our model against another model in various metrics. 5
Experimental Result and Analysis Experimental Result Experimental Result shows that our strategy, which dynamically assigns weights to entities and relations when aggregating neighborhood information and eliminates redundant feature, is correct. 6
Experimental Result and Analysis Ablation Study T-SNE visualization of sampled relations and their corresponding tail entities in the test set using TransE (above) and ViHAN (below) on FB15k-237. Obviously, ViHAN can better distinguish the few-shot entities. 7
Experimental Result and Analysis Ablation Study Ablation experiment results on few-shot entities, tail-10 entities and tail-20 entities represent the 10\% and 20\% tail entities with the lowest frequency respectively. 8
Conclusion In this paper, we propose link prediction method for few-shot entities based on the information bottleneck principle and evaluate the representation capability of the model on link prediction task. Specifically, we design heterogeneous attention networks to model the importance of neighborhood information and better enhance the representation of the central entities. In addition, we introduce VIB to mine the deep features of nodes and improves the performance on few-shot entities. Extensive experimental results show that our approach outperforms other baseline models on mainstream datasets. 9
Thanks! 10
