Revolutionizing Reddit Summarization with Multi-level Memory Networks
Explore the innovation of abstractive summarization on Reddit using Multi-level Memory Networks, surpassing previous methodologies. This method addresses challenges in understanding document levels, introduces the new Reddit TIFU dataset, and presents significant advancements in text abstraction techniques. Learn about the unique model named multi and its improved performance compared to current state-of-the-art approaches.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Abstractive Summarization of Abstractive Summarization of Reddit with Multi with Multi- -level Memory Networks level Memory Networks [ [NAACL NAACL 2019] RedditPosts Posts 2019] Group Presentation WANG, Yue 04/15/2019 WANG, Yue The Chinese University of Hong Kong
Outline Outline Background Dataset Method Experiment Conclusion WANG, Yue The Chinese University of Hong Kong 2/16
Background Background Challenge: Challenge: Previous abstractive summarization tasks focus on formal texts (e.g., news articles), which are not abstractive enough. not abstractive enough. Prior approaches neglect the different level understandings different level understandings of the document (i.e., sentence-level, paragraph-level and document-level). Contribution: Contribution: 1. Newly collect a large-scale abstractive summarization dataset named Reddit TIFU (the first informal texts the first informal texts for abstractive summarization). 2. Propose a novel model named multi which considers multi-level abstraction of the document and outperforms existing state-of-the-arts. multi- -level memory networks (MMN) level memory networks (MMN), WANG, Yue The Chinese University of Hong Kong 3/16
Dataset Dataset Dataset is crawled from a a subreddit https://www.reddit.com/r/tifu/ subreddit /r/ /r/tifu tifu The important rules The important rules under this subreddit: : The title must make an attempt to encapsulate the nature of your f***up All posts must end with a TL;DR summary that is descriptive of your f***up and its consequences. Smart adaption Smart adaption: The title short summary The TL;DR summary long summary WANG, Yue The Chinese University of Hong Kong 4/16
Dataset Dataset Example Example: : WANG, Yue The Chinese University of Hong Kong 5/16
Dataset Dataset Weak lead bias Strong abstractness WANG, Yue The Chinese University of Hong Kong 6/16
Method Method Multi Multi- -level level Memory Networks (MMN Memory Networks (MMN) ) The advantages of MMN The advantages of MMN Better handle long range dependency Build representations of not only multiple levels but also multiple ranges (e.g. sentences, paragraphs, and the whole document) The key components of MMN The key components of MMN Multi-level Memory Memory Writing with Dilated Convolution Normalized Gated Tanh Units State-Based Sequence Generation Read multi-level layers in the encoder WANG, Yue The Chinese University of Hong Kong 7/16
Method Method Encoder input: ?? ?=1 Encoder layers: ?? 1,2, ,? Decoder input: ?? ?=1 Decoder layers: ?? 1,2, ,? ? ? ? ,? = ?=1 ? ? ? ,? = ?=1 WANG, Yue The Chinese University of Hong Kong 8/16
Method Method Construction of Multi Construction of Multi- -level Memory Writing with Dilated Convolution: level Memory Memory Standard Convolution (d=1) Dilated Convolution (d=2) By stacking multi-layer dilated convolutions, we get: ?? ?=1 ,? = 1,2, ,? ? ? WANG, Yue The Chinese University of Hong Kong 9/16
Method Method Normalized Gated Normalized Gated Tanh Tanh Units Units WANG, Yue The Chinese University of Hong Kong 10/16
Method Method State State- -Based Sequence Generation Based Sequence Generation WANG, Yue The Chinese University of Hong Kong 11/16
Method Method Difference between MMN with ConvS2S Difference between MMN with ConvS2S MMN can be viewed as an extension of ConvS2S The term Memory network is inappropriately employed to some extent Attention Memory network ConvS2S ConvS2S Standardconvolution Gated TanhUnits Onlylook at the finallayer of the encoder MMN MMN Dilatedconvolution Normalized Gated TanhUnits Based ondifferent level memories of the encoder Motivation Motivation Capture larger range Empirical found Simulate different level of abstraction ConvolutionType ConvolutionOutput Unit During decoding WANG, Yue The Chinese University of Hong Kong 12/16
Experiments Experiments Qualitative Results Qualitative Results WANG, Yue The Chinese University of Hong Kong 13/16
Experiments Experiments Quantitative Quantitative Results User preference Summary examples Results WANG, Yue The Chinese University of Hong Kong 14/16
Conclusion Conclusion A new dataset Reddit TIFU for abstractive summarization on informal online texts A novel summarization model named multi-level memory networks (MMN) WANG, Yue The Chinese University of Hong Kong 15/16
Conclusion Conclusion Thanks WANG, Yue The Chinese University of Hong Kong 16/16
