Seq2Seq Models in Data-to-Text Generation

how do seq2seq models perform on end to end data n.w
1 / 30
Embed
Share

Explore the performance of Seq2Seq models in end-to-end data-to-text generation, analyzing traditional methods, neural generation methods, and prominent models like Transformer and Pointer Generator. Understand the applications, datasets, evaluation methods, and results, providing insights into the evolving landscape of data-to-text generation.

  • Seq2Seq
  • Data-to-Text
  • Neural Generation
  • Transformer Model
  • Pointer Generator
rayaanacharya
nrale Follow

Uploaded on | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.

Download Presentation

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.

E N D

Presentation Transcript

  1. How Do Seq2Seq Models Perform on End-to-End Data-to-Text Generation? Xunjian Yin, Xiaojun Wan Wangxuan Institute of Computer Technology, Peking University The MOE Key Laboratory of Computational Linguistics, Peking University ACL 2022

  2. How Do Seq2Seq Models Perform on End-to-End Data-to-Text Generation? Data-to-Text Generation Models Datasets Evaluation Method Result Analysis Conclusion 2025/7/7 2

  3. Data-to-Text Generation 1. Text Generation (Natural Language Generation) Applications: dialog system, machine translation, text summarization Input data: unstructured & structured 2. Data-to-Text Generation Applications: weather report, sports news, dialog response Input data: databases of records, spreadsheets, knowledge graph 2025/7/7 3

  4. Data-to-Text Generation 3. Traditional Methods (Kukich, 1983; Reiter and Dale, 2000; Mei et al., 2015) Pipeline modules: content planning, sentence planning and surface realization Template-based algorithm 4. Neural Generation Methods (Lebret et al., 2016; Wiseman et al., 2017a) Encoder-decoder architecture: end-to-end architecture based on Seq2Seq paradigm Pre-trained models Neural data-to-text generation: A comparison between pipeline and end-to-end architectures. 2019. https://arxiv.org/pdf/1908.09022.pdf 2025/7/7 4

  5. Models 1. Transformer (2017) self-attention and multi-head attention DUALENC (Zhao et al., 2020) Bridging the Structural Gap Between Encoding and Decoding for Data-To-Text Generation. 2020. https://aclanthology.org/2020.acl-main.224.pdf 2025/7/7 5

  6. Models 1. Transformer (2017) Graph Transformer (Wang et al., 2020) AMR-To-Text Generation with Graph Transformer. 2020. https://aclanthology.org/2020.tacl-1.2.pdf 2025/7/7 6

  7. Models 1. Transformer (2017) (Zhu et al., 2019) Modeling Graph Structure in Transformer for Better AMR-to-Text Generation. 2019. https://aclanthology.org/D19-1548.pdf 2025/7/7 7

  8. Models 2. Pointer Generator (2017) Pointer Network (Vinyals et al. , 2015) Pointer Networks. 2015. https://arxiv.org/pdf/1506.03134.pdf 2025/7/7 8

  9. Models Get To The Point: Summarization with Pointer-Generator Networks. 2017. https://aclanthology.org/P17-1099.pdf 2. Pointer Generator (2017) Pointer Generator Network (See et al. , 2017) 2025/7/7 9

  10. Models 2. Pointer Generator (2017) Copy mechanism Deep Graph Convolutional Encoders for Structured Data to Text Generation. 2018. https://arxiv.org/pdf/1810.09995.pdf Ahierarchical model for data-to-text generation. 2020. https://arxiv.org/pdf/1912.10011.pdf ToTTo: A Controlled Table-To-Text Generation Dataset. 2020. http://aclanthology.lst.uni-saarland.de/2020.emnlp-main.89.pdf 2025/7/7 10

  11. Models 3. BART (2019) pre-training method and a large number of parameters GPT BERT 2025/7/7 11

  12. Models HTLM: Hyper-Text Pre-Training and Prompting of Language Models. 2021. https://arxiv.org/pdf/2107.06955v1.pdf 3. BART (2019) HTLM (Armen et al. , 2021) structured prompt 2025/7/7 12

  13. Models 4. T5 (2019) T5-3B (Kale , 2020) ToTTo dataset Text-to-Text Pre-Training for Data-to-Text Tasks. 2020. https://arxiv.org/pdf/2005.10433v3.pdf T5-large WebNLG dataset T5-large with a two-step fine-tuning mechanism (Wang et al., 2021) Stage-wise Fine-tuning for Graph-to-Text Generation. 2021. https://aclanthology.org/2021.acl-srw.2.pdf 2025/7/7 13

  14. Models 4. T5 (2019) T5-large WebNLG dataset T5-large with controlprefixes tuning (Clive et al., 2021) Control Prefixes for Parameter-Efficient Text Generation. 2021. https://arxiv.org/pdf/2110.08329v2.pdf 2025/7/7 14

  15. Models Representative Seq2Seq models: Transformer(Vaswani et al., 2017) Transformer with Pointer Generator (See et al., 2017) T5(small&base) (Raffel et al., 2019) BART(base) (Lewis et al., 2019) 2025/7/7 15

  16. Datasets 2025/7/7 16

  17. Evaluation Method Automatic metric: BLEU (Bilingual Evaluation Understudy, 2002) n-gram precision 1-gram: 5/6 3-gram: 2/4 2025/7/7 17

  18. Evaluation Method Automatic metric: BLEU (Bilingual Evaluation Understudy, 2002) 1-gram: 2/7 1-gram: 1 2025/7/7 18

  19. Evaluation Method Automatic metric: BLEU (Bilingual Evaluation Understudy, 2002) Advantages: fast computation, low computational cost, easy to understand. Disadvantages: no consideration of grammatical accuracy andsynonyms Human Evaluation Multidimensional Quality Metric (MQM) Error-oriented fine-grained human evaluation method 2025/7/7 19

  20. Evaluation Method Human Evaluation Errors related to accuracy: Errors related to fluency: Severity: A. Addition B. Omission C. Inaccuracy Intrinsic D. Inaccuracy Extrinsic E. Positive-Negative Aspect A. Duplication B. Word Form C. Word Order A. Minor B. Major C. Critical ?? the set of all error segments of type t ?? the deduction ratio set as 1:3:7 ?? the word length of the error 2025/7/7 20

  21. Evaluation Method Human Annotation five annotators 80 data-text pairs for each dataset Each text is annotated by two different annotators respectively. Each annotator must label all the five outputs generated by five models of one input sequence. 2025/7/7 21

  22. Result Analysis Human evaluation scores BLEU scores 2025/7/7 22

  23. Result Analysis Human evaluation scores 2025/7/7 23

  24. Result Analysis Copy Mechanism 2025/7/7 24

  25. Result Analysis Pre-training 2025/7/7 25

  26. Result Analysis Model Size T5-small: 60M BART-base: 139M T5-base: 220M 2025/7/7 26

  27. Result Analysis Dataset Example in the ToTTo dataset. 2025/7/7 27

  28. Result Analysis Error Types Table: Average error score of each error type across all models and datasets (lower means better). 2025/7/7 28

  29. Conclusion using a fine-grained set of human evaluation metrics based on MQM using five representative Seq2Seq models on four commonly used data-to-text datasets analyze the effect of milestone techniques evaluate models performance in terms of different types of errors

  30. 2025/7/7 30

Related


No related presentations.

More Related Content