Chinese Zero Pronoun Resolution using Deep Memory Network
Explore a research paper on Chinese zero pronoun resolution using a deep memory network. The paper discusses the concept of zero pronouns, the goal of recovering ZPs in sentences, challenges in resolution, and the proposed approach using memory networks to overcome these challenges. Discover how the network utilizes contextual information to address semantic gaps in ZP resolution.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
A Deep Memory Network for Chinese Zero Pronoun Resolution Qingyu Qingyu Yin Yin, Yu Zhang, Weinan Zhang and Ting Liu Research Center for Social Computing and Information Retrieval, Harbin Institute of Technology, Harbin, China
1 INTRODUCTION
What is Zero Pronoun Zero Pronoun (ZP) The gap in the sentence <ZP> Express the sentence Represent something that is omitted Certain entities
Introduction Goal of this paper Recover the ZP in the sentence Entities Antecedent Noun Phrases ZP Antecedents <ZP> ZP is ubiquitous in Chinese Overt Pronoun 96%(English) 64%(Chinese)
A common way to ZP resolution Classification Problem ZP ( <ZP> ) Select a set of NP candidates Mention pair approach Classify for each pair ZP,NP1 ZP,NP2 ZP,NPn ZP NP2
Challenges of ZP resolution Overlook Semantic information Difficult to represent ZPs zero pronoun overt pronouns No descriptive information Gender Number Represent gaps with some available components Context information <ZP>
Challenges of ZP resolution Overlook Semantic information Represent gaps with some available components Context information Potential candidate antecedents Only some subsets of candidate antecedents are needed Select importance candidates Utilize them to build up representations for the ZP Memory Network Use the importance of candidate explicitly
2 THE APPROACH
The approach -- ZP Semantic information is overlooked ZP has no descriptive information No actual content
The approach -- ZP Semantic information is overlooked ZP has no descriptive information No actual content Represent ZP contextual information <ZP> taste sweet. apples books
The approach -- ZP For ZP ZP-centered LSTM Employ two LSTM one to model the preceding context one to model the following context h h 1h 2h nh nh 1 1 zp zp 1 ZP LSTMpre LSTMpre LSTMpre LSTMfol LSTMfol LSTMfol ...... ...... v zp w 1 w w w w w 2 1 1 zp zp 1 n n
The approach Select NP Represent NP Average content Head word of an NP LSTM-based approach for modeling NP Content information ' ' v h h [1] 1 c c c ch [ ] c m h ch nh [1] 1 Context information ... ... w 1 w w w w w 1 c [1] [ ] c m c 1 c n ... ... ... Candi ... ... ch [ ] c m h 1h ch [1] 1 v [ ] c m h h 1 c c
The approach Memory Network Memory: { r(np1), r(np2), r(np3), , r(npn) } Select NP to fill in the gap (ZP) v v v v v np np np np np 1 2 3 4 5 Weighted Sum v v v v v np np np np np 1 2 3 4 5 + Result Attention v v Linear zp zp hop1 hop2 hop3
The approach Memory Network Memory: { r(np1), r(np2), r(np3), , r(npn) } Select NP to fill in the gap (ZP) v v v v v np np np np np 1 2 3 4 5 Weighted Sum v v v v v np np np np np 1 2 3 4 5 + Result Attention v v Linear zp zp hop1 hop2 hop3
The approach Memory Network Memory: { r(np1), r(np2), r(np3), , r(npn) } Select NP to fill in the gap (ZP) v v v v v np np np np np 1 2 3 4 5 Weighted Sum v v v v v np np np np np 1 2 3 4 5 + Result Attention v v Linear zp zp hop1 hop2 hop3
The approach Get attention score for each NP candidate r(ZP), r(npi) feature vector ve(ZP,npi) si= tanh(W(r(ZP), r(npi) , ve(ZP,npi) )) For all the candidate NPs Add a softmax layer to gain the final attention score
3 EXPERIMENTAL RESULTS
Experimental Results Data set: Ontonotes 5.0 Experimental results R P F 51.0 53.0 53.7 53.9 54.4 51.4 53.3 54.0 54.2 54.7 51.2 53.1 53.9 54.1 54.3 Baseline system Our approach (hop 1) Our approach (hop 2) Our approach (hop 3) Our approach (hop 4) Baseline Chinese zero pronoun resolution: A deep learning approach. [C]. 2016 ACL. - Chen chen and Ng
Experimental Results Effectiveness of modeling ZPs and NPs R P F 52.0 51.7 51.9 ZPContextFree 51.4 51.7 51.5 AntContextAvg 52.2 53.0 52.5 53.3 52.3 53.1 AntContHead Our approach (hop 1)
Experimental Results Visualize Attention * pr o* * pr o* NPs: 1 0 0 0 0 0 1 1 0 1 0 0 0 hop 1 hop 2 hop 3 hop 4 hop 5 hop 6
4 CONCLUSION
Conclusion Effective memory network for modeling ZPs contextual information ZP-centered LSTM candidate antecedents Modeling candidate antecedents multi-layer attention Memory network Future work Embeddings UNK embeddings Avoid Feature engineering
Thanks ! Q&A
