AUTOMATIC GLOSS FINDING
This research work focuses on developing a system for automatically finding glosses for entities in a knowledge base by leveraging ontological constraints. The study addresses the need for gloss finding in various NLP applications and explores examples and problems related to matching glosses to entities effectively.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
1 AUTOMATIC GLOSS FINDING for a Knowledge Base using Ontological Constraints Bhavana Dalvi (PhD Student, LTI) Work done with: Prof. William Cohen, CMU Prof. Einat Mikov, University of Haifa Prof. Partha Talukdar, IISC Bangalore
Need for gloss finding 3 KBs are useful for many NLP tasks: E.g. Question answering Lot of research in fact extraction to populate KBs Glosses can help further in applications like - Word/Entity sense disambiguation - Information retrieval Automatically constructed KBs lack glosses e.g. NELL, YAGO
Example: Gloss finding 4 Class constraints: Inclusion: Every entity that is of type Fruit is also of type Food . Mutual Exclusion: If an entity is of type Food then it cannot be of type Organization
Example: Gloss finding 7 Knowledge Bases: NELL / Freebase / YAGO Candidate glosses: DBPedia abstracts/ Wiktionary definitions
Problem Definition 9 Input Example KB classes Food, Fruits, Company Ontological constraints: Subset, Mutex Fruit Food, Food Company = ? Entities E belonging to KB categories Banana, Microsoft Lexical strings L that refer to entities E E.g. MS , microsoft inc Candidate glosses E.g. G3: Apple, formerly Apple Computer Inc., is an American multinational corporation headquartered in Cupertino Output: Matching candidate glosses to entities in the KB E.g. (Apple, G3) Company:Apple
Can we use existing techniques? 10 Problem: Match potential glosses to appropriate entities in the KB. Entity linking: Assume existence of glosses on KB side Input KB does not have glosses Chicken & egg problem Ontology alignment: Both ends being matched are structured databases Asymmetric problem: Structured KB on one side, without glosses Candidate glosses contain text but no structure
Proposed Gloss Finding Procedure 11 Decide head-NP for a gloss: NP being defined G3: Apple, formerly Apple Computer Inc., is an American multinational corporation headquartered in Cupertino Select candidate glosses for which string match (head-NP, KB entity) For each gloss a set of candidate KB entities (Apple, G3) (Fruit:Apple, Company:Apple) Classify the head-NP into KB classes using ontological constraints (Apple, G3) Company Choose the KB entity match based on chosen KB category. (Apple, G3) Company:Apple
Training classifiers for KB Categories 13 Test: Ambiguous glosses Train: Unambiguous glosses
Assumptions 14 If a gloss has only one candidate entity matching in a KB, then it is correct i.e. we assume that KB is always correct and complete in terms of senses. Assumption holds for 81% for NELL dataset Given the category, a mention is unambiguous [Suchanek WWW 07, NakasholeACL 13] i.e. we can differentiate between entities of different category but not within a category.
Methods 15 Baselines SVM Learning Train binary classifiers using unambiguous glosses Predict categories for ambiguous glosses Label propagation PIDGIN [Wijaya et al. CIKM 13]: Graph-based label propagation method. GLOFIN: semi-supervised EM + use of ontological constraints.
Proposed Method: GLOFIN 16 Initialize model with few seeds per class Iterate till convergence (Data likelihood) E step: Predict labels for unlabeled points For each unlabeled datapoint Find P(Class | datapoint) for all classes Assign a consistent bit vector of labels in accordance with ontological constraints M step: Recompute model parameters using seeds + predicted labels for unlabeled points
Proposed Method: GLOFIN 17 Initialize model with few seeds per class Iterate till convergence (Data likelihood) E step: Predict labels for unlabeled points For each unlabeled datapoint Find P(Class | datapoint) for all classes Assign a consistent bit vector of labels in accordance with ontological constraints M step: Recompute model parameters using seeds + predicted labels for unlabeled points
Estimating class parameters and assignment probabilities 18 Na ve Bayes Independent multinomial distributions per word K-Means Cosine similarity between centroid and datapoint von-Mises Fisher Data distributed on a unit hypersphere
Proposed Method: GLOFIN 19 Initialize model with few seeds per class Iterate till convergence (Data likelihood) E step: Predict labels for unlabeled points For each unlabeled datapoint Find P(Class | datapoint) for all classes Assign a consistent bit vector of labels in accordance with ontological constraints M step: Recompute model parameters using seeds + predicted labels for unlabeled points
Mixed Integer Linear Program 20 Input: P(Cj | Xi), Class constraints: Subset, Mutex Output: Consistent bit vector yji forXi Max {likelihood of assignment constraint violation penalty}
Proposed Method: GLOFIN 21 Initialize model with few seeds per class Iterate till convergence (Data likelihood) E step: Predict labels for unlabeled points For each unlabeled datapoint Find P(Class | datapoint) for all classes Assign a consistent bit vector of labels in accordance with ontological constraints M step: Recompute model parameters using seeds + predicted labels for unlabeled points
Experiments 22
Candidate glosses 23 DBPedia is a database derived from Wikipedia We use short abstracts (definitions upto 500 characters, from Wikipedia page) E.g. McGill University is a research university located in Montreal Quebec Canada Founded in 1821 during the British colonial era the university bears the name of James McGill a prominent Montreal merchant from Glasgow Scotland and alumnus of Glasgow University whose bequest formed the beginning of the university.
GLOFIN vs. SVM & Label propagation 25 Freebase Dataset: Performance on ambiguous glosses 100 80 60 SVM Labal Propagation GLOFIN-Na ve-Bayes 40 20 0 Precision Recall F1
GLOFIN vs. SVM & Label propagation 26 NELL Dataset: Performance on ambiguous glosses 80 70 60 50 SVM Labal Propagation GLOFIN-Na ve-Bayes 40 30 20 10 0 Precision Recall F1
Are the datasets close to real world? 27 Large fraction of data used for training 80% of NELL 90% of Freebase In real world scenarios, amount of training data might be a small fraction of the dataset. We simulate this by using 10% of unambiguous glosses for training
Small amount of training data 28 Freebase Dataset: Performance on ambiguous glosses 100 80 60 SVM Labal Propagation GLOFIN-Na ve-Bayes 40 20 0 Precision Recall F1
Small amount of training data 29 NELL Dataset: Performance on ambiguous glosses 80 70 60 50 SVM Labal Propagation GLOFIN-Na ve-Bayes 40 30 20 10 0 Precision Recall F1
Compare variants of GLOFIN 30 Freebase Dataset 85 80 75 Flat Hierarchical 70 65 60 K-Means von-Mises Fisher Na ve Bayes
Compare variants of GLOFIN 31 NELL Dataset 70 60 50 40 Flat Hierarchical 30 20 10 0 K-Means von-Mises Fisher Na ve Bayes
Some more experiments 32 Evaluating quality of automatically acquired seeds Manually creating gold standard for NELL dataset Different ways of scaling GLOFIN NELL to Freebase mappings via common glosses http://www.cs.cmu.edu/~bbd
Conclusions 33 And Future Work ..
Conclusions 34 Completely unsupervised method for gloss finding - using unambiguous matches as training data - hierarchical classification instead of entity linking Our proposed method GLOFIN: GLOFIN Label Propagation SVM Variants of Hierarchical GLOFIN Na ve Bayes K-Means , von-Mises Fisher Ontological constraints help for all GLOFIN variants Hierarchical GLOFIN Flat GLOFIN In future, we will like to add new entities to the KB.
Candidate NELL entities Entity selected by GLOFIN head-NP Gloss McGill_University McGill University is a research university located in Montreal Quebec Canada Founded in 1821 during the British colonial era the university bears the name of James McGill a prominent University:E, Sports_team:E University:E Kingston_upon_ Hull Kingston upon Hull frequently referred to as Hull is a city and unitary authority area in the ceremonial county of the East Riding of Yorkshire England It stands on the River Hull at its junction with City:E, Visual_Artist:E City:E Robert_Southey Robert Southey was an English poet of the Romantic school one of the so called Lake Poets and Poet Laureate for 30 years from 1813 to his death in 1843 Although his fame has been long eclipsed by that Person_Europe:E, Person_Africa:E, Politician_USA:E Person_Europe:E 35
36 Thank You Questions?
Extra Slides 37
Eval: quality of seeds for NELL KB 39 Noisy seeds: Only 81% leaf category assignments are correct Hierarchical labeling can help: 94% higher level category labels are correct
Creating gold standard for NELL 40 Gold standard for evaluation on ambiguous glosses For most glosses, precise category is part of NELL
NELL Freebase mappings via common glosses 41
Pros and Cons of GLOFIN 42 Advantages Limitations Generative EM framework that can build on SSL methods: NBayes, K-Means, VMF Assumption: Input KB is complete and accurate. Can label unseen datapoints once models are learnt. All experiments are done in transductive setting: need to extend for missing entities and categories in the KB.
Future work 43 Adding new entities to existing KB categories KBs are usually incomplete w.r.t coverage of entities. GLOFIN: Classifies mentions into categories Introducing new clusters of entities: missing categories in the KB Extensions similar to Exploratory EM [Dalvi et al. ECML 13] New categories: entities belonging to them, along with glosses for those entities
