Machine Translation Learning & Semantic Models - Insights and Applications
Discover the world of machine translation learning, similarity measures, and deep semantic similarity models. Explore key concepts such as S2Net, DSSM, and relational similarities, along with practical applications in natural language processing and knowledge base embedding. Dive into the evolution of language processing technologies and the challenges of distinguishing synonyms and antonyms. Uncover how polarity is induced and cosine scores are utilized for target words, all while examining real-world examples and advancements in the field.
Uploaded on | 1 Views
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
? = Search Machine Translation Learning Similarity Measures S2Net [CoNLL-11, SIGIR-11] DSSM [CIKM-13, ACL-14] DSSM Deep Structured Semantic Model, or more general, Deep Semantic Similarity Model
? = Search Machine Translation Learning Similarity Measures S2Net [CoNLL-11, SIGIR-11] DSSM [CIKM-13, ACL-14]
? = Search Machine Translation Learning Similarity Measures S2Net [CoNLL-11, SIGIR-11] DSSM [CIKM-13*, ACL-14] ? = : : ????( , ) Relational Similarity Multi-Relational LSA Word Analogy [NAACL-13 x2] Word Relation [EMNLP-12, EMNLP-13] Knowledge Base Embedding [EMNLP-14]
Q: Who won the best actor Oscar in 1973? S1: Jack Lemmon was awarded the Best Actor Oscar for Save the Tiger (1973). S2: Academy award winner Kevin Spacey said that Jack Lemmon is remembered as always making time for others.
sunny rainy cloudy windy car emotion cab sad wheel joy feeling
Tomorrow will be rainy. Tomorrow will be sunny. ???????(rainy, sunny)? ???????(rainy, sunny)?
Distinguishing synonyms and antonyms is still perceived as a difficult open problem. [Poon & Domingos 09]
Inducing polarity Target word: row-vector Cosine Score: + ????????
Inducing polarity Target word: row-vector Cosine Score: ????????
ostrich is a bird engine is a part of car Idea #2: Encode multiple relations in a 3- way tensor (3-dim array)!
joyfulness gladden joyfulness gladden sad sad anger anger Antonym layer Synonym layer Construct a tensor with two slices
joyfulness gladden sad anger Hyponym layer
?1,?2,,?? ? ? ?1,?2, ,?? ?1,?2, ,?? ?1,?2, ,?? ? ? latent representation of words
latent representation of a relation ?1,?2, ,?? ? ? ? ?1,?2, ,?? ?1,?2, ,?? ? ? ? ? latent representation of words
??? joy,sadden = cos ?:,joy,???,?:,sadden,??? joyfulness gladden joyfulness gladden sad sad anger anger Antonym layer Synonym layer
??? joy,sadden = cos ?:,joy,???,?:,sadden,??? joyfulness gladden joyfulness gladden sad sad anger anger Antonym layer Synonym layer
?,?:,:,?????,: ? ??? w?,w? = cos ?:,:,?????,: , ) Cos ( ? ?1,?2, ,?? ? ? ?? ?
subj-pred-obj (?1,?,?2) Obama Born-in Hawaii Bill Gates Nationality USA Bill Clinton Satya Nadella Spouse-of Hillary Clinton Microsoft Work-at ?: # entities, ?: # relations
?-th slice k ?? e1 en Hawaii e1 en 1 Obama A 0 entry means: Incorrect (false) Unknown ??: born-in
R? (?1,?,?2) ????1,??2 [Nickel+, ICML-11]
1 2 2+1 2+ 2 ?? ? ??? ?? ? ? ? 2 ? ? ?? ? ? ?? RESCAL [Nickel+, ICML-11]
1 2 ?? ??? 2 ? ?? ? locations ? ?? ? ? ?? Relation: born-in persons
1 2 ?? ??? 2 ? ?? ? ? ?? ? ? ??
# Entities # Relation Types # Entity Types # Entity-Relation Triples 753k 229 300 1.8M
(??,??,?) (??,?,??) ?? ?? ?? RESCAL TransE
Dan Roth is a professor at UIUC. (Dan Roth, work-at, UIUC)
