Machine Translation Learning & Semantic Models - Insights and Applications

slide1 n.w
1 / 43
Embed
Share

Discover the world of machine translation learning, similarity measures, and deep semantic similarity models. Explore key concepts such as S2Net, DSSM, and relational similarities, along with practical applications in natural language processing and knowledge base embedding. Dive into the evolution of language processing technologies and the challenges of distinguishing synonyms and antonyms. Uncover how polarity is induced and cosine scores are utilized for target words, all while examining real-world examples and advancements in the field.

  • Machine Translation
  • Semantic Models
  • Natural Language Processing
  • Deep Learning
  • Similarity Measures

Uploaded on | 1 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.

E N D

Presentation Transcript


  1. ? = Search Machine Translation Learning Similarity Measures S2Net [CoNLL-11, SIGIR-11] DSSM [CIKM-13, ACL-14] DSSM Deep Structured Semantic Model, or more general, Deep Semantic Similarity Model

  2. ? = Search Machine Translation Learning Similarity Measures S2Net [CoNLL-11, SIGIR-11] DSSM [CIKM-13, ACL-14]

  3. ? = Search Machine Translation Learning Similarity Measures S2Net [CoNLL-11, SIGIR-11] DSSM [CIKM-13*, ACL-14] ? = : : ????( , ) Relational Similarity Multi-Relational LSA Word Analogy [NAACL-13 x2] Word Relation [EMNLP-12, EMNLP-13] Knowledge Base Embedding [EMNLP-14]

  4. Q: Who won the best actor Oscar in 1973? S1: Jack Lemmon was awarded the Best Actor Oscar for Save the Tiger (1973). S2: Academy award winner Kevin Spacey said that Jack Lemmon is remembered as always making time for others.

  5. sunny rainy cloudy windy car emotion cab sad wheel joy feeling

  6. Tomorrow will be rainy. Tomorrow will be sunny. ???????(rainy, sunny)? ???????(rainy, sunny)?

  7. Distinguishing synonyms and antonyms is still perceived as a difficult open problem. [Poon & Domingos 09]

  8. Target word: row-vector

  9. Inducing polarity Target word: row-vector Cosine Score: + ????????

  10. Inducing polarity Target word: row-vector Cosine Score: ????????

  11. ostrich is a bird engine is a part of car Idea #2: Encode multiple relations in a 3- way tensor (3-dim array)!

  12. joyfulness gladden joyfulness gladden sad sad anger anger Antonym layer Synonym layer Construct a tensor with two slices

  13. joyfulness gladden sad anger Hyponym layer

  14. ?1,?2,,?? ? ? ?1,?2, ,?? ?1,?2, ,?? ?1,?2, ,?? ? ? latent representation of words

  15. latent representation of a relation ?1,?2, ,?? ? ? ? ?1,?2, ,?? ?1,?2, ,?? ? ? ? ? latent representation of words

  16. ??? joy,sadden = cos ?:,joy,???,?:,sadden,??? joyfulness gladden joyfulness gladden sad sad anger anger Antonym layer Synonym layer

  17. ??? joy,sadden = cos ?:,joy,???,?:,sadden,??? joyfulness gladden joyfulness gladden sad sad anger anger Antonym layer Synonym layer

  18. ?,?:,:,?????,: ? ??? w?,w? = cos ?:,:,?????,: , ) Cos ( ? ?1,?2, ,?? ? ? ?? ?

  19. subj-pred-obj (?1,?,?2) Obama Born-in Hawaii Bill Gates Nationality USA Bill Clinton Satya Nadella Spouse-of Hillary Clinton Microsoft Work-at ?: # entities, ?: # relations

  20. ?-th slice k ?? e1 en Hawaii e1 en 1 Obama A 0 entry means: Incorrect (false) Unknown ??: born-in

  21. R? (?1,?,?2) ????1,??2 [Nickel+, ICML-11]

  22. 1 2 2+1 2+ 2 ?? ? ??? ?? ? ? ? 2 ? ? ?? ? ? ?? RESCAL [Nickel+, ICML-11]

  23. 1 2 ?? ??? 2 ? ?? ? locations ? ?? ? ? ?? Relation: born-in persons

  24. 1 2 ?? ??? 2 ? ?? ? ? ?? ? ? ??

  25. # Entities # Relation Types # Entity Types # Entity-Relation Triples 753k 229 300 1.8M

  26. (??,??,?) (??,?,??) ?? ?? ?? RESCAL TransE

  27. Dan Roth is a professor at UIUC. (Dan Roth, work-at, UIUC)

  28. Fig.1 of [Riedel+ 13]

More Related Content