Developing Visualization Tool for Interpretability in Word Embeddings
"Explore the development of a graphical user interface to enhance interpretability in word embeddings and language models. Understand how large language models work, address the issue of non-interpretable dimensions in word embedding vectors, and introduce the SensePOLAR framework for adding interpretability to pretrained word embeddings. Join the project team to delve into Natural Language Processing and Machine Learning with practical coding work and interactive discussions. Contribute to advancing trust, bias prevention, and error reduction in language model predictions."
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Decoding Word Embeddings: Developing a Visualization Tool for Interpretability Team Project FSS 2023 Marlene Lutz Chair for Data Science in the Economic and Social Sciences
Motivation Language models (LM) are getting better and larger ChatGPT: 175B parameters MT-NLG: 530B parameters LMs are Black-Boxes What do these models learn? How and why do they make a particular prediction? Understanding their behaviour is critical Increase trust in predictions Avoid biases Prevent errors https://chat.openai.com/chat 14.02.2023 2
Word embeddings Large language models represent words as high dimensional vectors that are learnt from huge amounts of training data ? Ocean waves are crashing on the rocks Learn embedding for wave ? ? Problem: dimensions are not interpretable for humans! 14.02.2023 3
SensePOLAR Framework Adding interpretability to pretrained word embeddings SensePOLAR Transformation Interpretable dimensions as scales between polar opposites (e.g. good bad) Engler et al. 2022 14.02.2023 4
Goals Developing a Graphical User Interface for SensePOLAR Visually appealing and user-friendly handling of the framework Basic functionalities Visualizing the most descriptive dimensions for a word Geting the score of a word on a particular dimension Displaying on which dimensions two words differ the most Extending the current range of functions Integration of user defined dimensions Interactive evaluation of interpretable dimensions Opportunity to contribute your own ideas! 14.02.2023 5
Logistics Language: English Duration: 6 months Participants: 3 6 Prerequisites: Foundations in Natural Language Processing and/or Machine Learning Programming skills (esp. Python) Work Process A lot of coding Regular meetings to present and discuss your progress Presentation of final result in an endterm presentation session Document and submit your work as a written report 14.02.2023 6
