Enhancing Turn Detection in Conversational AI Through Experimentation
Explore the journey of improving turn detection in conversational AI with experiments, data modifications, feature engineering tasks, and model evaluations. Results show significant accuracy improvements using various classification methods like BiLSTM with Attention. Further possibilities and challenges in utilizing multimodal representations and training chatbots are also discussed.
Uploaded on | 0 Views
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
TURN DETECTION IN CONVERSATIONAL AI Zahra Sayedi 97722104 Amin Pourdabiri 97722041
OPENSUBTITLE DATASET 3
OUR USED DATASET SwDA (Switchboard Dialog Act Corpus) Consists of THOUSANDS of telephone speech 4
DATASET MODIFICATION Changing caller values to match our problem Using only clean_text column for further processing Preprocessing texts to fit them with turn detection task! A little different with normal text preprocessing For example we don t delete punctuation marks like: ! ? . We need some postfixes or identifiers 5
AT THE BEGINNING Feature engineering tasks PoS-Tag Unigram Bigram BoW (Bag of Words) Feature selection Using one or more of these tasks Combining them based on our turn detection task (modified feature engineering) 6
OUR EXPERIMENTS At first we ran basic classification methods In this course: BERT LSTM BiLSTM BiLSTM with Attention Note that we used word embeddings like GloVe in addition to these methods 7
OUR RESULTS UP TO NOW The majority baseline was 52% It means that 52% of tags were Don t change or New All of the reported accuracies are based on SwDA dataset BERT: 64% LSTM: 70% BiLSTM: 83% BiLSTM with Attention: 86% 8
BESIDES We made a tagged dataset of OpenSubtitle They were about 1000 sentences We used them to evaluate our trained model of SwDA Our best model did not bad!!! The LSTM with attention model got about 70% accuracy It might overfit to SwDA (It might come in handy with bigger dataset!) 9
FURTHER MORE Using this model to tag OpenSubtitle dataset We may need multimodal representations to improve our model Use that dataset to train a chatbot Note that it may overfit to OpenSubtitle!!! And it may not! (Because of its conversation-based dataset) 10
THANK YOU FOR YOUR ATTENTION ANY QUESTIONS? 11
