Unveiling the Power of Retrieval Augmented Generation and Semantic Search in AI
Dive into the world of Retrieval Augmented Generation (RAG) and Semantic Search to explore word embeddings, fine-tuning techniques, and utilizing gigantic datasets. Understand key differences, the significance of word embeddings, and the potential of Vector Databases in AI applications. Discover the essence of Semantic Search and its applications in Natural Language data processing. Access valuable links to enhance your knowledge in this domain.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Retrieval Augmented Generation (RAG) Ashish Senior Research Data Scientist Rosen Center of Advanced Computing (RCAC)
Agenda 01 02 03 04 Explore Word embeddings Overview of Vector Database Semantic Search RAG Framework
Fine Tuning Query Pre-training Finetuning { } User Response Base LLM Pretend LLM Gigantic dataset Org/Domain Specific dataset
Retrieval-Augmented Generation (RAG) Pre-training Query { } Gigantic dataset Base LLM QA System User Response Search Query + relevant docs Org/Domain Specific dataset
Key differences Knowledge integration Adaptability Inference process Resource requirements Transparency and Explainability Application suitability
Word Embeddings Numerical Representation Semantic Similarity Machine Learning Integration Dimensionality reduction Context Sensitivity Applications
Vector Database (Chroma DB) AI-Focused Database Easy to Use Handles Complex Data Automatically Advance Search Features Flexible with Data Community-Driven and Open Source
Semantic Search Understanding Semantic Search Use of Natural Language data Vectorization of Text Creating a searchable collection Querying the collection Retrieval of the results
Links https://github.com/chroma-core/chroma (Chroma DB) https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2 (Sentence Transformer) https://www.kaggle.com/datasets/unanimad/the-oscar-award/data (Dataset)
