Retrieval-Augmented Generation: Advancing Language Models for Knowledge Extraction
Explore the innovative approach of Retrieval-Augmented Generation in language models, aiding knowledge extraction and information retrieval. Learn about Large Language Models and the challenges they face, along with the framework and process of Retrieval-Augmented Generation. Discover how this technology enhances user queries and generated text, offering real-time updates from external sources like Wikipedia.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Retrieval-Augmented Generation Qing Wang, Ph.D., Nov. 30, 2023
Large Language Model (Generation) LLMs don t store facts they store probability.
Large Language Model (Generation) LLMs user query = prompt x: how many species on land y: generated text (extracting knowledge from parameters) LLMs Challenges: 1. No source 2. Out of date Generated text
Retrieval-augmented Generation LLM + RAG x: how many species on land z: Wikipedia (be updated in real time) y: generated text
Retrieval-Augmented Generation 1. Preparation Embed Store Corpus Documents, PDFs Split Chunks size Using an embedding model to create vector representation Save each chunk and its embedding to DB 2. Retrieval Build Prompt User query Information from search Search User Query how many species on land? Embed Using the same embedding model 1. 2. Top k most relevant results on external sources LLM Generation Content Source from Entry Pointer AI
Retrieval-Augmented Generation Query --- ------ ------ LLM user --- Generated answer 1. Query 2. Retrieval information
Retrieval-augmented generation RAG framework: a pre-trained seq2seq model + a dense vector index of Wikipedia (accessed with a pre-trained neural retriever) The first author Patrick Lewis is from Facebook AI Research. RAG code in Hugging Face: https://huggingface.co/facebook/rag-token-nq
