Finetuning Open Source Models for Specific Domains
Learn about the power and limitations of Large Language Models (LLMs) and how finetuning can enhance their performance for specialized tasks in domains like healthcare, finance, and law. Explore different finetuning methods like Transfer Learning and Instruction Tuning, as well as the research objectives of FinGPT in the finance domain with sentiment analysis datasets.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Finetuning of open source LLMs for specific domains Stavropoulos Christos M.Sc. In Data Science
Introduction The Power and Limits of LLMs 01 02 03 04 Problems with certain domains Problem What is an LLM? LLMs solve tasks like: Advanced AI models Understand and generate text Trained on vast datasets Enormous number of parameters Text classification Question answering Document summarization Text generation Cannot retrain from scratch Healthcare Finance Law Extensive resources required Specialized language and terminologies
Introduction Finetuning to the rescue Adapting a pre-trained LLM to a specific task or domain How finetuning works: Builds upon existing knowledge of the LLM Enhances model performance for specialized tasks Advantages of finetuning: Specialization for specific domains Reduced training data required
Introduction Finetuning methods Transfer Learning Adapts existing learning to new tasks or domains Example: dataset of movie reviews with sentiment labels (positive/negative). Instruction Tuning Adjusts model behavior based on explicit instructions Tailors model outputs to meet specific requirements Alignment-tuning Aligns model representations with target task objectives Enhances model performance through alignment adjustments
Research Objectives Research Objectives FinGPT Setting up and experimenting on FinGPT HR Dataset Creation Find and tune dataset dataset to HR issues Finetuning in HR Integrate the approach of FinGPT to the HR related dataset
The FinGPT Approach Summary: Open-source, FinLLM Goal: Create a finetuned LLM tailored to the finance domain, Make FinLLMs more accessible Replace proprietary models like BloombergGPT Domain Specific Task: Sentiment analysis Finetuning Technique: Instruction Tuning
Data () Sentiment Analysis Finance Specialized Datasets: TFNS: FiQA SA: Finance-related tweets Training set: 9,938 instances Test set: 2,486 instances Financial Information Quality Assessment 2018 Training set: 1,063 instances Test set: 152 instances
Data (2/2) FPB: NWGI: Financial news articles Total: 3,872 sentences Test: 968 instances News With GPT Instruction Generated labels by ChatGPT Training set: 16,200 instances Test set: 4,052 instances
Prepare Dataset Label Transformation: Convert labels to the corresponding sentiment negative , positive or neutral . Instruction: Add instruction for task specific fine-tuning guidance/Instruction Tuning
Prepare Dataset By taking a sentence from our dataset: Instruction: "What is the sentiment of this tweet? Please choose an answer from {negative/neutral/positive}." Input: "$FTI - TechnipFMC downgraded at Berenberg but called Top Pick at Deutsche Bank https://t.co/XKcPDilIuU" Output: "negative"
LoRA Parameter Efficient Finetuning Enhances fine-tuning process of LLMs, by adapting based on: Model's architecture and the task at hand. Parameters to low rank representations Regularization, prevent overfitting Adaptability to task complexity Select components or layers to train
Quantization Parameter Efficient Finetuning Quantization is applied to reduce: memory usage computational requirements Quantizing the model weights and activations separately, can enhance model performance
QLoRA Parameter Efficient Finetuning A combination of LoRA and Quantization: Reduces computational overhead during finetuning processes High performance and resource efficiency in various applications Enables LLMs integration in real-world systems
Training Arguments Training Epochs: 2 Batch Size: 2 (initially 4) Learning Rate: 1e-4 FP16 enabled: faster training, lower memory (float s half memory)
Models Examine the performance of the following models Pretrained: 1. 2. Finetuned: a. b. c. FinGPT v3.3 - Llama2-13b ChatGLM2 Mistral-7b ChatGLM2 trained on TFNS Mistral-7b trained on TFNS Mistral-7b trained on all datasets
Conclusions Finetuning Improves performance Our Model Mistral Finetuned outperforms any other model
Extending Beyond Finance Detecting Bias in Job Descriptions Privacy sensitive domain Job Descriptions Dataset from Kaggle Task: Categorize Job Descriptions as biased or unbiased Ground truth: ChatGPT generated labels 2400 training set instances 600 test set instances Biased if implies racial or gender discrimination Similar Approach Instruction Tuning Is the following job description biased? Please choose an answer from {biased/unbiased} Mistral-7b which outperformed all other models QLoRA applied
Conclusions Finetuning improves performance
Discussion & Conclusions Finetuning Success Significant performance improvements in financial and HR tasks Bias Detection Promise The experiment showcases potential of finetuning for ethical applications Dataset Impact Using a dataset annotated by humans would be a better option
