1. Learning Objectives

By the end of this lesson, you will be able to:

• Understand the transformer architecture and the scaling laws underlying LLMs (e.g., GPT, LLaMA, BloombergGPT).

• Apply fine-tuning techniques (full fine-tuning, LoRA, QLoRA) for financial domain adaptation.

• Implement instruction tuning and reinforcement learning from human feedback (RLHF) for financial applications.

• Design retrieval-augmented generation (RAG) pipelines for proprietary financial documents.

• Address the challenges of hallucinations, numerical inaccuracies, and data privacy in financial LLM applications.

• Evaluate LLM performance using domain-specific benchmarks (e.g., FinQA, BloombergGPT benchmarks).

2. The Transformer Architecture – A Mathematical Refresher

2.1 Self-Attention

The core of the transformer is the self-attention mechanism. Given a sequence of tokens, each with a d-dimensional embedding, we compute queries (Q), keys (K), and values (V) using learned linear projections:

Q = X W_Q, K = X W_K, V = X W_V

where X ∈ ℝ^(T×d), W_Q, W_K, W_V ∈ ℝ^(d×d_k).

The attention scores are:

S = softmax( (Q K^T) / sqrt(d_k) ) ∈ ℝ^(T×T)

The output is:

Z = S V ∈ ℝ^(T×d_k)

The scaling factor sqrt(d_k) prevents the dot products from becoming too large.

2.2 Multi-Head Attention

Instead of a single attention, we use h heads:

head_i = Attention(X W_Q^i, X W_K^i, X W_V^i)
MultiHead(X) = Concat(head_1, ..., head_h) W_O

This allows the model to attend to different parts of the input for different reasons.

2.3 Positional Encoding

Since self-attention is permutation-invariant, we add positional encodings to the embeddings:

PE(pos, 2i) = sin(pos / 10000^(2i/d))
PE(pos, 2i+1) = cos(pos / 10000^(2i/d))

This provides the model with information about the position of each token.

2.4 The Full Transformer Block

1. Multi-head attention with residual connection and layer norm:
   X' = LayerNorm(X + MultiHead(X))

2. Feed-forward network (FFN) with residual connection:
   X'' = LayerNorm(X' + FFN(X'))
   where FFN(x) = GELU(x W₁ + b₁) W₂ + b₂

2.5 Scaling Laws

Kaplan et al. (2020) showed that the test loss of a transformer scales as a power law with the model size, dataset size, and compute:

L ≈ C / N^α

where N is the number of parameters, α ≈ 0.076 for compute-optimal scaling. This has driven the trend towards larger models.

3. Pre-Trained LLMs for Finance

3.1 General-Purpose Models

• GPT-4 (OpenAI): ~1.7T parameters, trained on a diverse corpus including web pages, books, and code. Zero-shot performance on many financial tasks is impressive.

• Claude (Anthropic): Focused on safety and helpfulness.

• Gemini (Google): Multi-modal capabilities.

3.2 Domain-Specific Models

• BloombergGPT: 50B parameters, trained on 363 billion tokens of financial data (news, SEC filings, earnings calls) plus general data. Outperforms general models on financial benchmarks.

• FinGPT: An open-source initiative to create a financial LLM using instruction tuning on LLaMA-2.

• FinBERT: A BERT variant for finance (encoder-only, smaller).

• TradingGPT: A model fine-tuned for trading-related tasks.

3.3 Choosing a Model

4. Fine-Tuning LLMs for Financial Domain Adaptation

4.1 Full Fine-Tuning

Full fine-tuning updates all parameters of the model on a domain-specific dataset. This is expensive (requires significant GPU memory) and can lead to catastrophic forgetting (loss of general knowledge).

Procedure:

1. Load the pre-trained model.

2. Initialize a new head (e.g., for classification) or use the existing head.

3. Train on the financial dataset (e.g., sentiment-labeled news, QA pairs) using the same pre-training objective (next-token prediction) or a supervised objective.

4.2 Parameter-Efficient Fine-Tuning (PEFT)

PEFT methods update only a small number of parameters, reducing memory and compute requirements.

LoRA (Low-Rank Adaptation):
LoRA injects trainable rank decomposition matrices into the transformer layers. For a weight matrix W ∈ ℝ^(d×k), the updated weight is:

W' = W + ΔW = W + B A

where B ∈ ℝ^(d×r), A ∈ ℝ^(r×k), and r << min(d, k). During fine-tuning, W is frozen, and only A and B are updated. The rank r is typically 4-64.

The forward pass becomes:

h = W x + B A x

LoRA reduces the number of trainable parameters by a factor of ~10,000 for large models.

QLoRA: Quantized LoRA. The base model is quantized to 4-bit, and the LoRA adapters are trained. This allows fine-tuning of 70B models on a single GPU.

AdaLoRA: Adaptively allocates the rank to different layers based on importance.

Prefix-Tuning: Adds trainable tokens (prefix) to the input sequence. The rest of the model is frozen.

4.3 Instruction Tuning

Instruction tuning trains the model to follow instructions. The dataset consists of triples (instruction, input, output):

Instruction: “Summarize the following earnings call transcript.”
Input: (The transcript)
Output: (A summary)

The model is fine-tuned on this dataset using supervised learning. This improves zero-shot performance on new tasks.

Self-Instruct: Generate synthetic instruction data using the LLM itself, then fine-tune on it.

4.4 Reinforcement Learning from Human Feedback (RLHF)

RLHF aligns the model with human preferences. The process:

1. Supervised Fine-Tuning (SFT): Fine-tune on instruction data.

2. Reward Modeling: Train a reward model to predict human preferences. Given two outputs for the same prompt, the reward model learns to assign a higher score to the preferred output.

3. Proximal Policy Optimization (PPO): Fine-tune the SFT model to maximize the reward model’s score, with a KL penalty to prevent deviation from the SFT model.

5. Retrieval-Augmented Generation (RAG)

5.1 Why RAG?

LLMs have a fixed knowledge cutoff and cannot access proprietary or up-to-date data. RAG solves this by retrieving relevant documents and including them in the prompt.

5.2 RAG Pipeline

1. Indexing: Chunk documents (e.g., SEC filings, earnings calls) into passages. Embed each passage using a dense retriever (e.g., DPR, BGE) and store in a vector database (e.g., FAISS, Pinecone).

2. Query Encoding: Encode the user’s question using the same retriever.

3. Retrieval: Find the top-k most similar passages (cosine similarity).

4. Generation: Construct a prompt: “Context: {passage1} {passage2} … Question: {question} Answer:” and feed it to the LLM.

5.3 Advanced RAG Techniques

• Hybrid search: Combine dense retrieval (semantic) with sparse retrieval (BM25) to improve recall.

• Re-ranking: Use a cross-encoder to re-rank the retrieved passages.

• Multi-hop retrieval: Retrieve passages that are connected (e.g., company A acquired company B, retrieve both).

• Self-RAG: The model decides whether to retrieve and what to retrieve.

• Corrective RAG: The model critiques its own output and retrieves more if needed.

5.4 RAG for Financial Applications

• Earnings call QA: Retrieve the relevant part of the transcript to answer questions.

• Regulatory compliance: Retrieve the relevant regulation section.

• Portfolio analysis: Retrieve company-specific data.

6. Prompt Engineering for Financial LLMs

6.1 Structured Prompt Templates

For consistent outputs, use structured templates:

You are a financial analyst. Given the following data, answer the question.

Context: {context}
Question: {question}
Format your answer as:
- Answer: ...
- Confidence: High/Medium/Low
- Sources: {citations}

6.2 Chain-of-Thought (CoT) Prompting

For complex reasoning, ask the model to “think step by step”:

Question: "If the Fed raises rates by 25 basis points, what is the likely impact on a bank's net interest margin?"
Let's think step by step:
1. A 25bps rate hike increases the yield on earning assets.
2. However, funding costs may also rise.
3. The impact depends on the bank's interest rate sensitivity (gap).
4. If the bank is asset-sensitive (positive gap), NIM will increase.

6.3 Few-Shot Prompting

Provide a few examples of the task to improve performance:

Task: Classify the sentiment of financial news.
Example 1: "Apple reports record revenue." -> Positive
Example 2: "Oil prices plunge on demand fears." -> Negative
Example 3: "Central bank holds rates steady." -> Neutral
News: {news_text}
Sentiment:

6.4 Limiting Hallucinations

• Explicit instruction: “If you don’t know the answer, say ‘I don’t know’.”

• Provide citations: Ask the model to cite the source of its information.

• Post-processing validation: Use a separate model to verify factual claims.

7. Evaluating Financial LLMs

7.1 Benchmarks

• FinQA: Numerical reasoning over financial reports.

• ConvFinQA: Conversational QA with multi-step reasoning.

• BloombergGPT benchmarks: Sentiment, QA, summarization, and named entity recognition.

7.2 Metrics

• Accuracy/Exact Match: For QA and classification.

• ROUGE: For summarization.

• BERTScore: For semantic similarity.

• Hallucination Rate: Percentage of generated claims that are factually incorrect.

• Latency: Time to generate a response.

7.3 Human Evaluation

For subjective tasks (e.g., investment recommendations), human evaluation is essential. Use:

• Likert scale: Rate from 1 to 5 on accuracy, relevance, and coherence.

• A/B testing: Compare outputs from different models.

8. Deployment Considerations

• Model size: Smaller models (7B) can be deployed on-premises; larger models (70B) require specialized hardware.

• Quantization: 4-bit or 8-bit quantization reduces memory and latency.

• Caching: Cache common queries to reduce latency and cost.

• Monitoring: Track performance metrics (accuracy, latency) and retrain if needed.

9. Summary for the AI Practitioner

• LLMs are powerful for a wide range of financial NLP tasks but require careful tuning and augmentation.

• Fine-tuning with LoRA/QLoRA is cost-effective and prevents catastrophic forgetting.

• RAG is essential for incorporating proprietary and up-to-date data.

• Prompt engineering (CoT, few-shot) is a low-cost way to improve performance.

• Hallucinations are a major risk; use retrieval, explicit instructions, and post-processing to mitigate them.

• Evaluation should be done on domain-specific benchmarks and with human oversight.