1. Learning Objectives

By the end of this lesson, you will be able to:

  • Understand the evolution of LLMs (GPT, LLaMA, BloombergGPT) and their financial capabilities.

  • Design effective prompts for financial tasks (sentiment, summarization, QA, earnings translation).

  • Implement few‑shot and chain‑of‑thought prompting for multi‑step financial reasoning.

  • Fine‑tune a pre‑trained LLM on financial instruction‑following datasets.

  • Address hallucinations and numerical inaccuracies in LLM outputs.

  • Evaluate LLM performance using task‑specific metrics and human evaluation.

2. LLMs in Finance: Landscape

General‑purpose LLMs (GPT‑4, Claude, Gemini) have shown remarkable zero‑shot performance on many tasks. Domain‑specific models like BloombergGPT (50B parameters, trained on financial data) and FinBERT (encoder‑only, smaller) are also available.

Capabilities:

  • Sentiment analysis, summarization, QA, entity extraction (without fine‑tuning).

  • Complex reasoning (e.g., “If the Fed raises rates by 25bps, how might bank stocks react?”).

  • Generation of financial reports, emails, and presentations.

Limitations:

  • Hallucinations: generating plausible but incorrect numbers or facts.

  • Numerical reasoning: can be unreliable; better to offload arithmetic to tools.

  • Knowledge cutoff: may not know recent events; need retrieval augmentation.

3. Prompt Engineering Fundamentals

A prompt is the input text provided to an LLM. The quality of the output heavily depends on the prompt design.

Components:

  • Instruction: Clear task description (e.g., “Summarise the following earnings call transcript.”).

  • Context: Relevant background (e.g., company name, sector, reference period).

  • Input text: The document to process.

  • Output format specification: e.g., JSON, bullet points, tables.

  • Few‑shot examples: Provide a few input‑output pairs to guide the model.

3.1 Zero‑shot, Few‑shot, and Chain‑of‑Thought

  • Zero‑shot: No examples; relies solely on instruction.

  • Few‑shot: Give 2–5 examples of the task. This dramatically improves performance on structured outputs.

  • Chain‑of‑Thought (CoT): Ask the model to “think step‑by‑step” before giving the final answer. This is crucial for multi‑step arithmetic or reasoning.

    Example:
    Prompt: “A company’s revenue grew from $100M to $130M over two years. What is the annualised growth rate? Explain step by step.”
    Output: “Step 1: Growth factor = 130/100 = 1.3. Step 2: Annualised rate = (1.3)^(1/2) – 1 ≈ 0.1402, i.e., 14.02%.”

3.2 Prompt Templates for Financial Tasks

Sentiment analysis:

text
Classify the sentiment of the following financial news as 'positive', 'negative', or 'neutral'.
News: {news_text}
Sentiment:

Earnings summary:

text
Given the following earnings call transcript, produce a summary with the following sections:
- Revenue and EPS (beat/miss)
- Key drivers mentioned
- Guidance
- Risks mentioned
Transcript: {transcript}

Tool use: For calculations, prompt the model to output a Python expression, then execute it.

4. Fine‑tuning LLMs for Finance

When general models underperform, fine‑tune on a financial instruction dataset. Use a base model (e.g., LLaMA‑2, GPT‑NeoX) and a dataset of prompts and desired responses.

Instruction tuning: Format as (instruction, input, output) triples. Use supervised fine‑tuning (SFT) with cross‑entropy loss.

Parameter‑Efficient Fine‑tuning (PEFT):

  • LoRA (Low‑Rank Adaptation): Freeze base model weights and insert trainable low‑rank matrices. Reduces memory and time.

Example:

python
from peft import LoraConfig, get_peft_model, TaskType
lora_config = LoraConfig(
    r=8, lora_alpha=32, target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05, bias="none", task_type=TaskType.CAUSAL_LM
)
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")
model = get_peft_model(model, lora_config)
# Then train with a trainer.

Data sources: Fin‑instruction datasets like FinGPT‑instruct, or create your own from SEC filings and earnings calls with human‑written summaries.

5. Retrieval‑Augmented Generation (RAG)

To incorporate the latest data or specific documents, use RAG: retrieve relevant passages from a vector database (using DPR or BM25) and append them to the prompt.

RAG pipeline:

  1. Chunk documents (e.g., 10‑K sections) and embed them.

  2. For each query, retrieve top‑k chunks.

  3. Build a prompt: “Context: {chunk1} {chunk2} … Question: {user_question}”

  4. Generate answer with the LLM.

Benefits: Reduces hallucinations, enables citing sources.

6. Handling Numerical Accuracy

LLMs are not calculators. For reliable arithmetic:

  • Use tool‑calling: have the model generate a Python code snippet to compute the result, then execute it.

  • Alternatively, use chain‑of‑thought and encourage the model to express the equation.

  • For exact numbers, prefer retrieval over generation.

7. Evaluation of LLM Outputs

  • Automatic metrics: For summarization: ROUGE, BERTScore. For QA: EM, F1.

  • Factual consistency: Use a separate verification model (e.g., QAFactEval) or check numerical correctness via ground truth.

  • Human evaluation: For subjective tasks (e.g., investment recommendations), use expert raters on a Likert scale.

8. Deployment Considerations

  • Latency: LLMs are slow; consider smaller models for real‑time tasks.

  • Cost: API‑based models can be expensive; open‑source models may be deployed on‑premise.

  • Security: Financial data is sensitive; use local deployments or trusted cloud with encryption.

  • Versioning: LLMs evolve; ensure reproducible outputs by pinning model versions.

9. Summary for the AI Practitioner

  • LLMs offer powerful zero‑shot capabilities for many financial NLP tasks.

  • Prompt engineering (instruction, few‑shot, chain‑of‑thought) is the first lever to improve performance.

  • For domain‑specific needs, fine‑tune with LoRA on a financial instruction dataset.

  • RAG is essential for incorporating fresh or proprietary documents and reducing hallucinations.

  • Offload numerical computations to external tools to ensure accuracy.

  • Always evaluate with both automatic metrics and human checks, especially for high‑stakes decisions.

10. References

  1. Brown, T. B., et al. (2020). Language models are few‑shot learners. NeurIPS.

  2. Wu, S., et al. (2023). BloombergGPT: A large language model for finance. arXiv.

  3. Hu, E. J., et al. (2022). LoRA: Low‑rank adaptation of large language models. ICLR.

  4. Lewis, P., et al. (2020). Retrieval‑augmented generation for knowledge‑intensive NLP tasks. NeurIPS.

  5. Wei, J., et al. (2022). Chain‑of‑thought prompting elicits reasoning in large language models. NeurIPS.

  6. Zhang, T., et al. (2023). FinGPT: Instruction tuning for financial large language models. Workshop on Financial NLP.

 

PreviousNext