Lesson 9.3: LLM Applications in Finance – Sentiment, Summarization, and Document QA

1. Learning Objectives

By the end of this lesson, you will be able to:

Design and implement LLM-based sentiment analysis systems for financial news, social media, and earnings calls.
Build automated summarization pipelines for financial documents (10-Ks, earnings transcripts, research reports).
Construct document-level question-answering systems for large financial corpora using RAG and LLMs.
Implement multi-document aggregation for investment research and due diligence.
Apply prompt engineering and fine-tuning to optimize LLM performance for each specific task.
Evaluate LLM-based applications using domain-specific metrics and human validation.

2. LLM-Based Sentiment Analysis

2.1 The Evolution from Lexicons to LLMs

Traditional sentiment analysis in finance relied on lexicons (Loughran-McDonald) or fine-tuned BERT models (FinBERT). LLMs offer several advantages:

Contextual understanding: LLMs capture nuance, sarcasm, and negation more effectively.
Zero-shot capability: No need for labeled data for many tasks.
Explainability: LLMs can provide reasoning for their sentiment classification.
Multi-dimensional sentiment: LLMs can classify along multiple axes (e.g., sentiment, uncertainty, urgency).

2.2 Prompt Design for Sentiment Analysis

Basic classification prompt:

Classify the sentiment of the following financial news as 'positive', 'negative', or 'neutral'.
Provide a brief justification.

News: "{news_text}"
Sentiment:
Justification:

Fine-grained sentiment prompt:

Analyze the sentiment of the following text on a scale from -5 (extremely negative) to +5 (extremely positive).
Provide specific evidence from the text to support your score.

Text: "{text}"
Score:
Evidence:

Multi-dimensional prompt:

Analyze the following text along the following dimensions:
- Sentiment (Positive/Negative/Neutral)
- Uncertainty (High/Medium/Low)
- Urgency (High/Medium/Low)
- Forward-looking (Yes/No)

Text: "{text}"
Output (JSON):

2.3 Earnings Call Sentiment Analysis

Earnings calls contain valuable forward-looking statements. LLMs can extract:

Management tone: Confidence, optimism, or defensiveness in the prepared remarks and Q&A.
Guidance sentiment: Whether the guidance is positive, negative, or cautious.
Analyst sentiment: The tone of analysts’ questions.

Prompt example:

You are a financial analyst analyzing an earnings call transcript.
Extract the following information:
1. Overall management tone (optimistic/cautious/defensive)
2. Key positive statements (3-5)
3. Key negative statements (3-5)
4. Sentiment on future guidance (positive/negative/neutral)

Transcript excerpt: "{transcript_excerpt}"
Output:

2.4 Social Media Sentiment Aggregation

Social media (Twitter, Reddit) provides real-time sentiment signals. LLMs can:

Classify individual posts at scale.
Aggregate sentiment across a corpus using a sliding window.
Detect sentiment spikes that may indicate market-moving events.

Aggregation prompt:

You are given a collection of 100 social media posts about a company.
Summarize the overall sentiment, key themes, and any notable sentiment shifts.

Posts: {posts}
Summary:

3. Financial Document Summarization

3.1 Types of Summarization

Type	Description	Example
Extractive	Selects important sentences from the document.	10-K summary with key financial figures.
Abstractive	Generates new sentences that capture the essence.	Earnings call highlights in a narrative form.
Query-focused	Summarizes based on a specific query.	“Summarize the risk factors for this company.”
Structured	Generates a summary in a specific format (e.g., table).	Financial metrics in a table.

3.2 Prompt Design for Summarization

Basic summarization:

Summarize the following earnings call transcript in 3-5 bullet points.
Focus on: revenue, earnings per share, guidance, and key drivers.

Transcript: "{transcript}"
Summary:

Structured summarization:

Extract the following information from the 10-K filing and format as JSON:
{
  "company": "",
  "fiscal_year": "",
  "total_revenue": "",
  "net_income": "",
  "eps": "",
  "assets": "",
  "liabilities": "",
  "equity": "",
  "cash_flow_operating": "",
  "risk_factors": ["", "", ""],
  "business_overview": ""
}

Query-focused summarization:

Given the following document, provide a summary that specifically addresses:
"What are the main risks facing the company's supply chain?"

Document: "{document}"
Summary:

3.3 Summarization for Investment Research

For investment research, we may need to summarize multiple documents:

Multi-document summarization:

You have the following documents:
1. {doc_1_title}: {doc_1_content}
2. {doc_2_title}: {doc_2_content}
3. {doc_3_title}: {doc_3_content}

Provide a single, coherent summary that synthesizes the key information from all documents.
Focus on: company performance, competitive position, and future outlook.

Synthesis:

3.4 Handling Long Documents

LLMs have context window limits (e.g., 4K, 8K, or 128K tokens for some models). For long documents (e.g., a 10-K with hundreds of pages):

Chunking: Split the document into chunks and summarize each chunk, then summarize the chunk summaries.
Map-Reduce: For each chunk, generate a summary (map). Then, summarize all the summaries (reduce).
Refine: For each chunk, refine the summary by updating it with information from the new chunk.

Map-Reduce prompt:

Summarize the following chunk of text in 2-3 sentences:

Chunk: "{chunk}"
Summary:

Synthesize the following chunk summaries into a single, coherent summary of the entire document:

Chunks: {summaries}
Synthesis:

4. Document-Level Question Answering (QA)

4.1 RAG-Based QA Architecture

As introduced in Lesson 9.2, RAG is the standard approach for document-level QA. The pipeline:

Document ingestion: Chunk the document into passages (e.g., 500-1000 tokens).
Indexing: Embed each passage using a dense retriever (e.g., DPR, BGE).
Query processing: Embed the user’s question.
Retrieval: Find the top-k most similar passages using cosine similarity.
Context construction: Combine the retrieved passages into a prompt.
Generation: Feed the prompt to the LLM to generate the answer.

4.2 Advanced RAG for Finance

Hybrid retrieval: Combine dense retrieval (semantic similarity) with sparse retrieval (BM25, TF-IDF) to improve recall. This is important because financial questions often require exact numeric matches.

Re-ranking: After initial retrieval, use a cross-encoder to re-rank the passages based on relevance. This improves the precision of the top-k results.

Query expansion: Expand the user’s question with synonyms or related terms to improve retrieval. For example, “earnings” might expand to “net income”, “profit”, “bottom line”.

Time-aware retrieval: When retrieving from financial documents (which are time-sensitive), ensure that the retrieved passages are relevant to the query’s time context. For example, if the question asks about “last year’s revenue”, passages from the correct fiscal year should be prioritized.

4.3 Handling Numerical Questions

Financial QA often requires arithmetic. The LLM can be prompted to produce a calculation:

Prompt:

Given the following information, answer the question.
If the question requires calculation, show your work.

Information: {retrieved_passages}
Question: "{question}"
Answer:

Chain-of-Thought for arithmetic:

Question: "What was the total revenue growth from 2022 to 2023?"
Let's think step by step:
1. I'll look for revenue figures for each year.
2. I'll calculate the difference and the percentage change.
3. I'll provide the answer.

Information: {retrieved_passages}

Output:
“2022 revenue: $100 million. 2023 revenue: $120 million.
Growth = ($120M – $100M) / $100M = 20%.
Answer: Revenue grew by $20 million, a 20% increase.”

4.4 Table Extraction and QA

Financial documents contain many tables. Approaches:

Table parsing: Use a table parser to extract the table structure (rows, columns, headers).
Conversion to text: Convert the table to a textual format (e.g., JSON, CSV, or a structured description).
Table-aware prompt: Include the table in the prompt and ask the model to answer questions about it.

Prompt:

The following table contains financial data:

{table_as_text}

Question: "{question}"
Answer:

Chain-of-Thought for table-based reasoning:

Question: "What is the sum of SG&A and R&D expenses?"
Table: {table_as_text}
Let's think step by step:
1. Find SG&A expenses.
2. Find R&D expenses.
3. Add them together.
Answer:

4.5 Handling Multi-Hop Questions

Some questions require combining information from multiple passages or documents:

Multi-hop reasoning prompt:

I'll answer the following question by breaking it down into steps:
Question: "{question}"
Step 1: Identify the relevant information.
Step 2: Answer sub-question 1.
Step 3: Answer sub-question 2.
Step 4: Synthesize the final answer.

Information: {retrieved_passages}

5. Multi-Document Aggregation for Investment Research

Investment research often requires analyzing multiple documents (e.g., 10-K, earnings calls, analyst reports, news). LLMs can synthesize this information.