Lesson 5.7: Graph Neural Networks for Financial Networks

1. Learning Objectives

By the end of this lesson, you will be able to:

Understand the mathematical foundations of graph theory and its applications in finance.
Define and construct graph representations of financial systems (interbank networks, correlation networks, supply chains).
Understand the mathematical formulation of Graph Convolutional Networks (GCNs) and Graph Attention Networks (GATs).
Implement GCNs for systemic risk detection and asset classification.
Apply Graph Neural Networks to portfolio diversification and risk factor identification.
Understand the message passing framework and its implementation in PyTorch Geometric.
Apply temporal graph networks for dynamic financial networks.
Understand the limitations and challenges of GNNs in finance.

2. Graph Theory Foundations for Finance

2.1 Graph Definition
A graph G = (V, E) consists of:

V: Set of vertices (nodes) – e.g., banks, stocks, companies.
E: Set of edges (connections) – e.g., lending relationships, correlations, supply chain links.

2.2 Graph Representations

Adjacency Matrix: A ∈ R^{N x N}, where A_{ij} = 1 if there is an edge between nodes i and j.

Degree Matrix: D ∈ R^{N x N}, diagonal matrix with D_{ii} = Σ_j A_{ij}.

Laplacian Matrix: L = D - A. The Laplacian spectrum encodes graph properties.

Normalised Laplacian: L_sym = D^{-1/2} L D^{-1/2}.

2.3 Financial Graph Construction

Correlation Networks:

Nodes: Assets (stocks, bonds, currencies).
Edges: Correlation between asset returns.
Edge Weight: w_{ij} = ρ_{ij} (correlation coefficient).

Thresholding: A_{ij} = 1 if |ρ_{ij}| > τ.

Interbank Networks:

Nodes: Banks.
Edges: Lending relationships.
Edge Weight: Amount of lending.

Supply Chain Networks:

Nodes: Companies.
Edges: Supplier-customer relationships.
Edge Weight: Transaction volume.

3. Graph Convolutional Networks (GCNs)

3.1 Mathematical Formulation

The GCN layer performs a graph convolution operation:
H^{(l+1)} = σ( \tilde{D}^{-1/2} \tilde{A} \tilde{D}^{-1/2} H^{(l)} W^{(l)} )

Where:

\tilde{A} = A + I (self-loops added).
\tilde{D}_{ii} = Σ_j \tilde{A}_{ij}.
H^{(l)} is the node feature matrix at layer l.
W^{(l)} is the learnable weight matrix.
σ is the activation function.

3.2 Derivation – Graph Convolution as Aggregation

The convolution operation can be interpreted as:

Aggregation: \hat{H}^{(l)} = \tilde{D}^{-1/2} \tilde{A} \tilde{D}^{-1/2} H^{(l)}
Transformation: H^{(l+1)} = σ(\hat{H}^{(l)} W^{(l)})

Each node aggregates information from its neighbours, weighted by degrees.

3.3 Implementation

import torch
import torch.nn as nn
import torch.nn.functional as F

class GCNLayer(nn.Module):
    def __init__(self, in_features, out_features, activation=F.relu):
        super(GCNLayer, self).__init__()
        self.linear = nn.Linear(in_features, out_features)
        self.activation = activation

    def forward(self, H, A):
        # Normalised adjacency matrix: D^{-1/2} A D^{-1/2}
        D = torch.diag(torch.pow(A.sum(dim=1), -0.5))
        A_norm = D @ A @ D

        # Graph convolution
        H = A_norm @ H
        H = self.linear(H)

        if self.activation is not None:
            H = self.activation(H)

        return H

class GCN(nn.Module):
    def __init__(self, n_features, n_hidden, n_classes, n_layers=2, dropout=0.3):
        super(GCN, self).__init__()

        self.layers = nn.ModuleList()
        self.layers.append(GCNLayer(n_features, n_hidden))

        for _ in range(n_layers - 2):
            self.layers.append(GCNLayer(n_hidden, n_hidden))

        self.layers.append(GCNLayer(n_hidden, n_classes, activation=None))
        self.dropout = nn.Dropout(dropout)

    def forward(self, H, A):
        for i, layer in enumerate(self.layers):
            H = layer(H, A)
            if i < len(self.layers) - 1:
                H = self.dropout(H)

        return F.log_softmax(H, dim=1)

4. Graph Attention Networks (GATs)

4.1 Attention Mechanism for Graphs

GATs use attention to assign different weights to different neighbours.

e_{ij} = LeakyReLU(a^T [W h_i || W h_j])
α_{ij} = softmax_j(e_{ij}) = exp(e_{ij}) / Σ_{k ∈ N(i)} exp(e_{ik})
h'_i = σ( Σ_{j ∈ N(i)} α_{ij} W h_j )

4.2 Multi-Head Attention
h'_i = ||_{k=1}^{K} σ( Σ_{j ∈ N(i)} α_{ij}^k W^k h_j )

4.3 Implementation

class GATLayer(nn.Module):
    def __init__(self, in_features, out_features, n_heads=1, alpha=0.2, dropout=0.3):
        super(GATLayer, self).__init__()

        self.n_heads = n_heads
        self.out_features = out_features
        self.alpha = alpha
        self.dropout = nn.Dropout(dropout)

        # Weight matrices for each head
        self.W = nn.Parameter(torch.zeros(n_heads, in_features, out_features))
        self.a = nn.Parameter(torch.zeros(n_heads, 2 * out_features, 1))

        nn.init.xavier_uniform_(self.W)
        nn.init.xavier_uniform_(self.a)

    def forward(self, H, A):
        # H: (N, in_features)
        # A: (N, N)

        N = H.size(0)
        outputs = []

        for head in range(self.n_heads):
            # Linear transformation
            W = self.W[head]  # (in_features, out_features)
            h = H @ W  # (N, out_features)

            # Attention coefficients
            h_concat = torch.cat([h.unsqueeze(1).repeat(1, N, 1),
                                  h.unsqueeze(0).repeat(N, 1, 1)], dim=-1)  # (N, N, 2*out_features)

            a = self.a[head]  # (2*out_features, 1)
            e = torch.matmul(h_concat, a).squeeze(-1)  # (N, N)

            # Leaky ReLU
            e = F.leaky_relu(e, self.alpha)

            # Mask (only consider neighbours)
            e = e * A
            e = e + (-1e9) * (1 - A)

            # Softmax
            alpha = F.softmax(e, dim=1)
            alpha = self.dropout(alpha)

            # Aggregation
            h_out = alpha @ h  # (N, out_features)
            outputs.append(h_out)

        # Concatenate or average heads
        if self.n_heads > 1:
            h_out = torch.cat(outputs, dim=1)
        else:
            h_out = outputs[0]

        return h_out

5. Financial Applications of GNNs

5.1 Systemic Risk Detection in Interbank Networks

class SystemicRiskGCN(GCN):
    def __init__(self, n_features, n_hidden, n_classes=2):
        super(SystemicRiskGCN, self).__init__(n_features, n_hidden, n_classes)

    def predict_systemic_risk(self, bank_features, adjacency_matrix):
        """
        Predict which banks are systemically important.
        """
        # Forward pass
        log_probs = self.forward(bank_features, adjacency_matrix)

        # Probability of being systemically important
        probs = torch.exp(log_probs)
        systemic_prob = probs[:, 1]  # Class 1: Systemically important

        return systemic_prob

# Example use
def build_interbank_network(lending_matrix):
    """
    Build adjacency matrix from interbank lending data.
    """
    # lending_matrix: (N, N) with lending amounts
    # Threshold to create binary adjacency
    threshold = 1e6  # $1 million
    adjacency = (lending_matrix > threshold).float()

    # Add self-loops
    adjacency = adjacency + torch.eye(len(adjacency))

    return adjacency

5.2 Asset Classification and Sector Identification

class AssetGraphNetwork(GAT):
    def __init__(self, n_features, n_hidden, n_classes, n_heads=4):
        super(AssetGraphNetwork, self).__init__(n_features, n_hidden, n_classes, n_heads)

    def classify_assets(self, asset_features, correlation_matrix):
        """
        Classify assets into sectors using correlation graph.
        """
        # Threshold correlation matrix
        threshold = 0.5
        adjacency = (torch.abs(correlation_matrix) > threshold).float()
        adjacency = adjacency + torch.eye(len(adjacency))

        # Forward pass
        log_probs = self.forward(asset_features, adjacency)
        predictions = torch.argmax(log_probs, dim=1)

        return predictions

5.3 Portfolio Diversification with GNNs

class PortfolioGNN(nn.Module):
    def __init__(self, n_assets, n_features, n_hidden, latent_dim=16):
        super(PortfolioGNN, self).__init__()

        # GCN for asset embeddings
        self.gcn = GCN(n_features, n_hidden, latent_dim)

        # Portfolio allocation head
        self.fc = nn.Linear(latent_dim, 1)

    def forward(self, asset_features, adjacency_matrix):
        # Get asset embeddings
        embeddings = self.gcn(asset_features, adjacency_matrix)

        # Portfolio weights (softmax across assets)
        weights = self.fc(embeddings)
        weights = F.softmax(weights.squeeze(-1), dim=0)

        return weights

6. Temporal Graph Networks (TGNs)

6.1 Dynamic Graph Representation

Nodes and edges change over time.
Temporal edges: (u, v, t) with timestamp t.
Node features evolve over time.

6.2 Memory Module

Each node has a memory state s_u(t) that evolves over time.

6.3 Implementation (Simplified)

class TemporalGraphNetwork(nn.Module):
    def __init__(self, n_features, n_hidden, n_classes, memory_dim=64):
        super(TemporalGraphNetwork, self).__init__()

        self.memory_dim = memory_dim
        self.memory = nn.Parameter(torch.zeros(n_features, memory_dim))

        # GNN for message passing
        self.gnn = GCN(memory_dim, n_hidden, n_classes)

    def update_memory(self, events):
        """
        Update node memories based on events.
        """
        # events: list of (u, v, t, features)
        for u, v, t, feat in events:
            # Compute message from v to u
            message = self.compute_message(self.memory[v], feat)
            self.memory[u] = self.update_node(self.memory[u], message)

    def compute_message(self, memory_v, event_feat):
        return torch.cat([memory_v, event_feat])

    def update_node(self, memory_u, message):
        return 0.9 * memory_u + 0.1 * message

    def forward(self, adjacency, node_features):
        # Update memory based on recent events
        # (in practice, this is done during training)

        # GNN on current graph
        return self.gnn(self.memory, adjacency)

7. Graph Neural Networks for Financial Fraud Detection

class FraudDetectionGNN(GCN):
    def __init__(self, n_features, n_hidden, n_classes=2):
        super(FraudDetectionGNN, self).__init__(n_features, n_hidden, n_classes)

    def detect_fraud(self, transaction_features, transaction_graph):
        """
        Detect fraudulent transactions using transaction graph.
        """
        # Transaction graph: nodes are accounts, edges are transactions
        # Features: transaction amount, time, frequency, etc.

        log_probs = self.forward(transaction_features, transaction_graph)
        fraud_prob = torch.exp(log_probs)[:, 1]

        return fraud_prob

8. Summary for the AI Practitioner

Graphs represent relationships in financial systems (interbank networks, correlation networks, supply chains).
GCNs perform convolution on graphs: H^{(l+1)} = σ( \tilde{D}^{-1/2} \tilde{A} \tilde{D}^{-1/2} H^{(l)} W^{(l)} ).
GATs use attention to weight neighbours differently: α_{ij} = softmax(LeakyReLU(a^T [W h_i || W h_j])).
Systemic risk detection identifies systemically important institutions in financial networks.
Temporal Graph Networks handle dynamic graphs where nodes and edges change over time.
Fraud detection identifies suspicious transactions in transaction networks.
PyTorch Geometric is the standard library for GNN implementation.
Challenges: Scalability to large graphs, temporal dynamics, and interpretability.

Diploma in Artificial Intelligence in Finance Lesson 5.7: Graph Neural Networks for Financial Networks

Diploma in Artificial Intelligence in Finance