1. Learning Objectives

By the end of this lesson, you will be able to:

• Understand the regulatory and business drivers for synthetic data in finance, including GDPR, CCPA, and privacy regulations.

• Implement differential privacy (DP) frameworks for financial data, including DP-SGD and the Gaussian mechanism.

• Apply generative models (VAEs, GANs, diffusion) for synthetic financial data generation with privacy guarantees.

• Design and evaluate privacy-preserving data sharing pipelines for fraud detection, credit scoring, and market risk modeling.

• Address the challenges of synthetic data utility, fidelity, and downstream task performance.

• Implement federated learning for privacy-preserving model training across financial institutions.

2. The Case for Synthetic Data in Finance

2.1 Regulatory Drivers

2.2 Business Drivers

• Data sharing: Financial institutions can share synthetic datasets for collaborative model development (e.g., fraud detection, credit scoring) without revealing customer information.

• Data augmentation: Synthetic data can augment limited datasets (e.g., for rare fraud types) to improve model performance.

• Testing and development: Developers can build and test models on realistic synthetic data without accessing production data.

• Scenario generation: Synthetic data can generate extreme but plausible market scenarios for stress testing.

3. Differential Privacy (DP) – The Mathematical Foundation

Differential privacy provides a rigorous mathematical guarantee that the output of a computation does not reveal too much about any individual in the dataset.

3.1 The Definition

A randomized mechanism M satisfies (ε, δ)-differential privacy if for any two datasets D and D’ that differ in at most one record (adjacent datasets), and for any output set S:

P(M(D) ∈ S) ≤ exp(ε) * P(M(D') ∈ S) + δ

where:

• ε (epsilon) is the privacy budget. Smaller ε means stronger privacy (but less utility).

• δ (delta) is a relaxation that allows a small probability of failure. Typically δ is very small (e.g., 1e-5).

When δ = 0, it is ε-differential privacy (pure DP).

3.2 The Privacy Budget

The privacy budget ε is a measure of how much privacy is sacrificed. Over multiple queries, the privacy budget accumulates. Composition allows us to calculate the total privacy loss over multiple queries.

Sequential composition: If we run k mechanisms, each with privacy budget ε_i, the total privacy budget is ∑ ε_i.

Parallel composition: If we run mechanisms on disjoint subsets of the data, the total privacy budget is max(ε_i) (no accumulation).

3.3 The Gaussian Mechanism

For a function f: D → ℝ^d with sensitivity Δ₂f = max_{D,D'} ||f(D) - f(D')||₂ (the maximum change in the output when one record is added or removed), the Gaussian mechanism adds noise:

M(D) = f(D) + N(0, σ² I)

where σ = (Δ₂f * sqrt(2 * ln(1.25/δ))) / ε for (ε, δ)-DP.

The Gaussian mechanism is the workhorse for DP in deep learning (DP-SGD).

3.4 DP-SGD

DP-SGD (Differentially Private Stochastic Gradient Descent) trains a model with privacy guarantees.

Algorithm:

1. Sample a batch of data points.

2. For each sample in the batch, compute the gradient.

3. Clip each gradient to a maximum norm C (to bound sensitivity):

   g_clipped = g * min(1, C / ||g||₂)

4. Compute the average of the clipped gradients.

5. Add Gaussian noise: g_noisy = (1/B) * ∑ g_clipped + N(0, (C * σ)² I)

6. Update the model: θ = θ - η * g_noisy

The privacy budget is calculated using a moments accountant to track the total privacy loss over training iterations.

4. Generative Models with Differential Privacy

4.1 DP-GANs

DP-GANs incorporate differential privacy into the GAN training process. The discriminator’s gradients are clipped and noised using DP-SGD. The generator is trained on the private data, but the privacy loss is accounted for.

Privacy guarantee: The generator is (ε, δ)-DP with respect to the training data.

Trade-off: Privacy comes at the cost of utility; generated samples may be less realistic.

4.2 DP-VAEs

VAEs can be made private by adding noise to the gradients of the encoder or by using a privacy-preserving ELBO. Alternatively, the latent variables can be privatized.

4.3 DP-Diffusion

Diffusion models can be trained with DP by applying DP-SGD to the denoising network. This is a recent and promising approach, as diffusion models produce higher-quality samples than GANs, though the training is more expensive.

4.4 Synthetic Data Generation Pipeline

1. Data pre-processing: Remove any direct identifiers (names, SSNs, account numbers).

2. Privacy budget allocation: Decide how much of the privacy budget to allocate to the generative model.

3. Model training: Train the generative model (GAN, VAE, or diffusion) with DP-SGD.

4. Generation: Generate a synthetic dataset of the desired size.

5. Validation: Validate the quality and privacy of the synthetic data.

5. Federated Learning (FL)

Federated learning enables multiple institutions to train a model collaboratively without sharing their raw data.

5.1 The Federated Averaging Algorithm (FedAvg)

1. Central server: Initializes a global model.

2. Each client (institution):
   a. Downloads the global model.
   b. Trains the model on its local data (using multiple epochs).
   c. Sends the updated model weights (or gradients) to the server.

3. Central server: Averages the updates from all clients:

   θ_global = (1/N) * ∑_{i=1}^{N} θ_i

4. Repeat until convergence.

5.2 Privacy in Federated Learning

• Secure aggregation: Use cryptographic techniques (e.g., secure multi-party computation) to ensure that the server cannot see individual client updates.

• Differential privacy: Add noise to the client updates before they are sent to the server (or to the aggregated update). This provides DP against the server or against other clients.

• Homomorphic encryption: Encrypt the gradients, allowing the server to average without decrypting.

5.3 Applications in Finance

• AML (Anti-Money Laundering): Multiple banks can train a fraud detection model together without sharing transaction data.

• Credit scoring: Multiple banks can train a credit scoring model on their joint data, improving accuracy.

• Market risk: Multiple banks can train a VaR model on their portfolio data.

Use case: Federated AML

Bank A → Local Model (on Bank A transactions)
Bank B → Local Model (on Bank B transactions)
Bank C → Local Model (on Bank C transactions)
        ↓
   Central Server
        ↓
   Global AML Model
        ↓
   Deployed to all banks

6. Evaluating Synthetic Data

6.1 Metrics for Synthetic Data Quality

6.2 Privacy Risk Metrics

• Membership inference: Can an adversary determine whether a specific record was in the training set? This is measured using an attack model trained to distinguish members from non-members.

• Attribute inference: Can an adversary infer a sensitive attribute of a record?

• Re-identification risk: Can an adversary link a synthetic record to a real individual?

6.3 The Utility-Privacy Trade-off

Increasing privacy (lower ε) generally decreases utility. The optimal ε depends on the use case and the acceptable level of risk.

Framework for choosing ε:

1. Determine the acceptable risk of re-identification.

2. Estimate the maximum utility required for the downstream task.

3. Find the ε that balances these constraints.

7. Limitations and Challenges

8. Summary for the AI Practitioner

• Synthetic data addresses privacy regulations and enables data sharing and augmentation in finance.

• Differential privacy provides a rigorous mathematical guarantee of privacy; the Gaussian mechanism is the workhorse.

• DP-GANs, DP-VAEs, and DP-diffusion models can generate private synthetic data.

• Federated learning enables collaborative model training without raw data sharing.

• Evaluate synthetic data on distributional similarity, utility, and privacy risk.

• The utility-privacy trade-off is a key consideration; choose ε based on the use case.