1. Learning Objectives

By the end of this lesson, you will be able to:

• Understand the ethical principles for AI in finance: fairness, accountability, transparency, privacy, and robustness.

• Identify and mitigate sources of bias in financial AI models (data bias, algorithmic bias, societal bias).

• Apply fairness metrics and mitigation techniques to credit scoring, hiring, and lending models.

• Design explainable AI systems with regulatory-grade interpretability.

• Address data privacy and security concerns in financial AI.

• Implement responsible AI governance frameworks.

2. Ethical Principles for AI in Finance

2.1 OECD Principles on AI

The OECD has established five key principles for responsible AI:

1. Inclusive growth and sustainable development: AI should benefit people and the planet.

2. Human-centred values and fairness: AI should be designed to respect human rights and democratic values.

3. Transparency and explainability: AI should be transparent and explainable.

4. Robustness and safety: AI should be robust, secure, and safe.

5. Accountability: AI actors should be accountable for their systems.

2.2 Financial-Specific Ethical Concerns

2.3 The Fairness-Accuracy Trade-off

There is often a trade-off between fairness and accuracy. For example, removing a protected attribute (e.g., race) may reduce accuracy if it is correlated with other predictive features. However, removing it may improve fairness.

The mathematical formulation:

• Accuracy: Acc = (TP + TN) / (TP + TN + FP + FN)

• Fairness: Various metrics (see below).

The goal is to find a model that achieves both high accuracy and high fairness. This is a multi-objective optimization problem:

min_{θ} L(θ) + λ * F(θ)

where L(θ) is the loss (error), F(θ) is a fairness penalty, and λ controls the trade-off.

3. Sources of Bias in Financial AI

3.1 Types of Bias

3.2 Proxy Variables

Protected attributes (race, gender, age) are often not directly used in models. However, proxy variables may encode them indirectly.

Common proxies:

• ZIP code (for race)

• Income (for race)

• Education (for race)

• Occupation (for gender)

• Age (for everything)

Mathematical detection: We can test whether a model’s predictions are independent of a protected attribute, even when the protected attribute is not included as a feature.

P(Y = 1 | X) = P(Y = 1 | X, A) for all X

If this condition is violated, the model is biased.

3.3 Historical Bias

Historical data often reflects past discrimination. If a model is trained on historical data, it will learn and perpetuate these patterns.

Example: If historical loan data shows that a certain group was charged higher rates (due to past discrimination), a model trained on this data will learn to charge that group higher rates.

Mitigation:

• Debiasing: Adjust the training data to reduce historical bias (e.g., through reweighting or resampling).

• Fairness constraints: Add constraints to the model to ensure fair outcomes.

• Disparate impact analysis: Test the model for disparate impact.

4. Fairness Metrics and Mitigation

4.1 Group Fairness Metrics

Group fairness metrics compare the outcomes of protected groups.

Formal definitions:

• Disparate impact: DI = P(Y=1 | A=0) / P(Y=1 | A=1) where A=0 is the protected group and A=1 is the non-protected group. DI < 0.8 indicates adverse impact.

• Equal opportunity: TPR_A0 - TPR_A1 = 0 where TPR = TP / (TP + FN).

• Equal odds: |TPR_A0 - TPR_A1| + |FPR_A0 - FPR_A1| = 0.

4.2 Individual Fairness Metrics

Individual fairness requires that similar individuals receive similar outcomes.

Definition: If two individuals are similar (according to a similarity metric), they should receive similar predictions.

|f(x_i) - f(x_j)| ≤ d(x_i, x_j) for all i, j

where f is the model and d is a similarity metric.

This is difficult to enforce in practice because the similarity metric is often subjective.

4.3 Fairness Mitigation Techniques

4.4 Implementation Example: Reweighting

def reweight_data(X, y, protected, sample_weight=None):
    """
    Reweights the data to achieve balance between protected groups.
    """
    # Calculate weights for each group
    weights = {}
    groups = np.unique(protected)
    for group in groups:
        idx = protected == group
        weights[group] = 1.0 / idx.mean()
    
    # Apply weights
    sample_weight = np.array([weights[p] for p in protected])
    return sample_weight

5. Explainable AI (XAI)

5.1 The Regulatory Requirement

Regulators (SR 11-7, EBA, GDPR) require that AI models be explainable. This means that the institution must be able to explain why a decision was made.

Levels of explainability:

1. Global explainability: Understanding the overall behavior of the model.

2. Local explainability: Understanding why a specific decision was made.

3. Interactive explainability: The ability to ask “what if” questions.

5.2 Methods for Explainability

5.3 SHAP in Practice

SHAP is the most widely used method for explainability. It provides both global and local explanations.

Global explanation:

• A bar chart showing the average SHAP value for each feature.

• A beeswarm plot showing the distribution of SHAP values for each feature.

Local explanation:

• A waterfall plot showing how each feature contributed to a specific prediction.

Implementation:

import shap

# Create explainer
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)

# Global explanation
shap.summary_plot(shap_values, X_test, feature_names=feature_names)

# Local explanation
shap.waterfall_plot(shap.Explanation(values=shap_values[0], 
                                     base_values=explainer.expected_value,
                                     data=X_test.iloc[0],
                                     feature_names=feature_names))

5.4 Counterfactual Explanations

A counterfactual explanation answers: “What minimal change would change the decision?”

Implementation:

def counterfactual(model, X, target_class, feature_bounds):
    """
    Find the minimal change to X that changes the prediction to target_class.
    """
    # This is an optimization problem
    def loss(delta):
        X_new = X + delta
        prediction = model.predict(X_new)
        return -prediction[target_class] + lambda * np.linalg.norm(delta)
    
    # Use gradient descent to find the minimal delta
    result = minimize(loss, x0=np.zeros(X.shape), 
                      bounds=feature_bounds)
    return X + result.x

6. Data Privacy and Security

6.1 Privacy Regulations

6.2 Privacy-Preserving Techniques

6.3 Differential Privacy (Review)

Differential privacy provides a mathematical guarantee of privacy. A mechanism M satisfies (ε, δ)-DP if:

P(M(D) ∈ S) ≤ exp(ε) * P(M(D') ∈ S) + δ

For model training, DP-SGD clips and adds noise to gradients.

Privacy budget accounting: The total privacy budget ε is accumulated over the training iterations. Using a moments accountant, we can compute the total (ε, δ)-DP.

7. Responsible AI Governance

7.1 The AI Governance Framework

┌─────────────────────────────────────────────────────────────────────────────┐
│                      RESPONSIBLE AI GOVERNANCE FRAMEWORK                    │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  ┌──────────────────────────────────────────────────────────────────────┐    │
│  │                          BOARD OVERSIGHT                            │    │
│  │  (Sets policy, approves high-risk AI, monitors compliance)          │    │
│  └──────────────────────────────────────────────────────────────────────┘    │
│                                    │                                        │
│                                    ▼                                        │
│  ┌──────────────────────────────────────────────────────────────────────┐    │
│  │                       AI ETHICS COMMITTEE                            │    │
│  │  (Reviews AI applications, assesses ethical risks)                   │    │
│  └──────────────────────────────────────────────────────────────────────┘    │
│                                    │                                        │
│                                    ▼                                        │
│  ┌──────────────────────────────────────────────────────────────────────┐    │
│  │                       MODEL RISK MANAGEMENT                          │    │
│  │  (Validates models, monitors performance, ensures compliance)        │    │
│  └──────────────────────────────────────────────────────────────────────┘    │
│                                    │                                        │
│                                    ▼                                        │
│  ┌──────────────────────────────────────────────────────────────────────┐    │
│  │                       DEVELOPMENT TEAMS                              │    │
│  │  (Build and deploy models with responsible AI practices)             │    │
│  └──────────────────────────────────────────────────────────────────────┘    │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

7.2 AI Ethics Committee

The AI Ethics Committee should:

1. Review high-risk AI applications: Assess ethical risks before deployment.

2. Develop ethical guidelines: Establish principles for AI use.

3. Monitor compliance: Ensure that AI systems comply with ethical guidelines.

4. Handle complaints: Investigate and resolve complaints about AI systems.

7.3 Model Risk Management (Review)

Model Risk Management (MRM) is the process of identifying, assessing, and mitigating model risk. It is a regulatory requirement (SR 11-7).

MRM activities:

1. Model inventory: Track all models in use.

2. Model validation: Validate models before deployment.

3. Model monitoring: Monitor models in production.

4. Model retirement: Retire models when they are no longer fit for purpose.

5. Documentation: Maintain comprehensive documentation.

8. Summary for the AI Practitioner

• Responsible AI is a regulatory and ethical imperative in finance.

• Bias can arise from data, algorithms, and societal factors. Proxy variables are a common source.

• Fairness metrics (disparate impact, equal opportunity, equal odds) help quantify and mitigate bias.

• Explainability is required by regulation; SHAP, LIME, and counterfactuals are standard methods.

• Data privacy requires differential privacy, federated learning, or synthetic data.

• AI governance requires board oversight, an ethics committee, and robust MRM processes.