1. Learning Objectives

By the end of this lesson, you will be able to:

  • Understand the regulatory framework for model risk management (SR 11-7, EBA guidelines) and its application to AI models.

  • Implement a comprehensive model lifecycle management process, including development, validation, implementation, and ongoing monitoring.

  • Apply statistical and ML-specific validation techniques for AI models, including backtesting, benchmarking, and sensitivity analysis.

  • Design an AI governance framework covering data lineage, model documentation, version control, and change management.

  • Implement fairness and bias testing for AI models, including disparate impact analysis and equal opportunity metrics.

  • Address the challenges of model explainability, reproducibility, and interpretability in the context of regulatory compliance.

2. The Regulatory Framework for Model Risk Management

2.1 SR 11-7 (Supervisory Guidance on Model Risk Management)

SR 11-7, issued by the Federal Reserve and OCC, is the cornerstone of model risk management (MRM) in the US banking system. It defines a model as “a quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into quantitative estimates.”

The guidance establishes three lines of defense:

  1. Model Development: The developers build the model, document its theoretical basis, assumptions, and limitations. They conduct initial testing (e.g., backtesting, benchmarking).

  2. Model Validation: An independent group (or external party) validates the model. This includes assessing the model’s conceptual soundness, data quality, and predictive power. Validation also includes ongoing monitoring.

  3. Model Governance: Senior management and the board oversee the model risk management process, ensuring appropriate policies and controls are in place.

2.2 Key Requirements for AI Models

  • Conceptual soundness: The model’s theoretical basis must be well-founded. For AI models, this means explaining why a particular architecture was chosen and how it captures the relationships in the data.

  • Data quality: The data used for training and testing must be complete, accurate, and representative. Data lineage (origin and transformations) must be documented.

  • Outcome analysis: The model’s outputs must be compared against actual outcomes (backtesting). The model must perform well across different scenarios (stress testing).

  • Limitations: The model’s limitations must be clearly documented (e.g., the types of data it was trained on, the expected error rates).

  • Monitoring: The model must be monitored continuously for performance degradation. Triggers for re-validation must be defined (e.g., when the model’s error rate exceeds a threshold).

2.3 EBA Guidelines on the Use of AI and ML

The European Banking Authority (EBA) has issued guidelines specifically for AI and ML models. Key points:

  • Transparency: Models must be transparent and explainable. For black-box models, the institution must justify why they are used and how they will be validated.

  • Fairness: Models must not discriminate against protected groups. Bias testing must be conducted.

  • Risk management: The institution must have robust risk management processes for AI models, including contingency plans for model failure.

  • Data protection: The processing of personal data must comply with GDPR.

3. The Model Lifecycle and Validation

3.1 Model Development

The development phase includes:

  1. Problem definition: Define the business problem, the target variable, and the evaluation metric.

  2. Data collection: Gather relevant data, ensure data quality, and perform exploratory data analysis (EDA).

  3. Feature engineering: Create features that capture predictive signals (as covered in Lessons 7.2 and 8.2).

  4. Model selection: Choose an appropriate algorithm based on the problem and data characteristics.

  5. Model training: Train the model on the training data, using cross-validation for hyperparameter tuning.

  6. Model documentation: Document all decisions, including data sources, feature definitions, model architecture, hyperparameters, and performance metrics.

Documentation standards: A comprehensive model document should include:

  • Executive summary: Purpose and key results.

  • Data description: Sources, definitions, transformations, and missing value treatment.

  • Model methodology: Theoretical basis, algorithm choice, and training process.

  • Validation results: In-sample and out-of-sample performance, sensitivity analysis.

  • Limitations and assumptions: What the model cannot capture, any simplifications.

  • Implementation details: Deployment architecture, integration with existing systems.

3.2 Model Validation

Validation is an independent assessment of the model’s quality. It includes:

Conceptual soundness:

  • Review the model’s theoretical basis and assumptions.

  • Assess the choice of algorithm and its suitability for the problem.

  • Evaluate the feature engineering process and the quality of the data.

Predictive power:

  • Backtesting: Compare the model’s predictions with actual outcomes on out-of-sample data.

  • Benchmarking: Compare the model’s performance with a simpler model (e.g., logistic regression) to ensure that the added complexity is justified.

  • Sensitivity analysis: Test how the model’s predictions change when inputs are varied (e.g., changing the assumed LGD or PD).

Robustness:

  • Stress testing: Test the model under extreme scenarios (e.g., a severe economic downturn).

  • Adversarial testing: Test the model with artificially generated data to see if it is vulnerable to manipulation.

Implementation testing:

  • Test the model’s code to ensure it is correct and reproducible.

  • Test the integration of the model with downstream systems.

3.3 Backtesting Techniques for AI Models

For classification models (PD, fraud detection):

  • Calibration: Compare the predicted probabilities with the actual default rates across deciles (Hosmer-Lemeshow test).

  • Discriminatory power: AUC, Gini, KS statistics.

  • Stability: Population Stability Index (PSI) to detect shifts in the feature distribution.

For regression models (LGD, EAD, VaR):

  • Mean absolute error (MAE) and root mean squared error (RMSE).

  • Forecast error decomposition: Bias (mean error), variance (RMSE – bias).

  • Diebold-Mariano test: Compare the forecast accuracy of two models.

For time series models (volatility forecasting):

  • Mincer-Zarnowitz regression: Regress actual realized volatility on the predicted volatility. The intercept should be 0 and the slope should be 1.

  • Diebold-Mariano test for comparing forecast accuracy.

4. AI Governance Framework

4.1 Data Lineage and Data Management

Data lineage tracks the origin and transformations of data used in the model. This is essential for:

  • Auditability: Regulators can trace the model’s predictions back to the input data.

  • Reproducibility: The model can be retrained and validated with the same data.

  • Error diagnosis: If the model performs poorly, we can trace the error to a specific data source or transformation.

Implementation: Use a data lineage tool (e.g., Apache Atlas, Amundsen) that records metadata about data sources, transformations, and dependencies. For code, use a version control system (e.g., Git) with commit messages that document changes.

4.2 Model Version Control

Models evolve over time as new data becomes available and as the business requirements change. Model version control ensures that:

  • Each version of the model is tagged and documented.

  • The changes between versions are visible.

  • We can roll back to a previous version if needed.

Implementation: Use a model registry (e.g., MLflow, Weights & Biases) that stores the model’s code, hyperparameters, data references, and performance metrics for each version.

4.3 Change Management

Any change to the model (e.g., retraining, fixing a bug, adding a feature) must be managed through a formal change management process. This includes:

  1. Request for change: A formal request describing the change and its justification.

  2. Impact assessment: Assess the impact of the change on model performance and risk.

  3. Approval: The change must be approved by an appropriate authority (e.g., model risk management committee).

  4. Implementation: The change is implemented in a controlled environment.

  5. Validation: The changed model is re-validated (at least partially) before being deployed.

  6. Documentation: The change and its rationale are documented.

5. Fairness and Bias Testing

5.1 Defining Protected Groups

In finance, protected groups are defined by legislation (e.g., the Equal Credit Opportunity Act in the US). Common protected attributes include:

  • Race

  • Gender

  • Age

  • National origin

  • Marital status

However, many AI models do not directly use these attributes. Bias can still arise through proxy variables (e.g., zip code, income, property value) that are correlated with protected attributes.

5.2 Fairness Metrics

  • Disparate impact: The ratio of the selection rate (e.g., loan approval rate) for the protected group to the selection rate for the non-protected group. A ratio below 0.8 (the “Four-Fifths Rule”) indicates disparate impact.

  • Equal opportunity: The true positive rate (TPR) should be equal across groups: TPR_protected = TPR_non-protected. This ensures that qualified individuals from the protected group are not unfairly rejected.

  • Equal odds: Both the TPR and the false positive rate (FPR) should be equal across groups: TPR_protected = TPR_non-protected and FPR_protected = FPR_non-protected.

  • Predictive parity: The positive predictive value (PPV) should be equal across groups: PPV_protected = PPV_non-protected.

5.3 Bias Mitigation Techniques

Pre-processing (data level):

  • Reweighting: Assign different weights to instances from different groups to balance the representation.

  • Removal of proxy variables: Remove features that are highly correlated with protected attributes.

  • Fair representation learning: Learn a representation of the data that preserves predictive power but is independent of the protected attribute (adversarial debiasing).

In-processing (model level):

  • Fairness constraints: Add a constraint to the model’s objective function that penalizes disparate impact or equal opportunity violations.

  • Regularization: Add a regularization term that encourages fairness (e.g., minimizing the difference in FPR between groups).

Post-processing (output level):

  • Threshold adjustment: Choose different thresholds for different groups to equalize TPR or FPR.

  • Reject option classification: For borderline cases, give a different outcome for the protected group to improve fairness.

5.4 Testing for Bias

Bias testing should be part of the validation process. The validation team should:

  1. Identify protected attributes.

  2. Compute fairness metrics on the validation set.

  3. Test for statistical significance using permutation tests or bootstrapping.

  4. If bias is detected, recommend mitigation measures.

6. Explainability and Interpretability in AI Governance

6.1 Regulatory Expectations

Regulators expect that models, especially those used for credit decisions, are explainable. This does not necessarily mean that the model must be simple; it means that the institution must be able to explain the model’s decisions to the regulator and to the affected individuals.

For complex AI models, the institution must provide:

  • A justification for using a black-box model (e.g., superior predictive power).

  • A description of the model’s architecture and training process.

  • A comprehensive explanation of the features and their importance (global explanation).

  • An ability to explain individual decisions (local explanation).

6.2 Explanation Methods for AI Models

  • Global explanations: Feature importance (permutation, SHAP), partial dependence plots (PDP), and accumulated local effects (ALE) plots.

  • Local explanations: SHAP, LIME, and counterfactual explanations.

SHAP is particularly recommended because it provides consistent and game-theoretically sound attributions. For tree-based models, TreeSHAP provides exact Shapley values. For neural networks, we can use DeepSHAP or GradientSHAP.

Counterfactual explanations: “If the customer’s income had been $5,000 higher, the loan would have been approved.” This is actionable and easy to understand.

6.3 Monitoring Explanations

Explanations should be monitored over time:

  • Feature importance drift: If the importance of a feature changes significantly, it may indicate a change in the data distribution (non-stationarity) or a model drift.

  • SHAP value distribution: The distribution of SHAP values for a feature should be stable over time. A significant change may indicate that the feature’s relationship with the target has changed.

7. Summary for the AI Practitioner

  • Model risk management is a regulatory requirement (SR 11-7, EBA guidelines) that applies to AI models.

  • The model lifecycle includes development, validation, implementation, and monitoring. Each phase must be well-documented.

  • Validation includes conceptual soundness, predictive power, robustness, and implementation testing.

  • Governance includes data lineage, model version control, and change management.

  • Bias and fairness are critical; institutions must test for disparate impact and equal opportunity violations.

  • Explainability is essential; use SHAP, LIME, and counterfactuals to explain model decisions.

 
PreviousNext