1. Learning Objectives

By the end of this lesson, you will be able to:

• Understand the credit risk framework: probability of default (PD), loss given default (LGD), exposure at default (EAD), and expected loss (EL).

• Implement traditional credit scoring models (logistic regression, LDA) and evaluate their discriminatory power using AUC and Gini.

• Apply advanced ML models (gradient boosting, random forests, neural networks) for PD estimation, handling imbalanced data and missing values.

• Model LGD and EAD using regression trees and survival analysis.

• Design a multi-model credit risk framework with explainability (SHAP) and stress testing.

• Understand the regulatory requirements (IFRS 9, CECL, Basel III) and the role of AI in meeting them.

2. Credit Risk: The Fundamental Framework

Credit risk is the risk of loss due to a borrower’s failure to make payments. The standard framework decomposes expected loss (EL) into three components:

EL = PD * LGD * EAD

where:

• PD (Probability of Default): The likelihood that the borrower defaults within a given horizon (e.g., 1 year).

• LGD (Loss Given Default): The percentage of the exposure that is lost in the event of default (after recoveries).

• EAD (Exposure at Default): The total exposure (outstanding balance) at the time of default.

2.1 Probability of Default (PD)

PD is typically modeled using a binary classification framework. The borrower is either “good” (non-default) or “bad” (default). The default event is rare, leading to imbalanced data.

The structural model (Merton model) derives PD from the firm’s asset value and leverage. If the firm’s asset value falls below its debt, it defaults. The distance-to-default (DD) is:

DD = (ln(V_A / D) + (r_f - 0.5 * σ_A²) * T) / (σ_A * sqrt(T))

where V_A is asset value, D is debt, r_f is risk-free rate, σ_A is asset volatility, and T is the horizon. PD = Φ(-DD) (assuming normal distribution). This is the foundation of the KMV model.

2.2 Loss Given Default (LGD)

LGD is the proportion of the exposure lost in default. It depends on the collateral, seniority, and recovery process. LGD is often modeled using a beta distribution (bounded between 0 and 1) or using regression.

2.3 Exposure at Default (EAD)

For fixed exposure (e.g., term loans), EAD is the outstanding balance. For revolving facilities (credit cards, lines of credit), EAD is uncertain; it is modeled using a drawdown factor (the percentage of the committed limit likely to be drawn at default).

2.4 Regulatory Capital

Basel III requires banks to hold capital against unexpected losses. The capital requirement for credit risk is:

Capital = EAD * LGD * K(PD) * M_adjustment

where K(PD) is the capital function (based on the asymptotic single risk factor model), and M is maturity adjustment.

3. Traditional Credit Scoring Models

3.1 Logistic Regression

Logistic regression is the workhorse of credit scoring. The probability of default is:

P(Y=1 | X) = 1 / (1 + exp(-(β_0 + β_1 X_1 + ... + β_p X_p)))

The parameters β are estimated by maximizing the log-likelihood:

L(β) = ∑_{i=1}^{N} [ y_i * ln(p_i) + (1-y_i) * ln(1-p_i) ]

where p_i = P(Y=1 | X_i). The model outputs a score, which can be transformed to a probability.

3.2 Linear Discriminant Analysis (LDA)

LDA assumes that the features follow a multivariate normal distribution within each class (good/bad) and have the same covariance matrix. The discriminant function is:

δ_k(X) = Xᵀ Σ^{-1} μ_k - 0.5 * μ_kᵀ Σ^{-1} μ_k + ln(π_k)

where π_k is the prior probability of class k. LDA is less flexible than logistic regression but can be more stable with small samples.

3.3 Evaluation: AUC and Gini

• AUC (Area Under the ROC Curve): Measures the discriminatory power of the model. AUC = 0.5 means random; AUC = 1.0 means perfect discrimination.

• Gini coefficient: Gini = 2 * AUC - 1. A Gini > 0.4 is considered good; > 0.6 is excellent.

The ROC curve plots the true positive rate (TPR) against the false positive rate (FPR) at various thresholds. The AUC is the area under this curve.

4. Advanced ML for Default Prediction

4.1 Handling Imbalanced Data

Default events are rare (1-5% of borrowers). Standard models are biased towards the majority class. Techniques to handle imbalance:

• Resampling: Oversample the minority class (SMOTE – Synthetic Minority Over-sampling Technique) or undersample the majority class.

• Cost-sensitive learning: Assign a higher weight to default cases in the loss function:

  L(θ) = ∑_{i=1}^{N} w_i * L(y_i, f(x_i)) where w_i = w_default if y_i=1, else w_non-default.

• Ensemble methods: Use bagging or boosting with resampled data (e.g., Balanced Random Forest, EasyEnsemble).

4.2 Gradient Boosting Machines (GBM) for Credit Scoring

GBM often outperforms logistic regression on credit scoring datasets. XGBoost, LightGBM, and CatBoost are popular choices. They handle non-linearities, interactions, and missing values natively.

Example:

import xgboost as xgb
params = {
    'objective': 'binary:logistic',
    'scale_pos_weight': (n_non_default / n_default),  # balance classes
    'max_depth': 6,
    'learning_rate': 0.01,
    'n_estimators': 1000,
    'early_stopping_rounds': 50
}
model = xgb.train(params, dtrain, evals=[(dval, 'val')])

4.3 Deep Learning for PD

Neural networks can capture complex patterns but require large datasets. A common architecture:

1. Input: Numerical features (standardized) and categorical features (embedded).

2. Hidden layers: Dense layers with ReLU activation, batch normalization, and dropout.

3. Output: A sigmoid output for the default probability.

Regularization: Dropout, L1/L2 regularization, and early stopping are essential to prevent overfitting.

4.4 Survival Analysis for PD

Survival analysis models the time-to-default. The Cox proportional hazards model is:

h(t | X) = h_0(t) * exp(Xᵀ β)

where h(t) is the hazard rate (default intensity). The survival function is:

S(t | X) = exp(-∫_0^t h(u | X) du)

The PD over horizon T is 1 - S(T | X).

Neural extensions (DeepSurv, Cox-MLP) can capture non-linearities. Survival analysis uses all available data, including censored observations (borrowers who have not defaulted yet), providing more efficient estimates than binary classification.

5. Modeling LGD and EAD

5.1 LGD Modeling

LGD is bounded between 0 and 1. Beta regression is a natural choice:

LGD ~ Beta(μ, φ) where μ = g^{-1}(Xᵀ β) and φ is the dispersion parameter. The logit link is common: g(μ) = log(μ/(1-μ)).

Neural networks can also be used with a sigmoid output and a beta (or bounded) loss. Alternatively, one can model the log of LGD (with appropriate transformation) using OLS.

5.2 EAD Modeling

For revolving facilities, we model the “credit conversion factor” (CCF) – the proportion of the undrawn limit that will be drawn at default. CCF is bounded between 0 and 1, so Beta regression is also suitable. For term loans, EAD is the outstanding balance at default (which may be known or forecasted).

5.3 Correlations in the Portfolio

The default events and LGD/EAD are correlated (especially during economic downturns). A common model is the Asymptotic Single Risk Factor (ASRF) model, which assumes that all defaults are driven by a single systemic factor (the economy). This model is used in the Basel capital calculations. More advanced models use a factor model with multiple common factors (e.g., industry, region) and specific factors.

6. Explainability and Validation

6.1 SHAP for Credit Models

SHAP provides global and local explanations. For credit models, it is crucial to ensure that the model does not discriminate against protected groups (race, gender, etc.). SHAP can identify which features drive predictions and whether they are acceptable.

For example, if “zip code” (a proxy for race) has a large SHAP value, the model may be discriminatory. We can then adjust the model (e.g., by excluding the feature or using a fairness constraint).

6.2 Model Validation

• Discriminatory power: AUC, Gini, Kolmogorov-Smirnov (KS) statistic.

• Calibration: The Hosmer-Lemeshow test checks if the predicted probabilities match the observed default rates across deciles.

• Stability: Population stability index (PSI) measures how the distribution of scores changes over time. PSI > 0.2 indicates a significant shift.

• Backtesting: Compare predicted PDs with actual default rates over time (e.g., for each rating grade).

6.3 Regulatory Requirements

• Basel III: Uses PD, LGD, EAD for capital calculation. Models must be approved by the supervisor.

• IFRS 9 (International Financial Reporting Standard 9): Requires expected credit losses (ECL) to be recognized on day 1, based on forward-looking PD estimates (including macro scenarios).

• CECL (Current Expected Credit Loss) – US: Similar to IFRS 9, requires lifetime expected losses.

These regulations require models that are interpretable, well-documented, and subject to robust validation.

7. Summary for the AI Practitioner

• Credit risk is decomposed into PD, LGD, and EAD. Each component can be modeled separately.

• Traditional credit scoring uses logistic regression; advanced ML (XGBoost, neural networks) often improves predictive power.

• Imbalanced data handling is crucial; use SMOTE or cost-sensitive learning.

• LGD and EAD can be modeled with Beta regression or neural networks.

• Explainability (SHAP) is essential for regulatory compliance and fairness.

• Model validation must include discriminatory power, calibration, and stability testing.