SECTION 1: LEARNING OBJECTIVES

By the end of this lesson, you will be able to:

  • Extend simple regression to multiple predictors and interpret the coefficients.

  • Understand the concept of partial effects – the effect of one variable holding others constant.

  • Assess overall model fit using adjusted R² and F‑statistic.

  • Detect and address multicollinearity – a common issue in financial data.

  • Perform variable selection using stepwise methods and information criteria (AIC, BIC).

  • Apply cross‑validation to evaluate predictive performance.

  • Interpret regression output for business decision‑making in banking.

  • Recognise overfitting and how to avoid it.

SECTION 2: THE MULTIPLE REGRESSION MODEL

When a financial outcome is influenced by multiple factors, we use multiple linear regression:

Y=β0+β1X1+β2X2+⋯+βkXk+ε

Each β coefficient represents the average change in Y for a one‑unit change in that predictor, holding all other predictors constant. This is known as a partial regression coefficient.

Example – House Price Prediction in a Mortgage Portfolio:
Y = house price
X₁ = square footage, X₂ = number of bedrooms, X₃ = age, X₄ = location score

The coefficient for square footage tells us the price increase per additional square foot, after controlling for bedrooms, age, and location.

SECTION 3: ADJUSTED R‑SQUARED AND OVERFITTING

Adding more predictors always increases R², even if they are irrelevant. Adjusted R² penalises the addition of unnecessary variables:

Radj2=1−(1−R2)(n−1)n−k−1

where n = sample size, k = number of predictors.

  • Adjusted R² increases only if the new variable improves the model more than would be expected by chance.

  • It is a better measure for comparing models with different numbers of predictors.

F‑test for overall significance: tests whether at least one coefficient is non‑zero (i.e., the model explains variation beyond the mean). A significant F‑statistic indicates the model is useful.

SECTION 4: MULTICOLLINEARITY

Multicollinearity occurs when predictors are highly correlated with each other. This makes it difficult to isolate the unique effect of each variable.

Consequences:

  • Coefficients become unstable and have large standard errors.

  • P‑values may become non‑significant even when the variables are important.

  • The model’s predictions may still be accurate, but interpretation is compromised.

Detection:

  • Variance Inflation Factor (VIF): VIF > 10 indicates severe multicollinearity.

  • Correlation matrix: look for correlations > 0.8 between predictors.

Remedies:

  • Remove one of the correlated variables.

  • Combine them into a composite index (e.g., using principal components).

  • Use regularisation techniques (ridge or lasso regression) – covered in later modules.

Financial Example: In a mortgage default model, include both “debt-to-income” and “loan-to-value” – these are often correlated. Avoid including both if they are highly collinear, or use domain knowledge to keep the most relevant.

SECTION 5: VARIABLE SELECTION

5.1 Why Select Variables?

  • To improve interpretability.

  • To reduce overfitting and increase out‑of‑sample predictive power.

  • To satisfy regulatory requirements (models should be as simple as possible while adequate).

5.2 Common Methods
 
 
Method Description Pros / Cons
Forward selection Start with no predictors; add the most significant one at each step. Fast, but may miss interactions.
Backward elimination Start with all predictors; remove the least significant one iteratively. More thorough, but may not work if k > n.
Stepwise Combination of forward and backward. Convenient but can be unstable.
Information criteria (AIC, BIC) Choose the model that minimises AIC or BIC (penalises complexity). BIC penalises more strongly; good for large datasets.

Caveat: Automated selection should be guided by business logic and domain expertise. A variable that is statistically insignificant may still be economically important.

5.3 Regularisation (Brief Introduction)

Ridge and Lasso regression are techniques that shrink coefficients to reduce overfitting, especially when you have many predictors. These are covered in depth in later modules but are worth mentioning here.

SECTION 6: ASSESSING PREDICTIVE PERFORMANCE

6.1 Training vs. Test Set

Split the data into training (e.g., 70%) and testing (30%) sets. Build the model on training data and evaluate its predictions on test data. This gives a realistic estimate of how the model will perform in practice.

6.2 Cross‑Validation

k‑fold cross‑validation repeats the train‑test split k times, each time using a different fold as the test set. The average performance across folds is a robust estimate of generalisation error.

Why it matters in banking: Regulators (e.g., SR 11‑7) require that models be validated on out‑of‑sample data to ensure they are not overfitted.

6.3 Common Error Metrics

  • Mean Absolute Error (MAE) – average absolute difference between actual and predicted.

  • Mean Squared Error (MSE) – penalises large errors more heavily.

  • Root Mean Squared Error (RMSE) – in the same units as Y, easier to interpret.

  • Mean Absolute Percentage Error (MAPE) – useful for relative comparisons.

SECTION 7: INTERPRETING REGRESSION OUTPUT FOR BUSINESS

Assume we have a bank’s customer profitability model:

 
 
Predictor Coefficient P‑value Interpretation
Intercept -5,000 0.001 Base profit (not meaningful alone)
Age (years) 150 0.002 Each additional year adds $150 profit, holding others constant.
Income ($000) 25 <0.001 Each $1,000 income adds $25 profit.
Transaction count 8 0.10 Not statistically significant; may be dropped.
Online engagement score 120 <0.001 Each point on the engagement scale adds $120 profit.

Business actions:

  • Focus marketing efforts on high‑income, highly engaged customers.

  • The non‑significant transaction count suggests re‑evaluating its use in segmentation.

Always report:

  • Coefficients with confidence intervals.

  • The model’s R² and adjusted R².

  • Results of residual diagnostics and multicollinearity checks.

  • Any transformations applied.

SECTION 8: SUMMARY FOR THE DATA PRACTITIONER

  • Multiple regression allows modelling of complex financial relationships with several predictors.

  • Adjusted R² and F‑test assess overall model quality.

  • Multicollinearity can undermine interpretation – detect via VIF.

  • Variable selection should balance statistical significance and business relevance.

  • Cross‑validation is essential for reliable out‑of‑sample performance.

  • In banking, multiple regression is used for credit scoring, customer lifetime value, price elasticity, and risk factor models.

  • Always interpret coefficients in the context of the business problem and avoid over‑automation.

SECTION 9: NEXT STEPS

  • Apply multiple regression to your own datasets.

  • Use VIF to check for multicollinearity.

  • Compare models using adjusted R² and cross‑validation.

  • Consider transformations (log, polynomial) to capture non‑linear relationships.

  • Prepare for more advanced topics like logistic regression (for binary outcomes) and regularisation.

[END OF LESSON 5]

 
 
PreviousNext