Lesson 7.3: Supervised Learning for Trading – Regression, Classification, and Ensemble Models

1. Learning Objectives

By the end of this lesson, you will be able to:

Formulate trading signal generation as a supervised learning problem, distinguishing between regression (predicting returns) and classification (predicting price direction).
Implement and derive the mathematical foundations of linear models (OLS, Ridge, LASSO) and their application to factor-based forecasting.
Apply tree-based ensemble methods (Random Forest, Gradient Boosting Machines, XGBoost) to capture non-linear interactions and high-dimensional feature spaces.
Design a robust backtesting framework that accounts for transaction costs, slippage, and data-snooping bias.
Evaluate model performance using Sharpe ratio, maximum drawdown, hit rate, and profit factor, with appropriate statistical significance tests.
Address common pitfalls: overfitting, look-ahead bias, non-stationarity, and regime changes.

2. Problem Formulation: Regression vs. Classification

2.1 The Forecasting Horizon

Given a feature vector x_t (constructed from data up to time t), we want to predict a target variable y_{t+h} at a future horizon h. The choice of h is critical:

Short-term (h = 1 to 5 minutes): High-frequency trading, market making. Targets are often the mid-price change or the probability of a price move.
Intraday (h = 15 minutes to 2 hours): Momentum strategies, news trading.
Daily (h = 1 to 5 days): Swing trading, factor investing.
Long-term (h = 1 month to 1 year): Value investing, trend following.

The target variable can be:

Regression: y_{t+h} = r_{t+h} (continuous return). The model outputs a predicted return.
Classification: y_{t+h} = sign(r_{t+h}) (binary: +1 for up, -1 for down) or a multi-class target (e.g., up > 1%, flat, down < 1%). The model outputs class probabilities.

2.2 The Predictive Model

We seek a function f that minimises a loss function L:

f^* = argmin_f E[ L(y_{t+h}, f(x_t)) ]

where the expectation is taken over the joint distribution of features and targets. In practice, we use empirical risk minimisation:

f^* = argmin_f (1/N) * ∑_{i=1}^{N} L(y_i, f(x_i))

The choice of loss function depends on the problem:

Regression: Mean Squared Error (MSE): L(y, ŷ) = (y - ŷ)^2. This penalises large errors quadratically. Alternatively, Mean Absolute Error (MAE): L(y, ŷ) = |y - ŷ| is more robust to outliers.
Classification: Cross-Entropy Loss (Log Loss): L(y, p) = -[y * log(p) + (1-y) * log(1-p)] for binary classification. This penalises confident wrong predictions.

2.3 Feature-Target Alignment and Stationarity

A critical step is ensuring that the features are constructed using only data available at time t (no look-ahead bias). This means:

Rolling windows: x_t uses data from t-L to t-1 (not including time t, as that data may not be available at the time of prediction).
Avoid using t as a feature (e.g., day-of-week should use t-1).

Additionally, the relationship between features and targets must be stationary (or at least slowly varying). We achieve this by:

Using returns instead of prices.
Differencing and normalising features.
Using exponentially weighted moments that decay older observations.

3. Linear Models for Financial Forecasting

3.1 Ordinary Least Squares (OLS)

The OLS model assumes a linear relationship:

ŷ_t = β_0 + β_1 x_{t,1} + ... + β_p x_{t,p}

The parameters β are estimated by minimising the sum of squared residuals:

β = argmin_β ∑_{t=1}^{T} (y_t - x_t^T β)^2

The solution is the closed-form normal equation:

β = (X^T X)^{-1} X^T y

where X is the T × p design matrix. The variance of the residuals is estimated as σ^2 = RSS / (T - p). The standard errors of the coefficients are the diagonal elements of σ^2 (X^T X)^{-1}.

Problems with OLS in finance:

Overfitting: With many features (p > T), the matrix X^T X is singular.
Multicollinearity: Financial features are often highly correlated (e.g., momentum and volatility), leading to unstable coefficient estimates.
Non-stationarity: The true β may change over time.

3.2 Ridge Regression (L2 Regularisation)

Ridge adds an L2 penalty to the OLS objective:

β = argmin_β [ ∑_{t=1}^{T} (y_t - x_t^T β)^2 + λ ∑_{j=1}^{p} β_j^2 ]

The solution is:
β = (X^T X + λ I)^{-1} X^T y

The penalty λ > 0 shrinks the coefficients towards zero, reducing variance but introducing bias. The optimal λ is chosen via cross-validation (using time-series cross-validation). Ridge is useful when features are correlated; it distributes the weight among correlated predictors.

3.3 LASSO Regression (L1 Regularisation)

LASSO adds an L1 penalty:

β = argmin_β [ ∑_{t=1}^{T} (y_t - x_t^T β)^2 + λ ∑_{j=1}^{p} |β_j| ]

The L1 penalty forces some coefficients to be exactly zero, performing feature selection. This is highly beneficial when we have many noisy features. The LASSO problem does not have a closed-form solution; it is solved via coordinate descent or the LARS algorithm.

Adaptive LASSO: A variation that applies different penalties to different coefficients, based on an initial estimate (e.g., from Ridge). This improves the oracle properties (selection consistency).

3.4 Elastic Net

Elastic Net combines both L1 and L2 penalties:

β = argmin_β [ ∑_{t=1}^{T} (y_t - x_t^T β)^2 + λ_1 ∑_{j=1}^{p} |β_j| + λ_2 ∑_{j=1}^{p} β_j^2 ]

This can select groups of correlated features (unlike LASSO) while still performing feature selection. The parameters λ_1 and λ_2 are tuned via cross-validation.

3.5 Time-Varying Parameters: Kalman Filter

Since financial relationships change over time, we can model the coefficients as a state-space model:

y_t = x_t^T β_t + ε_t (observation equation)
β_t = β_{t-1} + η_t (state equation)

where ε_t ~ N(0, σ_ε^2) and η_t ~ N(0, Q) are independent. The Kalman filter provides a recursive algorithm to estimate β_t as new data arrives. The prediction step:

β_{t|t-1} = β_{t-1|t-1}
P_{t|t-1} = P_{t-1|t-1} + Q

and the update step (when y_t is observed):

K_t = P_{t|t-1} x_t (x_t^T P_{t|t-1} x_t + σ_ε^2)^{-1}
β_{t|t} = β_{t|t-1} + K_t (y_t - x_t^T β_{t|t-1})
P_{t|t} = (I - K_t x_t^T) P_{t|t-1}

This provides time-varying coefficients that adapt to changing market conditions. The parameters σ_ε^2 and Q are estimated via maximum likelihood (EM algorithm).

4. Tree-Based Ensemble Methods

Tree-based models are non-parametric and can capture complex non-linear interactions without requiring explicit feature engineering. They are also robust to outliers and multicollinearity.

4.1 Decision Trees

A decision tree partitions the feature space into regions and assigns a constant prediction to each region. The splits are chosen to minimise an impurity measure. For regression, we use the Mean Squared Error (MSE) as the impurity:

Impurity(S) = (1/|S|) ∑_{i∈S} (y_i - \bar{y}_S)^2

For a split on feature j at value v, the impurity reduction is:

The split that maximises the gain is selected. The tree is grown until a stopping criterion is met (minimum leaf size, maximum depth).

Limitations: Single trees are high-variance and prone to overfitting. Ensembles address this.

4.2 Random Forest

Random Forest builds multiple trees (B trees) on bootstrap samples of the data. At each split, only a random subset of features is considered (typically sqrt(p) for classification or p/3 for regression). This decorrelates the trees, reducing variance.

The prediction is the average of the individual tree predictions:

ŷ = (1/B) * ∑_{b=1}^{B} Tree_b(x)

The out-of-bag (OOB) error provides an unbiased estimate of generalisation performance without requiring a validation set.

Feature importance: In Random Forest, feature importance can be computed as the average reduction in impurity across all trees, normalised by the number of trees.

4.3 Gradient Boosting Machines (GBM)

GBM builds trees sequentially, where each new tree corrects the errors of the previous ensemble. The algorithm is:

Initialize F_0(x) = argmin_γ ∑_{i=1}^{N} L(y_i, γ) (e.g., mean for regression, log-odds for classification).
For m = 1 to M:
a. Compute pseudo-residuals: r_{im} = -[∂L(y_i, F(x_i)) / ∂F(x_i)] evaluated at F = F_{m-1}.
b. Fit a tree h_m(x) to the residuals r_{im}.
c. Compute the step size γ_m = argmin_γ ∑_{i=1}^{N} L(y_i, F_{m-1}(x_i) + γ * h_m(x_i)).
d. Update: F_m(x) = F_{m-1}(x) + ν * γ_m * h_m(x), where ν is the learning rate.

For regression with L2 loss, r_{im} = y_i - F_{m-1}(x_i) (the negative gradient is simply the residual).

The learning rate ν (typically 0.01 to 0.1) shrinks the contribution of each tree, requiring more trees but reducing overfitting. The number of trees M is chosen via early stopping (monitoring validation error).

4.4 XGBoost (Extreme Gradient Boosting)

XGBoost is a scalable implementation of GBM with several enhancements:

Regularised objective: It adds L1 and L2 penalties on the leaf weights to prevent overfitting.
Second-order approximation: It uses a second-order Taylor expansion of the loss function, leading to faster convergence.
Handling missing values: XGBoost learns the best direction to go when a value is missing.
Parallelisation and cache awareness: Efficiently handles large datasets.

The objective at step m is:

Obj_m = ∑_{i=1}^{N} L(y_i, F_{m-1}(x_i) + h_m(x_i)) + Ω(h_m)

where Ω(h_m) = γ * T + 0.5 * λ * ∑_{j=1}^{T} w_j^2 (T is the number of leaves, w_j are leaf weights). The gain for a split is:

Gain = 0.5 * [ G_L^2 / (H_L + λ) + G_R^2 / (H_R + λ) - (G_L + G_R)^2 / (H_L + H_R + λ) ] - γ

where G_L = ∑_{i∈left} g_i, H_L = ∑_{i∈left} h_i, g_i = ∂L/∂F_{m-1}, h_i = ∂^2L/∂F_{m-1}^2. This formula is computationally efficient and allows for pruning.

4.5 LightGBM and CatBoost

LightGBM: Uses gradient-based one-side sampling (GOSS) and exclusive feature bundling (EFB) to speed up training and reduce memory usage. It also uses leaf-wise tree growth (vs. level-wise) which can lead to better accuracy.
CatBoost: Handles categorical features natively using a target-based encoding and uses symmetric trees (oblique trees) to reduce overfitting. It is known for its robustness to hyperparameters.

5. Backtesting Framework: Design and Implementation

A rigorous backtest simulates the live performance of a trading strategy using historical data. It must be realistic.

5.1 Data Splitting and Walk-Forward Validation

We cannot use random cross-validation on time series because it introduces look-ahead bias. We must use a walk-forward or expanding window approach:

Training: Data from t=1 to t=T_train.
Validation: Data from t=T_train+1 to t=T_val (used to tune hyperparameters).
Test: The next period t=T_val+1 to t=T_test.
We then roll the window forward, retrain the model (or keep it fixed) and repeat.

This simulates the real-world scenario where we only use past data to make predictions.

5.2 Trade Execution: Slippage and Costs

Transaction costs: Brokerage fees, exchange fees, and market impact. A typical estimate for liquid stocks is 10-20 basis points (0.10%-0.20%) per round trip.
Slippage: The difference between the expected execution price and the actual executed price. For a market order, the execution price is the VWAP of the consumed LOB levels. We can model slippage as a function of order size relative to the average daily volume (ADV):

Slippage(Q) = c * σ * (Q / ADV)^{0.5} (the square-root law from Lesson 7.1).

We simulate this by, for each trade, adjusting the price by a random or deterministic impact factor.

5.3 Position Sizing and Risk Management

Fixed fractional: Allocate a fixed percentage of capital per trade (e.g., 5%).
Kelly criterion: For a binary outcome (win/loss), the optimal fraction to bet is f* = (p * b - (1-p)) / b, where p is the win probability and b is the odds (win/loss ratio). However, Kelly is aggressive; we often use a fraction of Kelly (e.g., half-Kelly).
Volatility targeting: Set position size inversely proportional to the forecasted volatility. This keeps the portfolio’s risk constant over time:

Position_t ∝ 1 / σ_t where σ_t is the rolling standard deviation of returns.
Stop-loss: Exit a trade when the loss exceeds a threshold (e.g., 2% of capital). This limits downside but can also reduce profitability if the market later recovers.

5.4 Performance Metrics

Sharpe Ratio: SR = (Mean(Excess Returns)) / Std(Excess Returns) * sqrt(252) (for daily data). The Sharpe ratio measures risk-adjusted returns. A SR > 1 is good; > 2 is excellent.
Sortino Ratio: Similar to Sharpe but uses only downside deviation (negative returns) as the risk measure:

Sortino = (Mean(Excess Returns)) / Std(Negative Returns) * sqrt(252)

This penalises only downside risk, which is more appropriate for investors.
Maximum Drawdown (MDD): The maximum peak-to-trough decline in the equity curve:

MDD = max_{t} ( (Peak_t - Trough_t) / Peak_t )

This measures the worst-case loss.
Calmar Ratio: Sharpe-like ratio using MDD as the risk measure: Calmar = (Annualised Return) / MDD.
Hit Rate: The percentage of profitable trades: Hit Rate = (# Winning Trades) / (# Total Trades).
Profit Factor: The ratio of gross profit to gross loss: PF = (∑ Positive Returns) / |∑ Negative Returns|. A PF > 1.5 is good.
Information Ratio (IR): The annualised excess return over a benchmark (e.g., buy-and-hold) divided by the tracking error (standard deviation of the excess returns). This measures the skill of the strategy.
Statistical Significance: We can use the t-test to test if the mean return is significantly different from zero. The test statistic is:

t = (Mean Return) / (Std Error of Mean)

A t-statistic > 2 is often considered significant (p < 0.05). However, due to the non-normal distribution of returns, we can also use the Kolmogorov-Smirnov or Wilcoxon signed-rank test.

6. Dealing with Overfitting and Non-Stationarity

Overfitting is the biggest risk in financial machine learning. Strategies to mitigate:

Simplicity: Start with a simple model (linear) and only add complexity if it improves validation performance.
Regularisation: Use L1/L2 penalties (LASSO, Ridge, Elastic Net).
Feature selection: Remove highly correlated features; use domain knowledge to filter out spurious signals.
Out-of-sample testing: Reserve the most recent data (e.g., the last 2 years) as a final hold-out test set. Only use it once.
Regime switching: Detect structural breaks (e.g., using the CUSUM test). If a regime change is detected, reset the model (re-train from scratch).
Monte Carlo simulation: Generate many random features and evaluate the model’s performance. If the model performs similarly on random noise, it is overfitting.

The Deflated Sharpe Ratio (DSR): Proposed by Marcos Lopez de Prado, the DSR adjusts the Sharpe ratio for multiple testing. If you have tested N strategies, the probability of finding a high Sharpe ratio by chance increases. The DSR corrects for this.

7. Summary for the AI Practitioner

Supervised learning for trading can be framed as regression or classification, depending on the target variable and the trading objective.
Linear models (OLS, Ridge, LASSO) provide a baseline, are interpretable, and serve as benchmarks. Regularisation is essential to handle high-dimensional features.
Ensemble methods (Random Forest, XGBoost) capture complex non-linear relationships and are often state-of-the-art in many financial forecasting competitions.
Backtesting must be realistic: include transaction costs, slippage, and robust position sizing. Use walk-forward validation to avoid look-ahead bias.
Overfitting is the primary challenge; use regularisation, feature selection, and strict out-of-sample testing. Always be aware of non-stationarity and regime changes.

Diploma in Artificial Intelligence in Finance Lesson 7.3: Supervised Learning for Trading – Regression, Classification, and Ensemble Models

Diploma in Artificial Intelligence in Finance