1. Learning Objectives

By the end of this lesson, you will be able to:

• Rigorously define and test for stationarity using ADF and KPSS tests, and apply appropriate transformations (differencing, Box-Cox).

• Estimate and interpret the Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF) for model selection.

• Formulate, estimate, and interpret ARIMA and SARIMA models and use them as benchmark features for AI models.

• Test for cointegration using the Engle-Granger and Johansen procedures, and construct cointegrated spread features for pairs trading.

• Formulate and estimate GARCH-family models (GARCH, EGARCH, GJR-GARCH) to extract conditional volatility features.

• Apply Vector Autoregression (VAR) and Granger Causality tests to multivariate macro-financial datasets.

• Understand the theoretical link between these econometric models and modern deep learning architectures (RNNs, Transformers).

---

2. Stationarity – The Foundation of Predictive Modelling

A time series is weakly stationary if its first two moments are time-invariant:

1. E[Y_t] = μ (constant mean).

2. Var(Y_t) = σ² (constant variance).

3. Cov(Y_t, Y_{t-k}) = γ_k (autocovariance depends only on lag k).

Why AI Needs Stationarity: A neural network trained on a non-stationary series learns spurious correlations that change over time. The network’s weights become obsolete as market regimes shift.

2.1 The Augmented Dickey-Fuller (ADF) Test – The Unit Root Test
H_0: The series has a unit root (Non-Stationary). H_1: The series is stationary.
The ADF regression:
ΔY_t = α + β*t + γ*Y_{t-1} + Σ_{i=1}^{p} δ_i ΔY_{t-i} + ε_t
If the test statistic for γ is less than the critical value (p < 0.05), we reject H_0 and conclude stationarity.

2.2 The KPSS Test – The Reverse Test
H_0: The series is stationary. H_1: The series has a unit root.
Use both ADF and KPSS. If ADF rejects and KPSS does not → Stationary. If ADF does not reject and KPSS rejects → Non-Stationary. If both reject → Deterministic trend.

2.3 Transformations to Enforce Stationarity

• Differencing: ∇Y_t = Y_t - Y_{t-1}. For prices, 1st difference gives returns. For log-prices, 1st difference gives log-returns.

• Log Transformation: ln(Y_t) stabilises variance.

• Box-Cox Transform: Y_t^* = (Y_t^λ - 1)/λ for λ ≠ 0, and ln(Y_t) for λ = 0. The optimal λ is found via MLE.

---

3. Autocorrelation and Partial Autocorrelation – The Memory of Markets

These functions tell us if past returns predict future returns (momentum/reversal).

3.1 Autocorrelation Function (ACF)
ρ_k = Cov(Y_t, Y_{t-k}) / Var(Y_t).
For an i.i.d. series, ρ_k ≈ 0 for all k ≠ 0. The Ljung-Box Q-test tests the null that all autocorrelations up to lag m are zero:
Q = T(T+2) Σ_{k=1}^{m} (ρ_k² / (T-k)) ~ χ²_m.
If p < 0.05, significant serial correlation exists – your AI must use lagged features.

3.2 Partial Autocorrelation Function (PACF)
PACF measures the correlation between Y_t and Y_{t-k}} after removing the effects of intermediate lags (Y_{t-1}, ..., Y_{t-k+1}). It is the coefficient of Y_{t-k} in a linear regression of Y_t on all previous lags.

AI Application: The ACF and PACF of returns guide the p and q orders for ARIMA models. For an AI model, these functions tell you the minimum number of lagged returns to include as features. If PACF cuts off after lag 5, include exactly 5 lagged returns.

---

4. ARIMA Models – The Classical Baseline

The AutoRegressive Integrated Moving Average (ARIMA) model is the gold standard for univariate financial forecasting. It serves as an excellent benchmark to compare against your deep learning model.

4.1 Mathematical Formulation
An ARIMA(p, d, q) model:
(1 - Σ_{i=1}^{p} φ_i L^i) * (1-L)^d * Y_t = (1 + Σ_{j=1}^{q} θ_j L^j) * ε_t

• L is the lag operator: L^k Y_t = Y_{t-k}.

• p = AR order (lags of the dependent variable).

• d = degree of differencing.

• q = MA order (lags of the error term).

• ε_t ~ White Noise (0, σ²).

4.2 Interpretation of Parameters

• AR(1) model: Y_t = φ_1 Y_{t-1} + ε_t. If φ_1 > 0, momentum; if φ_1 < 0, mean reversion.

• MA(1) model: Y_t = ε_t + θ_1 ε_{t-1}. Captures shocks that persist for exactly one period (e.g., bid-ask bounce).

4.3 SARIMA – Seasonal ARIMA
SARIMA(p,d,q)(P,D,Q)_s, where s is the seasonal period (e.g., s=5 for weekly seasonality in daily returns, s=12 for monthly). It includes seasonal lags like Y_{t-s}}.

AI Integration: You should fit an ARIMA model on your training set, record its 1-step-ahead prediction Y_hat_{t+1}, and use Y_hat_{t+1} as a feature in your deep learning model. The neural network learns to weight and correct the ARIMA forecast.

---

5. Cointegration – The Holy Grail of Statistical Arbitrage

Two non-stationary series are cointegrated if a linear combination of them is stationary. This implies a long-run equilibrium relationship.

5.1 Engle-Granger Two-Step Method (Single Cointegrating Vector)

1. Regress Y_t on X_t using OLS: Y_t = β_0 + β_1 X_t + z_t. The residual z_t is the “spread.”

2. Test z_t for stationarity using the ADF test (without a trend). If z_t is stationary (p < 0.05), the pair is cointegrated.

3. The spread z_t is a powerful mean-reverting feature for AI.

5.2 Johansen Test (Multiple Vectors)
For N assets, there can be up to N-1 cointegrating vectors. Johansen uses a VAR-based approach to test the rank r of the cointegration matrix. The trace statistic and maximum eigenvalue statistic determine r.

AI Application: Train a neural network to predict the future value of the spread z_{t+1}. If the model predicts the spread will widen, the strategy goes long the cheap asset and short the expensive one. This is a classic Pairs Trading AI.

---

6. Volatility Modelling – The ARCH/GARCH Family

Since volatility is unobservable but is the primary input for risk management, we model it as a time-varying process.

6.1 ARCH(q) – AutoRegressive Conditional Heteroskedasticity
Let ε_t = σ_t * v_t, where v_t ~ N(0,1).
The conditional variance equation:
σ_t² = ω + Σ_{i=1}^{q} α_i ε_{t-i}².
High past squared errors predict high future variance (volatility clustering).

6.2 GARCH(p, q) – The Generalised Form
σ_t² = ω + Σ_{i=1}^{q} α_i ε_{t-i}² + Σ_{j=1}^{p} β_j σ_{t-j}².
The β terms capture the persistence of volatility shocks. The sum Σ α_i + Σ β_j measures the “volatility half-life.” If the sum is close to 1, shocks are permanent (IGARCH).

6.3 EGARCH – Asymmetric GARCH (Leverage Effect)
Markets react asymmetrically: bad news (negative returns) increases volatility more than good news.
ln(σ_t²) = ω + Σ α_i * |ε_{t-i}/σ_{t-i}| + Σ γ_i * (ε_{t-i}/σ_{t-i}) + Σ β_j ln(σ_{t-j}²).
The γ term captures the asymmetry. If γ < 0, negative shocks increase volatility more (the leverage effect).

6.4 GJR-GARCH (Threshold GARCH)
σ_t² = ω + α ε_{t-1}² + γ * I_{t-1} * ε_{t-1}² + β σ_{t-1}², where I_{t-1} = 1 if ε_{t-1} < 0, else 0.

AI Integration: Use the one-step-ahead conditional variance σ_{t+1}² from a GARCH model as a feature. This provides the neural network with a model-based volatility forecast, which is better than a simple rolling standard deviation.

---

7. Vector Autoregression (VAR) – Multivariate Dynamics

When we have K endogenous time series (e.g., returns of 10 sectors), we use VAR.

7.1 Mathematical Formulation (VAR(p))
Y_t = c + Φ_1 Y_{t-1} + Φ_2 Y_{t-2} + ... + Φ_p Y_{t-p} + ε_t
Where:

• Y_t is a (K x 1) vector.

• c is a (K x 1) vector of intercepts.

• Φ_i are (K x K) matrices of coefficients.

• ε_t ~ N(0, Σ) (contemporaneous correlation).

7.2 Granger Causality
Variable X “Granger-causes” Y if lagged values of X improve the prediction of Y beyond using lagged values of Y alone. Test via F-test on the coefficients of X in the VAR equation for Y.

AI Application: Before feeding 50 macro series into a Transformer, run Granger causality tests. Remove all variables that do not Granger-cause your target asset. This drastically reduces input dimensionality and prevents the Transformer from learning spurious correlations.

---

8. Cointegration vs. Correlation – A Critical Distinction

Correlation is a contemporaneous measure of co-movement. It is transient and can change sign.
Cointegration is a long-term equilibrium relationship. It is invariant to short-term noise.

AI Rule: If you are building a pairs trading strategy, never use Pearson correlation to select pairs. Use cointegration (Engle-Granger). Correlation can be high during a bull market but the spread can drift apart infinitely. Cointegration ensures the spread is mean-reverting.

---

9. From Econometrics to Deep Learning – The Theoretical Bridge

Understanding these time-series models reveals the inductive biases of neural networks:

---

10. Summary for the AI Practitioner

1. Stationarity is non-negotiable. Always run ADF and KPSS. Difference non-stationary features. Apply Box-Cox to variance-stabilise.

2. ACF/PACF tell you the minimum lag context required. Use them to set the seq_length for your LSTM/Transformer.

3. ARIMA is your baseline. If your LSTM doesn’t beat a simple ARIMA on the same task, stop and debug your feature engineering.

4. Cointegration is the mathematical foundation for relative value strategies (pairs trading). Use the spread z_t as a core feature.

5. GARCH extracts the conditional variance, which is a better volatility feature than rolling standard deviation.

6. Granger Causality is a filter. Use it to prune irrelevant features before training expensive deep learning models.

7. Always benchmark: If an econometric model (ARIMA/GARCH) provides a given Sharpe, your AI model must significantly exceed it to justify the operational complexity.

In Lesson 1.6, we will move from classical time-series to Advanced Financial Data Sources. We will cover alternative data (satellite imagery, credit card transactions, social media sentiment), tick-by-tick order book data (Limit Order Books), and the mathematical challenges of handling irregularly spaced, ultra-high-frequency data using point processes and Hawkes processes.