1. Learning Objectives

By the end of this lesson, you will be able to:

  • Formally define a probability space (Ω, F, P) and interpret its components in a market context.

  • Distinguish between discrete, continuous, and mixed random variables and derive their respective density and mass functions.

  • Derive and interpret the key theoretical distributions underlying financial returns (Normal, Log-Normal, Student-t, and Binomial).

  • Compute and interpret the four standardised moments (Mean, Variance, Skewness, Kurtosis) and explain their economic meaning for risk.

  • Construct joint, marginal, and conditional distributions, and prove the Law of Total Expectation and Law of Total Variance.

  • Apply the Central Limit Theorem (CLT) and understand its critical failure points in high-frequency finance.

  • Define Value at Risk (VaR) and Expected Shortfall (ES) purely from a probabilistic quantile perspective.

2. The Probability Space – The Formal Architecture of Uncertainty

Before we model prices, we must model uncertainty. In rigorous finance, we operate within a probability space defined by the triplet (Ω, F, P).

  • Ω (Sample Space): The set of all possible future states of the world. In finance, Ω is unimaginably large (e.g., all possible paths of the S&P 500 over the next year).

  • F (Sigma-Algebra): The collection of all observable events (subsets of Ω) to which we can assign a probability. In AI terms, this represents the information set available to our model. As we process more data (features), our information set F grows (filtered filtration).

  • P (Probability Measure): A function P: F → [0, 1] that assigns a real number to every event, satisfying the Kolmogorov Axioms:

    1. Non-negativity: P(A) ≥ 0 for all A ∈ F.

    2. Unitarity: P(Ω) = 1 (something must happen).

    3. Additivity: For mutually disjoint events A_1, A_2, ... (where A_i ∩ A_j = ∅), P(∪_{i=1}^{∞} A_i) = Σ_{i=1}^{∞} P(A_i).

Crucial AI Insight: When we train a neural network to predict a price movement, we are implicitly learning a parametric approximation of the conditional probability measure P(Price_rise | Features). The network’s softmax output or regression loss is an attempt to map from the feature space (a subset of F) to a probability in [0,1].

3. Random Variables – The Translation from Events to Numbers

A random variable (RV) X is a deterministic function X: Ω → R that assigns a real number to every state of the world. We don’t care about the states themselves; we care about the distribution of X.

3.1 Discrete Random Variables

If X takes a countable set of values {x_1, x_2, ...}. Defined by the Probability Mass Function (PMF):
p_X(x_i) = P(X = x_i), where Σ_i p_X(x_i) = 1.
Finance Example: A Bernoulli trial for a binary credit default. Let D = 1 if a firm defaults in 1 year, D = 0 otherwise. P(D=1) = p (Probability of Default, PD).

3.2 Continuous Random Variables

If X takes uncountably infinite values (e.g., a stock price). Defined by the Probability Density Function (PDF) f_X(x), which satisfies:

  1. f_X(x) ≥ 0 for all x ∈ R.

  2. ∫_{-∞}^{∞} f_X(x) dx = 1.

  3. P(a ≤ X ≤ b) = ∫_{a}^{b} f_X(x) dx.

The Cumulative Distribution Function (CDF) is the integral of the PDF:
F_X(x) = P(X ≤ x) = ∫_{-∞}^{x} f_X(t) dt
By the Fundamental Theorem of Calculus: f_X(x) = d/dx F_X(x).
Crucial AI Insight: AI models often output the parameters of a distribution (e.g., a mean μ and variance σ²) rather than the point estimate. This is the core of Distributional Reinforcement Learning and Probabilistic Forecasting in finance.

4. The “Big Four” Distributions in Financial AI

Every AI model’s performance hinges on the distributional assumptions (or lack thereof) built into its loss function. Here are the mathematical formulations you must internalise.

4.1 The Normal (Gaussian) Distribution – X ~ N(μ, σ²)

The foundational distribution due to the CLT. PDF:
f(x) = (1 / (σ * sqrt(2π))) * exp( - (x - μ)^2 / (2σ^2) ) for -∞ < x < ∞.

  • μ = mean (location parameter). σ = standard deviation (scale parameter).

  • Moment Generating Function (MGF): M_X(t) = E[exp(tX)] = exp( μt + (σ² t²)/2 ).

  • Standard Normal: Z = (X - μ)/σ ~ N(0,1).

4.2 The Log-Normal Distribution – Y ~ LogN(μ, σ²)

If Y is log-normally distributed, then X = ln(Y) ~ N(μ, σ²). This is the canonical distribution for asset prices because prices cannot go below zero (limited liability). PDF:
f(y) = (1 / (y σ sqrt(2π))) * exp( - (ln(y) - μ)^2 / (2σ^2) ), for y > 0.

Critical derived moments (you must memorize these for option pricing):

  • Mean (Expected Price): E[Y] = exp( μ + σ²/2 )

  • Variance: Var(Y) = exp( 2μ + σ² ) * ( exp(σ²) - 1 )

  • Skewness: Positive (right-skewed). The mean is pulled higher than the median due to the lower bound at zero.

AI Application: If your neural network predicts stock prices directly (absolute levels), using Mean Squared Error (MSE) implicitly assumes a symmetric distribution. However, prices are Log-Normal; thus, logarithmic transformation ln(Price) is almost always performed before feeding into an AI to stabilise variance and make the distribution more Gaussian.

4.3 The Student’s t-Distribution – X ~ t_ν(μ, σ²)

This is the workhorse for risk management. It features “fat tails” (excess kurtosis). PDF:
f(x) = ( Γ((ν+1)/2) / ( Γ(ν/2) * sqrt(νπ) * σ ) ) * ( 1 + (x-μ)^2 / (ν σ^2) )^{-(ν+1)/2}

  • ν = degrees of freedom. As ν → ∞, the t-distribution converges to the Normal.

  • For ν ≤ 2, the variance is infinite (undefined). For ν ≤ 1, the mean is undefined.

  • Why Finance Loves It: Empirical returns exhibit kurtosis > 3. The t-distribution captures this, leading to more accurate tail-risk calculations (VaR).

4.4 The Binomial Distribution – X ~ Bin(n, p)

The discrete building block for option pricing (Cox-Ross-Rubinstein model). If X is the number of successes in n independent Bernoulli trials:
P(X = k) = C(n, k) * p^k * (1-p)^{n-k}, where C(n, k) = n! / (k! (n-k)!).

  • Mean: E[X] = np

  • Variance: Var(X) = np(1-p)

5. Moments of a Distribution – The Quantitative Fingerprint

Moments summarise the shape of a distribution. AI models primarily learn to match the first moment (conditional mean), but advanced models (GANs, normalising flows) match all moments.

5.1 Raw Moments vs Central Moments

  • k-th Raw Moment (about zero): μ'_k = E[ X^k ].

  • k-th Central Moment (about the mean μ): μ_k = E[ (X - μ)^k ].

5.2 The Four Standardised Measures

Let μ = E[X] and σ = sqrt( Var(X) ).

  1. Mean (1st Central Moment – 0): Measures central tendency. In finance, this is the expected return. E[R].

  2. Variance (2nd Central Moment): Measures total risk (dispersion). Var(X) = E[ (X-μ)^2 ] = E[X²] - μ².

    • Proof: Expand (X-μ)^2 = X² - 2μX + μ². Taking expectation: E[X²] - 2μE[X] + μ² = E[X²] - 2μ² + μ² = E[X²] - μ².

  3. Skewness (3rd Standardised Moment): Measures asymmetry. Skew = E[ ((X-μ)/σ)^3 ].

    • If Skew < 0: Left-skewed (fat left tail, high crash risk). Equity returns often exhibit slight negative skewness.

    • If Skew > 0: Right-skewed (frequent small losses, rare massive gains). Options strategies often have positive skew.

  4. Kurtosis (4th Standardised Moment): Measures tail heaviness. Kurt = E[ ((X-μ)/σ)^4 ].

    • For a Normal distribution, Kurtosis = 3. Excess Kurtosis = Kurt - 3.

    • If Excess Kurtosis > 0 (Leptokurtic): Fat tails and a sharp central peak. This is the reality of financial data.

6. Joint, Marginal, and Conditional Probabilities (The Core of Multivariate AI)

In a portfolio context, we never deal with a single asset. We have an N-dimensional vector of returns.

6.1 Joint CDF and PDF

For two continuous variables X and Y:
F_{XY}(x, y) = P(X ≤ x, Y ≤ y) = ∫_{-∞}^{y} ∫_{-∞}^{x} f_{XY}(u, v) du dv
The joint PDF: f_{XY}(x, y) = ∂²/∂x∂y F_{XY}(x, y).

6.2 Marginal Distribution

To get the distribution of a single variable, we integrate out the other:
f_X(x) = ∫_{-∞}^{∞} f_{XY}(x, y) dy
AI Insight: Autoencoders often perform non-linear marginalisation by projecting high-dimensional financial data (yield curves) onto a lower-dimensional latent space.

6.3 Conditional Distribution and Bayes’ Theorem (The AI Engine)

The distribution of Y given that X = x is:
f_{Y|X}(y|x) = f_{XY}(x, y) / f_X(x), provided f_X(x) > 0.
This directly leads to Bayes’ Theorem, which is the mathematical foundation for filtering (Kalman filters) and Bayesian neural networks:
P(θ | Data) = P(Data | θ) * P(θ) / P(Data)

  • P(θ) = Prior belief about model parameters.

  • P(Data | θ) = Likelihood (what the data tells us).

  • P(θ | Data) = Posterior (updated belief).

6.4 Conditional Expectation (The Best Predictor)

The conditional expectation E[Y | X = x] is the expected value of Y given that X is known. In AI terms, a regression neural network f(X) trained with MSE converges to E[Y | X]. This is the minimum mean square error (MMSE) estimator.

7. The Law of Total Expectation and Law of Total Variance (Risk Decomposition)

These are the most underrated equations in financial AI. They allow us to decompose predictable and unpredictable components of risk.

7.1 Law of Total Expectation (LTE) – “The Tower Rule”

The unconditional expectation of a variable equals the expectation of its conditional expectation:
E[Y] = E[ E[Y | X] ]
Proof (Discrete): E[Y] = Σ_x E[Y | X=x] * P(X=x) = Σ_x (Σ_y y * P(Y=y|X=x)) * P(X=x) = Σ_x Σ_y y * P(Y=y, X=x) = Σ_y y * P(Y=y) = E[Y].
Finance Implication: If an AI model gives a conditional expected return of 5% in a “bull regime” and -2% in a “bear regime”, the overall expected return is the probability-weighted average of these.

7.2 Law of Total Variance (LTV) – The Source of “Aleatoric vs Epistemic”

The total variance of Y is the variance of the conditional expectation plus the expected conditional variance:
Var(Y) = E[ Var(Y | X) ] + Var( E[Y | X] )

  • E[ Var(Y | X) ] (Irreducible Noise): The average variance within each regime. This is the aleatoric uncertainty (data noise) that no amount of data can reduce.

  • Var( E[Y | X] ) (Model-able Component): The variance between regimes. This is the epistemic uncertainty that our AI model can capture with better features.

AI Application: When your validation loss plateaus, you are hitting the E[Var(Y|X)] floor. To improve, you must add new features X to change the conditional expectation and increase the explanatory power of Var(E[Y|X]).

8. Convergence Concepts: LLN and CLT (Why Backtests Work and Fail)

8.1 Law of Large Numbers (LLN)

If X_1, X_2, ..., X_n are i.i.d. with finite mean μ, then the sample average converges almost surely to the expected value as n → ∞:
\bar{X}_n = (1/n) Σ_{i=1}^{n} X_i → μ
AI Implication: In backtesting, we rely on the LLN to think that the average return observed in our 10-year backtest will approximate the true expected return. The catch: Financial returns are not i.i.d. (they are autocorrelated and heteroskedastic). Therefore, backtested Sharpe ratios are biased estimators.

8.2 Central Limit Theorem (CLT) – The “Pivot”

Given i.i.d. variables with mean μ and variance σ² < ∞, the standardised sample mean converges in distribution to a standard normal:
sqrt(n) * ( \bar{X}_n - μ ) / σ → N(0, 1) (in distribution).
The Finance Fallacy: The CLT justifies the use of normal distributions for aggregated returns (e.g., monthly returns). However, convergence is only valid for σ² < ∞. For heavy-tailed distributions (like Cauchy), σ² = ∞, and the CLT fails completely. This is why extreme market events (Black Swans) occur far more frequently than Gaussian VaR models predict.

9. Quantile Functions and Probabilistic Risk Metrics

AI models are increasingly used to forecast the entire distribution, not just the mean. This is done via Quantile Regression (e.g., Pinball Loss).

9.1 The Quantile Function (Inverse CDF)

For a probability α ∈ (0,1), the α-quantile is:
q_α = F_X^{-1}(α) = inf { x ∈ R : F_X(x) ≥ α }
This means there is an α% probability that the return will be below q_α.

9.2 Value at Risk (VaR)

VaR at confidence level α is simply the negative of the α-quantile of the loss distribution. For the return R:
VaR_α = -q_α, where q_α is the quantile of the return distribution.
Example: If the 5%-quantile of daily returns is -2%, then the daily VaR_95% = 2%. We are 95% confident we won’t lose more than 2% today.

9.3 Expected Shortfall (ES) – Coherent Risk Measure

VaR tells us the threshold of loss, but not the expected loss if that threshold is breached. ES (also known as Conditional VaR) does this:
ES_α = E[ -R | -R > VaR_α ] = -(1/α) * ∫_{0}^{α} q_β dβ
AI Application: Neural networks with a Quantile Loss (Tilted Absolute Error) can directly output q_0.05 and q_0.95. A single network with multiple output nodes can map features to the entire quantile function, enabling AI-driven portfolio tail-risk hedging.

10. The Stochastic Process View (Linking to Time Series)

A financial time series {P_t} is not a sequence of independent draws; it is a stochastic process. For AI, we must define:

  1. Martingale Property: E[ P_{t+1} | F_t ] = P_t. This means prices are unpredictable from past information (Efficient Market Hypothesis). AI models bet against this by trying to find predictive features.

  2. Stationarity (Weak): A process is weakly stationary if E[P_t] = μ (constant), Var(P_t) = σ² (constant), and Cov(P_t, P_{t-k}) = γ_k (only depends on lag k, not on t).

    • Price levels P_t are non-stationary (they trend upward).

    • Returns r_t = ln(P_t/P_{t-1}) are stationary (they fluctuate around a constant mean).

    • AI Rule: Never feed raw price levels into a deep learning model without detrending or differentiating, because the non-stationarity will cause the neural network’s weight gradients to explode or vanish over long sequences.

11. Summary for the AI Practitioner

This lesson provides the mathematical “operating system” for your models:

  1. Random Variables are how you translate market states into numbers for PyTorch/TensorFlow.

  2. Log-Normal is for prices; Normal is for log-returns (approximately); t-Distribution is for robust risk estimation.

  3. Conditional Expectation E[Y|X] is precisely what your regression model approximates. The Law of Total Variance explicitly quantifies the maximum potential improvement your model can achieve.

  4. The CLT is a double-edged sword—it justifies conventional statistics but fails spectacularly during crises (fat tails), which is why you must use robust loss functions (Huber, Quantile).

  5. Quantile functions move you from point forecasts to distributional forecasts, which is mandatory for regulatory capital calculations (Basel III/IV).

In Lesson 1.3, we will operationalise these probabilistic concepts into Statistical Inference and Hypothesis Testing, covering Maximum Likelihood Estimation (MLE), A/B testing in trading strategies, and the p-value pitfalls specific to financial backtesting. We will derive the MLE estimator explicitly and prove why standard errors in finance are notoriously understated.

 
 
PreviousNext