1. LEARNING OBJECTIVES

By the end of this lesson, you will be able to:

  • Define random variables and distinguish between discrete and continuous types.

  • Compute and interpret the Probability Density Function (PDF), Cumulative Distribution Function (CDF), and Quantile Function.

  • Compute the expected value, variance, and higher moments of random variables.

  • Derive and apply the Moment Generating Function (MGF) and Characteristic Function (CF).

  • Construct joint distributions, marginal distributions, and conditional distributions.

  • Understand and apply the Law of Total Probability and Bayes’ Theorem.

  • Compute the distribution of sums of independent random variables using convolution.

  • Identify and apply key distributions used in finance (Normal, Log-Normal, Poisson, Exponential, Gamma, Chi-Square, and Stable distributions).

  • Apply the Central Limit Theorem to approximate sums of random variables.

2. RANDOM VARIABLES – DEFINITION AND TYPES

2.1 Definition

A random variable X is a function from the sample space Ω to the real numbers R:

X: Ω → R

2.2 Types of Random Variables

  • Discrete Random Variable: Takes on a countable number of values (e.g., number of defaults, stock price movements measured in ticks).

  • Continuous Random Variable: Takes on an uncountable number of values in an interval (e.g., stock returns, interest rates).

2.3 Distribution Function (CDF)

For any random variable X, the Cumulative Distribution Function (CDF) is:

F_X(x) = P(X ≤ x)

Properties:

  1. F_X is non-decreasing.

  2. F_X is right-continuous.

  3. lim_{x→-∞} F_X(x) = 0 and lim_{x→∞} F_X(x) = 1.

2.4 Probability Density Function (PDF)

For a continuous random variable X, the PDF is:

f_X(x) = dF_X(x) / dx

Properties:

  1. f_X(x) ≥ 0 for all x.

  2. ∫_{-∞}^{∞} f_X(x) dx = 1.

  3. P(a ≤ X ≤ b) = ∫_a^b f_X(x) dx.

2.5 Quantile Function (Inverse CDF)

The quantile function (or percentile function) is the inverse of the CDF:

Q(p) = F_X^{-1}(p) = inf{ x ∈ R : F_X(x) ≥ p }

This is used for calculating Value at Risk (VaR).

Financial Application – Value at Risk (VaR):
The 95% VaR is the 5th percentile of the return distribution:

VaR_{95%} = -Q(0.05)

3. EXPECTED VALUE, VARIANCE, AND MOMENTS

3.1 Expected Value (Mean)

For a discrete random variable X with probability mass function p(x):

E[X] = Σ_{x} x * p(x)

For a continuous random variable X with PDF f_X(x):

E[X] = ∫_{-∞}^{∞} x * f_X(x) dx

3.2 Expectation of a Function of X

For any function g(X):

E[g(X)] = ∫_{-∞}^{∞} g(x) * f_X(x) dx

3.3 Variance

Var(X) = E[(X – E[X])²] = E[X²] – (E[X])²

3.4 Standard Deviation

σ_X = sqrt(Var(X))

3.5 Higher Moments

  • Third Central Moment: E[(X – E[X])³] → Measures skewness.

  • Fourth Central Moment: E[(X – E[X])⁴] → Measures kurtosis.

Skewness:

Skewness = E[(X – E[X])³] / σ³

  • Skewness = 0: Symmetric distribution (Normal).

  • Skewness > 0: Right-skewed (positive skew).

  • Skewness < 0: Left-skewed (negative skew).

Kurtosis:

Kurtosis = E[(X – E[X])⁴] / σ⁴

  • Normal Distribution: Kurtosis = 3.

  • Excess Kurtosis = Kurtosis – 3.

  • Excess Kurtosis > 0: Fat tails (heavy tails).

4. MOMENT GENERATING FUNCTION (MGF) AND CHARACTERISTIC FUNCTION (CF)

4.1 Moment Generating Function (MGF)

The MGF of a random variable X is:

M_X(t) = E[e^{tX}]

Properties:

  1. M_X(0) = 1.

  2. The nth moment is E[X^n] = M_X^{(n)}(0) (the nth derivative at t = 0).

  3. If X and Y are independent, M_{X+Y}(t) = M_X(t) * M_Y(t).

4.2 Characteristic Function (CF)

The CF of a random variable X is the Fourier transform of the PDF:

φ_X(t) = E[e^{itX}]

Where i = √(-1).

Properties:

  1. φ_X(0) = 1.

  2. |φ_X(t)| ≤ 1.

  3. The CF uniquely determines the distribution (inversion theorem).

  4. If X and Y are independent, φ_{X+Y}(t) = φ_X(t) * φ_Y(t).

4.3 Why the Characteristic Function is Preferred

The CF always exists (even when the MGF does not). This is because e^{itX} is bounded (|e^{itX}| = 1), whereas e^{tX} can grow exponentially.

4.4 Inversion Theorem

The PDF can be recovered from the CF:

f_X(x) = (1/2π) ∫_{-∞}^{∞} e^{-itx} φ_X(t) dt

5. JOINT, MARGINAL, AND CONDITIONAL DISTRIBUTIONS

5.1 Joint Distribution

For two random variables X and Y, the joint CDF is:

F_{X,Y}(x, y) = P(X ≤ x, Y ≤ y)

The joint PDF (for continuous variables) is:

f_{X,Y}(x, y) = ∂²F_{X,Y}(x, y) / ∂x∂y

5.2 Marginal Distribution

The marginal distribution of X is obtained by integrating out Y:

f_X(x) = ∫{-∞}^{∞} f{X,Y}(x, y) dy

5.3 Conditional Distribution

The conditional PDF of X given Y = y is:

f_{X|Y}(x | y) = f_{X,Y}(x, y) / f_Y(y), provided f_Y(y) > 0

5.4 Independence

X and Y are independent if:

f_{X,Y}(x, y) = f_X(x) * f_Y(y)

Equivalently:

F_{X,Y}(x, y) = F_X(x) * F_Y(y)

6. THE LAW OF TOTAL PROBABILITY AND BAYES’ THEOREM

6.1 Law of Total Probability

For a partition {A_i} of the sample space:

P(B) = Σ_i P(B | A_i) * P(A_i)

In continuous form:

f_X(x) = ∫ f_{X|Y}(x | y) * f_Y(y) dy

6.2 Bayes’ Theorem

P(A | B) = P(B | A) * P(A) / P(B)

In continuous form:

f_{X|Y}(x | y) = f_{Y|X}(y | x) * f_X(x) / f_Y(y)

6.3 Financial Application – Parameter Estimation

Bayes’ theorem is the foundation of Bayesian inference in finance:

Posterior ∝ Likelihood * Prior

7. CONVOLUTION – THE DISTRIBUTION OF SUMS

7.1 Definition

If X and Y are independent continuous random variables with PDFs f_X and f_Y, the PDF of Z = X + Y is:

f_Z(z) = ∫_{-∞}^{∞} f_X(x) * f_Y(z – x) dx

7.2 The Convolution of Normals

If X ~ N(μ_X, σ_X²) and Y ~ N(μ_Y, σ_Y²) are independent, then:

Z = X + Y ~ N(μ_X + μ_Y, σ_X² + σ_Y²)

7.3 Financial Application – Portfolio Returns

The return of a portfolio is the weighted sum of individual asset returns:

R_p = Σ_{i=1}^n w_i R_i

If each R_i is normally distributed, then R_p is also normally distributed with:

μ_p = Σ w_i μ_i
σ_p² = Σ_{i} Σ_{j} w_i w_j σ_{ij}

This is the foundation of Markowitz portfolio theory.

8. KEY DISTRIBUTIONS IN FINANCE

8.1 Normal Distribution (Gaussian)

PDF:

f_X(x) = [1 / (σ √(2π))] * exp( -(x – μ)² / (2σ²) )

MGF:

M_X(t) = exp( μt + (1/2) σ² t² )

CF:

φ_X(t) = exp( iμt – (1/2) σ² t² )

Properties:

  • Symmetric (skewness = 0).

  • Kurtosis = 3.

  • Tails decay exponentially fast (thin tails).

  • Used for asset returns (approximately) and Brownian motion.

Financial Application: Black-Scholes model assumes log-normal stock prices (which means normal log-returns).

8.2 Log-Normal Distribution

If Y = ln(X) ~ N(μ, σ²), then X is log-normally distributed.

PDF:

f_X(x) = [1 / (x σ √(2π))] * exp( -(ln(x) – μ)² / (2σ²) ), for x > 0

Mean:

E[X] = exp( μ + σ²/2 )

Variance:

Var(X) = exp(2μ + σ²) * (exp(σ²) – 1)

Financial Application: Stock prices are assumed to be log-normally distributed (positive prices, multiplicative returns).

8.3 Poisson Distribution

For a discrete random variable X with rate λ:

P(X = k) = (e^{-λ} λ^k) / k!, for k = 0, 1, 2, …

Mean: E[X] = λ
Variance: Var(X) = λ

Financial Application: Number of defaults in a portfolio, number of trades in a time interval, number of operational risk events.

8.4 Exponential Distribution

PDF:

f_X(x) = λ e^{-λx}, for x ≥ 0

CDF:

F_X(x) = 1 – e^{-λx}, for x ≥ 0

Mean: E[X] = 1/λ
Variance: Var(X) = 1/λ²

Financial Application: Time between defaults, inter-arrival times for trades.

8.5 Gamma Distribution

PDF:

f_X(x) = (1 / Γ(α)) * β^α * x^{α-1} * e^{-βx}, for x ≥ 0

Where Γ(α) is the Gamma function.

Mean: E[X] = α/β
Variance: Var(X) = α/β²

Special Cases:

  • α = 1: Exponential distribution.

  • α = n/2, β = 1/2: Chi-Square distribution with n degrees of freedom.

Financial Application: Sum of squared normal variables (Chi-Square) is used in hypothesis testing and for testing the significance of regression coefficients.

8.6 Chi-Square Distribution (χ²)

If Z_1, Z_2, …, Z_n are independent standard normal variables, then:

X = Σ_{i=1}^n Z_i² ~ χ²(n)

PDF:

f_X(x) = [1 / (2^{n/2} Γ(n/2))] * x^{(n/2) – 1} * e^{-x/2}

Mean: E[X] = n
Variance: Var(X) = 2n

Financial Application: Testing the significance of portfolio performance (Jensen’s alpha), testing for autocorrelation (Ljung-Box test).

8.7 Stable Distributions

A distribution is stable if the sum of two independent copies of the distribution has the same shape (up to scaling and shifting).

Characteristic Function of a Stable Distribution:

φ_X(t) = exp( iμt – γ^α |t|^α [1 + iβ sign(t) Φ(t, α)] )

Where:

  • α ∈ (0, 2] is the tail index (stability parameter).

  • β ∈ [-1, 1] is the skewness parameter.

  • γ > 0 is the scale parameter.

  • μ ∈ R is the location parameter.

Special Cases:

  • α = 2: Normal distribution (β is irrelevant).

  • α = 1, β = 0: Cauchy distribution.

  • α = 1, β = 1: Lévy distribution.

Financial Application: Asset returns exhibit fat tails that are better captured by stable distributions than the normal distribution.

9. THE CENTRAL LIMIT THEOREM (CLT) AND ITS IMPLICATIONS

9.1 The Theorem

Let X_1, X_2, …, X_n be independent and identically distributed (i.i.d.) random variables with mean μ and variance σ² < ∞. Then:

(1/√n) Σ_{i=1}^n (X_i – μ) → N(0, σ²)

Equivalently:

(1/n) Σ_{i=1}^n X_i → N(μ, σ²/n)

9.2 Implications for Finance

  1. Portfolio Returns: A well-diversified portfolio of n assets has returns that are approximately normal, regardless of the distribution of individual asset returns.

  2. Risk Aggregation: Aggregated risk measures (e.g., total loss from a portfolio of loans) are approximately normal.

  3. Asset Pricing: The CLT justifies the assumption of normality in many asset pricing models.

9.3 Limitations of the CLT in Finance

  1. Tail Dependence: The CLT assumes independence, but financial returns exhibit tail dependence (crashes and contagion).

  2. Infinite Variance: Some financial returns have infinite variance (e.g., Pareto-distributed losses), violating the CLT assumption.

  3. Convergence Rate: The convergence to normality can be slow for skewed or fat-tailed distributions.

9.4 The Lindeberg-Feller CLT (for Non-Identical Distributions)

If X_1, X_2, …, X_n are independent but not identically distributed, the CLT still holds under the Lindeberg condition:

lim_{n→∞} (1/s_n²) Σ_{i=1}^n E[ (X_i – μ_i)² * 1_{|X_i – μ_i| > ε s_n} ] = 0

Where s_n² = Σ_{i=1}^n σ_i². This condition ensures that no single variable dominates the sum.

10. PRACTICAL IMPLEMENTATION

A. Computing Moments and Distributions:

python
import numpy as np
from scipy import stats
import matplotlib.pyplot as plt

# Generate normal data
data = np.random.normal(loc=0, scale=1, size=10000)

# Compute moments
mean = np.mean(data)
var = np.var(data)
skew = stats.skew(data)
kurt = stats.kurtosis(data)

print(f"Mean: {mean:.4f}")
print(f"Variance: {var:.4f}")
print(f"Skewness: {skew:.4f}")
print(f"Excess Kurtosis: {kurt:.4f}")

# Compute PDF and CDF
x = np.linspace(-4, 4, 100)
pdf = stats.norm.pdf(x, 0, 1)
cdf = stats.norm.cdf(x, 0, 1)

# Quantile function (VaR)
VaR_95 = stats.norm.ppf(0.05, 0, 1)
print(f"95% VaR: {VaR_95:.4f}")

B. Fitting a Distribution to Data:

python
# Fit a normal distribution to data
mu_fit, sigma_fit = stats.norm.fit(data)
print(f"Fitted μ: {mu_fit:.4f}, Fitted σ: {sigma_fit:.4f}")

# Fit a stable distribution
alpha_fit, beta_fit, gamma_fit, delta_fit = stats.levy_stable.fit(data)
print(f"Fitted α: {alpha_fit:.4f}, β: {beta_fit:.4f}")

C. Verifying the Central Limit Theorem:

python
# Generate exponential data (skewed)
data = np.random.exponential(scale=1, size=(1000, 100))

# Compute sample means
sample_means = np.mean(data, axis=1)

# Plot histogram of sample means
plt.hist(sample_means, bins=50, density=True)
x = np.linspace(0, 2, 100)
plt.plot(x, stats.norm.pdf(x, 1, 1/np.sqrt(100)), 'r-')
plt.show()

D. Convolution of Independent Variables:

python
# Sum of two normals
X = np.random.normal(0, 1, 10000)
Y = np.random.normal(2, 0.5, 10000)
Z = X + Y

# Check the mean and variance
print(f"Mean of Z: {np.mean(Z):.4f}")
print(f"Variance of Z: {np.var(Z):.4f}")
print(f"Expected: μ = 2, σ² = 1.25")
 
PreviousNext