1. Learning Objectives

By the end of this lesson, you will be able to:

  • Derive the mathematical formulations of classic technical indicators (moving averages, RSI, MACD, Bollinger Bands) and understand their statistical properties.

  • Formulate the construction of financial alpha factors (momentum, value, quality, volatility) using cross-sectional and time-series data.

  • Apply transformations for handling non-stationarity, scaling, and orthogonalisation to create robust features.

  • Incorporate alternative data sources (satellite imagery, sentiment, ESG) into a unified feature set.

  • Evaluate the predictive power of features using information coefficients (IC), Sharpe ratio of decile portfolios, and cross-validation.

2. Returns and Transformations: The Foundation

Before building any indicator, we need to standardise the return computation. The raw price series P_t is non-stationary. The primary transformation is the log return:

r_t = ln(P_t / P_{t-1})

We can also define multi-period returns:

r_t(k) = ln(P_t / P_{t-k}) = ∑_{i=0}^{k-1} r_{t-i}

For modelling, we often need volatility – the standard deviation of returns. Volatility is typically estimated using an exponential weighted moving average (EWMA) or a GARCH model. The EWMA volatility is defined as:

σ_t^2 = (1-λ) * r_t^2 + λ * σ_{t-1}^2

where λ is the decay factor (commonly 0.94 for daily data). This is the RiskMetrics approach. The half-life of the volatility estimate is ln(0.5)/ln(λ).

Z-score normalisation: To compare features across different scales, we standardise using a rolling window of length L:

z_t = (x_t - μ_t) / σ_t

where μ_t and σ_t are the rolling mean and standard deviation. This transforms the feature into a stationary process with zero mean and unit variance, facilitating the use of many machine learning models.

3. Classic Technical Indicators: Mathematical Derivations

Technical indicators are deterministic functions of historical price and volume data. They are widely used as features for AI models because they encapsulate complex market dynamics in a single numeric series.

3.1 Simple and Exponential Moving Averages (SMA & EMA)

The SMA of price over window n at time t is:

SMA_t(n) = (1/n) * ∑_{i=0}^{n-1} P_{t-i}

This gives equal weight to all observations. The EMA assigns exponentially decreasing weights:

EMA_t(n) = (P_t - EMA_{t-1}(n)) * α + EMA_{t-1}(n)

where α = 2/(n+1) is the smoothing factor. The EMA responds more quickly to recent price changes. The relationship between SMA and EMA is that the EMA can be viewed as an IIR filter with impulse response h_k = α * (1-α)^k. The gain of the EMA at frequency zero is 1, and its cut-off frequency is approximately α/2π.

A common feature is the price-momentum crossover:
Momentum = EMA_t(short) / EMA_t(long) - 1

This is a trending indicator; positive values indicate upward momentum.

3.2 Relative Strength Index (RSI)

The RSI is a momentum oscillator that measures the speed and change of price movements. It is computed as:

Let Up_t = max(r_t, 0) and Down_t = max(-r_t, 0). Compute the average gains and losses over a window n (e.g., 14 days):

AvgGain_t = EMA(Up_t, n)
AvgLoss_t = EMA(Down_t, n)

Then the Relative Strength (RS) is:
RS_t = AvgGain_t / AvgLoss_t

And the RSI is:
RSI_t = 100 - 100 / (1 + RS_t) = 100 * AvgGain_t / (AvgGain_t + AvgLoss_t)

The RSI ranges from 0 to 100. Values above 70 are considered overbought, and below 30 oversold. The statistical interpretation of RSI is that it is a non-linear transformation of the ratio of recent positive to negative returns, effectively a non-parametric measure of the probability of positive versus negative returns.

3.3 MACD (Moving Average Convergence Divergence)

The MACD is a trend-following momentum indicator. It is defined as:

MACD_t = EMA_t(12) - EMA_t(26) (using 12 and 26 days as standard).

A signal line is the EMA of the MACD, usually over 9 days:
Signal_t = EMA(MACD_t, 9)

The histogram is the difference:
Histogram_t = MACD_t - Signal_t

This is equivalent to a band-pass filter. The MACD line emphasises short-term momentum relative to long-term momentum. Crossovers of the MACD above the signal line generate buy signals. In a feature set, we can use the MACD, the signal line, and the histogram as separate features, or simply the normalised MACD value.

3.4 Bollinger Bands

Bollinger Bands are volatility bands placed above and below a moving average. They are defined as:

Middle Band_t = SMA_t(n)
Upper Band_t = Middle Band_t + k * σ_t
Lower Band_t = Middle Band_t - k * σ_t

where σ_t is the standard deviation of the price over the same n periods, and k is a multiplier (typically 2).

The Bollinger Bandwidth is a feature: BBW_t = (Upper - Lower) / Middle, which measures the relative volatility. The %B indicator is:

%B_t = (P_t - Lower) / (Upper - Lower)

This shows where the price is relative to the bands (0 below lower band, 1 above upper band). Statistically, %B can be seen as a normalised measure of price relative to its recent range and volatility.

4. Factor Investing: The Modern Alpha Framework

Beyond simple technicals, institutional investment management uses factor models. Factors are common sources of risk and return. The classic Fama-French 3-factor model expands the CAPM:

E[R_i] - R_f = β_i (E[R_m] - R_f) + s_i * SMB + h_i * HML

  • SMB (Small Minus Big): return spread between small and large market capitalization stocks.

  • HML (High Minus Low): return spread between high book-to-market (value) and low book-to-market (growth) stocks.

Later models add Momentum (WML – winners minus losers), Quality (profitable, stable firms), and Low Volatility.

To construct these factors as features for a single stock (or portfolio), we can compute:

  • Value factor: Book-to-Market ratio = Book Value / Market Capitalization.

  • Momentum factor: 12-month trailing return excluding the most recent month (to avoid short-term reversal).

  • Size factor: Log of Market Cap.

  • Volatility factor: σ_ann = sqrt(252) * rolling_std(returns, 60).

  • Liquidity factor: Amihud illiquidity = (1/T) * ∑ |r_t| / V_t (the average absolute return per dollar volume). Higher values indicate lower liquidity.

For cross-sectional models (predicting returns of many stocks), we rank these factors within a universe and use the ranked values as features. A common transformation is the cross-sectional z-score:

z_{i,t} = (x_{i,t} - median_t(x_t)) / mad_t(x_t)

where mad is the median absolute deviation, which is robust to outliers.

5. Alternative Data Sources and Feature Extraction

5.1 Textual and Sentiment Data

As covered in Module 6, sentiment scores from news, social media, and earnings calls are powerful features. We can aggregate daily sentiment to generate features:

  • Daily_Net_Sentiment = (∑ positive_scores - ∑ negative_scores) / total_mentions

  • Sentiment_Volatility = rolling standard deviation of net sentiment.

  • Sentiment_Surprise = difference between actual sentiment and the prior day’s expected sentiment (modelled from historical patterns).

5.2 Satellite and Geospatial Data

For retail or logistics companies, satellite imagery of parking lots or shipping ports can be used to estimate foot traffic or inventory levels. The feature extraction pipeline involves:

  1. Image processing: Use computer vision models (e.g., ResNet, YOLO) to count cars or containers.

  2. Time-series aggregation: Smooth the daily counts with a 7-day moving average to remove weekly seasonality.

  3. Relative change: Δ_activity = (count_t - count_{t-30}) / count_{t-30}.

  4. Input to model: Use the transformed activity as a predictive variable for future revenue or earnings surprises.

5.3 ESG (Environmental, Social, Governance) Data

ESG scores are increasingly important. Features can be:

  • Total ESG score (normalised 0-100).

  • Pillar scores: E, S, G individually.

  • Trend: ESG_t / ESG_{t-12} (year-over-year improvement).

  • Controversy score: A high score indicates recent negative events.

These features are often predictive of long-term performance and risk, especially during market downturns.

5.4 Macroeconomic Indicators

For global equities, macro features such as:

  • Yield curve slope: 10-year Treasury yield - 2-year Treasury yield.

  • Inflation rate: CPI annual change.

  • PMI (Purchasing Managers’ Index): A diffusion index for manufacturing activity.

  • VIX (Volatility Index): Market’s expectation of 30-day volatility.

These can be lagged by one day (or one month) to avoid look-ahead bias and used as features.

6. Feature Engineering for High-Frequency Data

In high-frequency trading (HFT), we have access to the LOB features. Popular HFT features include:

  • Imbalance at depth: I_t(d) = (V_t^B(d) - V_t^A(d)) / (V_t^B(d) + V_t^A(d)), where V_t^B(d) is the cumulative bid volume up to depth d.

  • Order flow imbalance: OFI_t = (B_t - S_t) - (A_t - B_t) over a small time interval.

  • Spread components: Spread_t = A_t^1 - B_t^1. Spread_ratio_t = Spread_t / M_t.

  • Volume-weighted average price (VWAP) of the last minute.

  • Trade direction: The side of the last trade (buy vs sell). A simple feature is Trade_Direction = +1 if last trade is at ask, -1 if at bid.

Microstructural invariance: Recent research suggests that many of these features can be transformed to be “invariant” across assets using scaling laws (e.g., dividing by volatility and average trading volume). This improves model generalisation.

7. Evaluating Features: The Information Coefficient (IC)

The Information Coefficient (IC) is a measure of the predictive power of a feature. It is the correlation between the feature at time t and the future return over a horizon h:

IC_{t,h}(x) = corr( x_{i,t}, r_{i, t+h} ) across all assets i at a given time t (cross-sectional IC), or across time for a single asset (time-series IC).

We typically compute the rank IC (Spearman correlation) to be robust to outliers. A rank IC above 0.02 (in absolute value) is considered economically significant. The ICIR (IC Information Ratio) is the mean IC divided by its standard deviation; a high ICIR (> 0.5) indicates a stable, predictable feature.

Decile portfolio test:
Sort stocks by the feature value, create 10 portfolios (deciles), compute the average return of each decile over the next period, and calculate the monotonicity (spread between the top and bottom decile). This is a classic quantitative finance test. The t-statistic of the long-short portfolio return (top minus bottom) indicates whether the feature can generate significant alpha.

8. Handling Data Snooping and Overfitting

Given the large number of potential features (thousands), there is a severe risk of overfitting. Techniques to mitigate this:

  • Hold-out validation: Use a strict time-based split (e.g., train 2000-2010, validation 2011-2015, test 2016-2020). Never use future data.

  • Feature selection: Use LASSO regression or tree-based feature importance to select a parsimonious set.

  • Orthogonalisation: Remove the common factors (e.g., market, size, value) from each feature. This ensures the feature captures idiosyncratic information.

  • Combinatorial Purged Cross-Validation: A technique by Marcos Lopez de Prado that ensures cross-validation folds do not contain overlapping observations (time-series data), preventing leakage.

9. Summary for the AI Practitioner

  • Technical indicators (SMA, EMA, RSI, MACD, Bollinger Bands) are deterministic transforms that provide rich, non-linear representations of price and volatility.

  • Factor models (Fama-French, momentum, quality) provide a systematic framework for constructing features that capture systematic risk premia.

  • Alternative data (sentiment, satellite, ESG) requires domain-specific feature extraction pipelines and careful alignment (time-stamping) to avoid look-ahead bias.

  • Feature evaluation should be rigorous: use IC, ICIR, and decile portfolio tests; always perform time-series cross-validation.

  • Feature standardisation and orthogonalisation are critical for stable ML models.

10. References

  1. Fama, E. F., & French, K. R. (1993). Common risk factors in the returns on stocks and bonds. Journal of Financial Economics.

  2. Wilder, J. W. (1978). New Concepts in Technical Trading Systems. Trend Research.

  3. Murphy, J. J. (1999). Technical Analysis of the Financial Markets. New York Institute of Finance.

  4. Lopez de Prado, M. (2018). Advances in Financial Machine Learning. John Wiley & Sons.

  5. Amihud, Y. (2002). Illiquidity and stock returns. Journal of Financial Markets.

  6. Jegadeesh, N., & Titman, S. (1993). Returns to buying winners and selling losers. Journal of Finance.

 
PreviousNext