LESSON 5: ADVANCED TIME SERIES FORECASTING – ARIMA, SARIMA, AND PROPHET

SECTION 1: LEARNING OBJECTIVES

By the end of this lesson, you will be able to:

Understand the limitations of traditional regression when applied to time series data and why specialised models are required.
Define and apply the ARIMA (AutoRegressive Integrated Moving Average) framework – one of the most widely used classical time series forecasting methods in finance.
Understand the three components of ARIMA: AR (AutoRegressive), I (Integrated/Differencing), and MA (Moving Average).
Use the ACF and PACF plots to identify the orders of AR and MA components ( $p$ and $q$ ).
Apply the Augmented Dickey‑Fuller (ADF) test to determine the required order of differencing ( $d$ ).
Extend ARIMA to SARIMA to handle seasonality in financial data (e.g., quarterly earnings, holiday effects).
Apply Facebook Prophet – a modern, scalable forecasting tool designed for business time series with strong seasonal patterns and holiday effects.
Evaluate forecast accuracy using metrics such as MAE, RMSE, MAPE, and MASE.
Understand the business applications of time series forecasting in banking: loan demand forecasting, deposit growth, interest rate predictions, and economic scenario analysis.

SECTION 2: WHY TIME SERIES MODELS ARE DIFFERENT

In Lesson 6 of Module 3, we introduced time series concepts. Now we build predictive models that explicitly account for temporal dependence.

Key challenges in time series forecasting:

Challenge	Description	Financial Example
Autocorrelation	Observations are correlated with their past values.	Today’s stock return is correlated with yesterday’s.
Trend	Long‑term upward or downward movement.	Bank deposits grow over decades.
Seasonality	Regular patterns within a fixed period.	Credit card spending peaks in December.
Non‑stationarity	Statistical properties change over time.	Volatility changes during crises.
External shocks	Unexpected events that permanently alter the series.	COVID‑19 impact on loan defaults.

Why linear regression fails:

Violates the independence assumption (autocorrelation leads to inefficient estimates).
Cannot capture time‑dependent dynamics.
Standard errors are underestimated (spurious significance).

SECTION 3: THE ARIMA FRAMEWORK – MATHEMATICAL FOUNDATION

ARIMA stands for AutoRegressive Integrated Moving Average. It is denoted as ARIMA(p, d, q) where:

p = order of the AutoRegressive (AR) component
d = degree of differencing (to achieve stationarity)
q = order of the Moving Average (MA) component

3.1 The AR Component (AutoRegressive)

An AR model of order $p$ assumes that the current value depends linearly on its previous $p$ values:

$y_{t} = c + ϕ_{1} y_{t - 1} + ϕ_{2} y_{t - 2} + \dots + ϕ_{p} y_{t - p} + ε_{t}$

$ϕ_{1}, ϕ_{2}, \dots, ϕ_{p}$ are the AR coefficients.
$ε_{t}$ is white noise (random error).

Interpretation: If $ϕ_{1} = 0.5$ , then 50% of yesterday’s value persists into today.

3.2 The MA Component (Moving Average)

An MA model of order $q$ assumes that the current value depends linearly on the previous $q$ forecast errors:

$y_{t} = μ + θ_{1} ε_{t - 1} + θ_{2} ε_{t - 2} + \dots + θ_{q} ε_{t - q} + ε_{t}$

$θ_{1}, θ_{2}, \dots, θ_{q}$ are the MA coefficients.
$μ$ is the mean of the series.

Interpretation: MA models capture “shock” effects that persist for a short period (e.g., a one‑off market event).

3.3 The I Component (Integration/Differencing)

Differencing removes trend and makes the series stationary. The first difference is:

$y_{t'} = y_{t} - y_{t - 1}$

If the series has a linear trend, first differencing $d = 1$ is sufficient. If it has a quadratic trend, $d = 2$ may be needed.

The complete ARIMA equation:

$(1 - i = 1 \sum p ϕ_{i} L^{i}) (1 - L)^{d} y_{t} = (1 + j = 1 \sum q θ_{j} L^{j}) ε_{t}$

where $L$ is the lag operator ( $L y_{t} = y_{t - 1}$ ).

SECTION 4: IDENTIFYING ARIMA ORDERS – ACF AND PACF

The Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF) are the primary tools for identifying $p$ and $q$ .

Component	ACF Pattern	PACF Pattern	Order
AR(p)	Decays exponentially or sinusoidally	Cuts off after lag $p$	$p$ = lag where PACF cuts off
MA(q)	Cuts off after lag $q$	Decays exponentially	$q$ = lag where ACF cuts off
ARMA(p, q)	Decays after lag $q - p$	Decays after lag $p - q$	More complex; use AIC/BIC

Additional rules of thumb:

The ADF test determines $d$ (the minimum number of differences for stationarity).
AIC (Akaike Information Criterion) and BIC (Bayesian Information Criterion) help choose the best model among candidates:
- AIC = -2 log(L) + 2k (penalises complexity lightly)
- BIC = -2 log(L) + k log(n) (penalises complexity more strongly)

SECTION 5: SARIMA – HANDLING SEASONALITY

SARIMA (Seasonal ARIMA) extends ARIMA by adding seasonal components. It is denoted SARIMA(p, d, q)(P, D, Q)_m where:

P = seasonal AR order
D = seasonal differencing order
Q = seasonal MA order
m = number of periods per season (e.g., 12 for monthly data, 4 for quarterly)

Financial examples of seasonality:

Monthly retail sales (holiday peaks)
Quarterly earnings (end‑of‑year accruals)
Daily transaction volumes (weekend vs. weekday)
Weekly stock returns (Monday effect)

The SARIMA equation extends ARIMA with seasonal lags:

$Φ_{P} (L^{m}) ϕ_{p} (L) (1 - L^{m})^{D} (1 - L)^{d} y_{t} = Θ_{Q} (L^{m}) θ_{q} (L) ε_{t}$

$Φ_{P}$ is the seasonal AR polynomial.
$Θ_{Q}$ is the seasonal MA polynomial.

SECTION 6: FACEBOOK PROPHET – A MODERN ALTERNATIVE

Prophet, developed by Facebook, is designed for business forecasting where:

Strong seasonality exists (daily, weekly, yearly).
Holidays and special events impact the series.
There are missing data points and outliers.
The forecast needs to be interpretable and adjustable by non‑experts.

Prophet decomposes the time series into three components:

$y (t) = g (t) + s (t) + h (t) + ε_{t}$

$g (t)$ = Trend (piecewise linear or logistic growth with changepoints).
$s (t)$ = Seasonality (Fourier series for yearly, weekly, daily patterns).
$h (t)$ = Holiday effects (user‑specified special days).
$ε_{t}$ = Error term.

Advantages of Prophet:

Handles missing data and outliers robustly.
Fast fitting and forecasting.
Intuitive hyperparameters (changepoint_prior_scale, seasonality_prior_scale).
Built‑in uncertainty intervals.

Limitations in finance:

Assumes relatively clean, regular time series (less suitable for high‑frequency financial data).
May not capture complex autoregressive dynamics as well as ARIMA/SARIMA.
Less interpretable from a statistical perspective (but easier for business users).

SECTION 7: IMPLEMENTATION IN PYTHON – ARIMA, SARIMA, AND PROPHET

We’ll apply these methods to forecast a bank’s monthly deposit volume.

# ===================================================================
# MODULE 4, LESSON 5: ADVANCED TIME SERIES FORECASTING
# ===================================================================

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from statsmodels.tsa.stattools import adfuller, acf, pacf
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf
from statsmodels.tsa.arima.model import ARIMA
from statsmodels.tsa.statespace.sarimax import SARIMAX
from sklearn.metrics import mean_absolute_error, mean_squared_error
from prophet import Prophet
import warnings
warnings.filterwarnings('ignore')

# Set style and reproducibility
np.random.seed(42)
sns.set_style("whitegrid")

print("="*70)
print("ADVANCED TIME SERIES FORECASTING – BANK DEPOSIT VOLUME")
print("="*70)

# ----------------------------------------------------------------
# PART A: GENERATE SYNTHETIC BANK DEPOSIT DATA
# ----------------------------------------------------------------

# Generate 5 years of monthly data (60 months)
n_months = 60
dates = pd.date_range(start='2019-01-01', periods=n_months, freq='MS')

# Components:
# 1. Trend: gradual growth of 0.5% per month
trend = np.linspace(100, 100 * (1.005)**n_months, n_months)

# 2. Seasonality: annual pattern (higher in Dec, lower in Jan)
seasonal = 20 * np.sin(2 * np.pi * np.arange(n_months) / 12 + 0.5)

# 3. Cyclical: economic cycle (2-year cycle)
cycle = 15 * np.sin(2 * np.pi * np.arange(n_months) / 24 + 1)

# 4. Random noise
noise = np.random.normal(0, 10, n_months)

# Combine
deposits = trend + seasonal + cycle + noise + 100
deposits = np.maximum(deposits, 50)  # No negative values

# Create DataFrame
df = pd.DataFrame({
    'date': dates,
    'deposits': deposits
})
df.set_index('date', inplace=True)

print("\nDataset shape:", df.shape)
print("\nFirst 12 months:")
print(df.head(12).round(2))
print("\nSummary Statistics:")
print(df.describe().round(2))

# Visualise the series
fig, axes = plt.subplots(2, 1, figsize=(14, 10))

ax = axes[0]
ax.plot(df.index, df['deposits'], 'b-', linewidth=1.5)
ax.set_title('Bank Deposits – Monthly Time Series', fontsize=14)
ax.set_ylabel('Deposits ($ millions)')
ax.grid(True, alpha=0.3)

# Decompose components (visual)
ax = axes[1]
components = pd.DataFrame({
    'Trend': trend,
    'Seasonal': seasonal,
    'Cycle': cycle,
    'Noise': noise
}, index=df.index)
components.plot(ax=ax)
ax.set_title('Time Series Components', fontsize=14)
ax.set_ylabel('Contribution')
ax.legend()
ax.grid(True, alpha=0.3)

plt.tight_layout()
plt.savefig('time_series_data.png', dpi=300)
plt.show()

# Split into train (first 48 months) and test (last 12 months)
train = df.iloc[:-12]
test = df.iloc[-12:]

print(f"\nTraining set: {len(train)} months")
print(f"Test set: {len(test)} months")

# ----------------------------------------------------------------
# PART B: STATIONARITY CHECK (ADF Test)
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART B: STATIONARITY CHECK (Augmented Dickey-Fuller Test)")
print("-"*60)

def adf_test(series, title=''):
    """Perform ADF test and print results."""
    result = adfuller(series, autolag='AIC')
    print(f"\nADF Test for {title}:")
    print(f"  Test Statistic: {result[0]:.4f}")
    print(f"  P-Value: {result[1]:.4f}")
    print(f"  Critical Values:")
    for key, value in result[4].items():
        print(f"    {key}: {value:.4f}")
    if result[1] < 0.05:
        print(f"  Conclusion: Series is STATIONARY (reject H0).")
    else:
        print(f"  Conclusion: Series is NON-STATIONARY (fail to reject H0).")
    return result

# Test original series
adf_test(train['deposits'], 'Original Deposits')

# First differencing
train_diff = train['deposits'].diff().dropna()
adf_test(train_diff, 'First Difference')

# Determine d
d = 1  # First difference makes it stationary
print(f"\nChosen order of differencing (d): {d}")

# ----------------------------------------------------------------
# PART C: ACF and PACF Plots (to determine p and q)
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART C: ACF and PACF Analysis")
print("-"*60)

fig, axes = plt.subplots(1, 2, figsize=(14, 5))

# ACF
plot_acf(train_diff, ax=axes[0], lags=20, alpha=0.05)
axes[0].set_title('ACF – First Difference', fontsize=12)

# PACF
plot_pacf(train_diff, ax=axes[1], lags=20, alpha=0.05)
axes[1].set_title('PACF – First Difference', fontsize=12)

plt.tight_layout()
plt.savefig('acf_pacf_analysis.png', dpi=300)
plt.show()

print("\nInterpretation:")
print("  - ACF cuts off after lag 1? → MA(1) or MA(q)")
print("  - PACF cuts off after lag 1? → AR(1) or AR(p)")
print("  - For this series, we see: ACF decays slowly, PACF cuts off at lag 1.")
print("  - Suggests AR(1) or ARMA(1,1) model.")

# ----------------------------------------------------------------
# PART D: ARIMA MODEL FITTING AND FORECASTING
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART D: ARIMA Model Fitting")
print("-"*60)

# Based on ACF/PACF, try ARIMA(1,1,0) - AR model
model_arima = ARIMA(train['deposits'], order=(1, 1, 0))
fitted_arima = model_arima.fit()

print("\nARIMA(1,1,0) Model Summary:")
print(fitted_arima.summary())

# Diagnostics
print("\nModel Diagnostics:")
print(f"  AIC: {fitted_arima.aic:.2f}")
print(f"  BIC: {fitted_arima.bic:.2f}")
print(f"  Log-Likelihood: {fitted_arima.llf:.2f}")

# Check residuals
residuals = fitted_arima.resid
print(f"\nResidual Diagnostics:")
print(f"  Mean of residuals: {residuals.mean():.4f}")
print(f"  Std of residuals: {residuals.std():.4f}")

# Ljung-Box test for autocorrelation in residuals
from statsmodels.stats.diagnostic import acorr_ljungbox
lb_test = acorr_ljungbox(residuals, lags=10, return_df=True)
print(f"\nLjung-Box Test (p-values):")
print(lb_test['lb_pvalue'].tail(1))

# ----------------------------------------------------------------
# PART E: SARIMA WITH SEASONALITY
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART E: SARIMA (with Seasonality)")
print("-"*60)

# SARIMA(1,1,1)(1,0,1,12) – monthly data with yearly seasonality
model_sarima = SARIMAX(
    train['deposits'],
    order=(1, 1, 1),
    seasonal_order=(1, 0, 1, 12),
    enforce_stationarity=False,
    enforce_invertibility=False
)
fitted_sarima = model_sarima.fit(disp=False)

print("\nSARIMA(1,1,1)(1,0,1,12) Model Summary:")
print(fitted_sarima.summary())

print(f"\nModel Comparison:")
print(f"  ARIMA(1,1,0) AIC: {fitted_arima.aic:.2f}")
print(f"  SARIMA AIC:       {fitted_sarima.aic:.2f}")
print(f"  Lower AIC indicates better model (SARIMA is better).")

# ----------------------------------------------------------------
# PART F: FORECASTING AND EVALUATION
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART F: Forecasting and Evaluation")
print("-"*60)

# Forecast horizon (12 months)
forecast_steps = len(test)

# ARIMA forecast
forecast_arima = fitted_arima.forecast(steps=forecast_steps)
forecast_arima_index = test.index

# SARIMA forecast
forecast_sarima = fitted_sarima.forecast(steps=forecast_steps)

# Evaluate
def evaluate_forecast(actual, forecast, name):
    """Calculate forecast error metrics."""
    actual = np.array(actual)
    forecast = np.array(forecast)
    mae = mean_absolute_error(actual, forecast)
    rmse = np.sqrt(mean_squared_error(actual, forecast))
    mape = np.mean(np.abs((actual - forecast) / actual)) * 100
    
    print(f"\n{name} Forecast Evaluation:")
    print(f"  MAE:  {mae:.2f}")
    print(f"  RMSE: {rmse:.2f}")
    print(f"  MAPE: {mape:.2f}%")
    return {'MAE': mae, 'RMSE': rmse, 'MAPE': mape}

results_arima = evaluate_forecast(test['deposits'], forecast_arima, "ARIMA(1,1,0)")
results_sarima = evaluate_forecast(test['deposits'], forecast_sarima, "SARIMA")

# Visualise forecasts
fig, ax = plt.subplots(figsize=(14, 7))

# Plot historical data
ax.plot(df.index, df['deposits'], 'b-', linewidth=1.5, label='Historical')

# Plot training/test split
ax.axvline(train.index[-1], color='black', linestyle='--', alpha=0.7, label='Train/Test Split')

# Plot forecasts
ax.plot(forecast_arima_index, forecast_arima, 'r--', linewidth=2, label='ARIMA Forecast')
ax.plot(forecast_arima_index, forecast_sarima, 'g--', linewidth=2, label='SARIMA Forecast')
ax.plot(test.index, test['deposits'], 'bo', markersize=8, label='Actual (Test)')

ax.set_title('Time Series Forecast Comparison', fontsize=14)
ax.set_xlabel('Date')
ax.set_ylabel('Deposits ($ millions)')
ax.legend()
ax.grid(True, alpha=0.3)

plt.tight_layout()
plt.savefig('forecast_comparison.png', dpi=300)
plt.show()

# ----------------------------------------------------------------
# PART G: FACEBOOK PROPHET
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART G: Facebook Prophet Forecast")
print("-"*60)

# Prophet requires columns 'ds' (date) and 'y' (value)
prophet_df = df.reset_index()
prophet_df.columns = ['ds', 'y']

# Split into train and test
prophet_train = prophet_df.iloc[:-12]
prophet_test = prophet_df.iloc[-12:]

# Initialise and fit Prophet model
model_prophet = Prophet(
    yearly_seasonality=True,
    weekly_seasonality=False,
    daily_seasonality=False,
    changepoint_prior_scale=0.05,
    seasonality_prior_scale=10.0,
    interval_width=0.95
)
model_prophet.fit(prophet_train)

# Create future dataframe
future = model_prophet.make_future_dataframe(periods=12, freq='MS')
forecast_prophet = model_prophet.predict(future)

# Extract forecast
prophet_forecast = forecast_prophet[['ds', 'yhat', 'yhat_lower', 'yhat_upper']].tail(12)
prophet_forecast.set_index('ds', inplace=True)

# Evaluate
results_prophet = evaluate_forecast(test['deposits'], prophet_forecast['yhat'], "Prophet")

# Visualise Prophet forecast
fig = model_prophet.plot(forecast_prophet)
plt.title('Prophet Forecast – Bank Deposits', fontsize=14)
plt.xlabel('Date')
plt.ylabel('Deposits ($ millions)')
plt.savefig('prophet_forecast.png', dpi=300)
plt.show()

# Plot components
fig2 = model_prophet.plot_components(forecast_prophet)
plt.savefig('prophet_components.png', dpi=300)
plt.show()

# ----------------------------------------------------------------
# PART H: COMPREHENSIVE MODEL COMPARISON
# ----------------------------------------------------------------

print("\n" + "="*70)
print("PART H: MODEL COMPARISON SUMMARY")
print("="*70)

comparison_results = pd.DataFrame({
    'Model': ['ARIMA(1,1,0)', 'SARIMA', 'Prophet'],
    'MAE': [results_arima['MAE'], results_sarima['MAE'], results_prophet['MAE']],
    'RMSE': [results_arima['RMSE'], results_sarima['RMSE'], results_prophet['RMSE']],
    'MAPE (%)': [results_arima['MAPE'], results_sarima['MAPE'], results_prophet['MAPE']],
    'Interpretability': ['High', 'High', 'Medium'],
    'Computational Speed': ['Fast', 'Medium', 'Fast'],
    'Seasonality Handling': ['Manual', 'Built-in', 'Built-in']
})
print(comparison_results.to_string(index=False))

print("\n Business Recommendation:")
print("  - Use ARIMA for simple, non-seasonal series where interpretability is key.")
print("  - Use SARIMA for seasonal series (e.g., monthly retail, quarterly earnings).")
print("  - Use Prophet when you have holidays, missing data, or need an intuitive interface.")
print("  - In banking, SARIMA is preferred for regulatory submissions due to its statistical rigour.")

SECTION 8: BUSINESS APPLICATIONS IN BANKING

Application	Recommended Model	Rationale
Deposit Growth Forecasting	SARIMA	Monthly data with strong year‑end seasonality.
Loan Demand Prediction	SARIMA / Prophet	Seasonal patterns (e.g., higher demand in spring for mortgages).
Non-Performing Loan (NPL) Ratio	ARIMA	Tends to be less seasonal; focus on autoregressive dynamics.
Net Interest Margin (NIM) Forecasting	SARIMA	Quarterly data with clear seasonal patterns.
Transaction Volume (ATM/POS)	Prophet	Daily data with weekly seasonality; holiday effects (Christmas, Ramadan).
Regulatory Capital Forecasting	ARIMA	Conservative approach; focus on trend rather than seasonality.
Stress Testing (CCAR/DFAST)	Multiple models	Combine ARIMA, SARIMA, and Prophet for scenario analysis.

SECTION 9: SUMMARY FOR THE DATA PRACTITIONER

ARIMA is the classical workhorse for time series forecasting, combining AR, I (differencing), and MA components.
ACF and PACF identify the orders $p$ and $q$ ; ADF test determines $d$ .
SARIMA extends ARIMA with seasonal components – essential for financial data with annual, quarterly, or monthly patterns.
Prophet is a modern, user‑friendly alternative that handles holidays, missing data, and business‑specific seasonality.
Evaluation metrics (MAE, RMSE, MAPE) guide model selection.
In banking, SARIMA is the preferred regulatory‑friendly approach, while Prophet is excellent for internal business forecasting.

SECTION 10: RECOMMENDED NEXT STEPS

Apply ARIMA/SARIMA to a real‑world dataset (e.g., Federal Reserve economic data, stock prices).
Experiment with auto‑ARIMA (pmdarima) for automatic order selection.
Learn about GARCH models for volatility forecasting (essential for risk management).
Explore Deep Learning for Time Series (LSTM, Transformer) – covered in later modules.
Understand the regulatory requirements for forecast validation in CCAR/DFAST.

[END OF LESSON 5 – MODULE 4]

Diploma in Financial Data Analytics LESSON 5: ADVANCED TIME SERIES FORECASTING – ARIMA, SARIMA, AND PROPHET

Diploma in Financial Data Analytics