INTRODUCTION: BEYOND STATIC RULES

In Lessons 8.1 through 8.3, we built a deterministic fraud detection system: rules, watchlists, and anomaly detection thresholds. This system is effective against known fraud patterns. However, fraudsters are adaptive. They will probe the system, discover the rules, and modify their behaviour to evade detection. A rule-based system can never catch a fraud pattern that it has not been explicitly programmed to detect.

Machine Learning (ML) for fraud detection solves this problem. Instead of writing rules, we train a model on historical fraud data. The model learns the patterns of fraud directly from the data, without explicit programming. When a new transaction arrives, the model outputs a fraud probability. The model can detect emerging fraud patterns (patterns that are novel but similar to previously seen fraud) and evolving fraud patterns (patterns that have mutated).

This lesson deconstructs the ML pipeline for fraud detection in Open Banking. We define the feature engineering process: converting raw transaction data (amount, timestamp, payee, device fingerprint, consent velocity) into features for the model. We use Supervised Learning (XGBoost, Random Forest) when we have labelled fraud data (we know which transactions are fraudulent). We use Unsupervised Learning (Isolation Forest, Autoencoders) when we do not have labelled data (we are looking for outliers). We derive the mathematical foundations of each algorithm: XGBoost (gradient boosting with a logistic loss function), Random Forest (bagging of decision trees, ensemble averaging), Isolation Forest (partitioning the feature space, isolating anomalies), and Autoencoders (neural networks that reconstruct the input; high reconstruction error indicates fraud). We also address the class imbalance problem: fraud transactions are rare (typically < 1% of all transactions). We implement SMOTE (Synthetic Minority Over-sampling Technique) to generate synthetic fraud samples, and we use cost-sensitive learning (assigning a higher penalty to misclassifying fraud). We quantify the model performance using the Precision-Recall curve (since the class is imbalanced) and the F1-score (harmonic mean of Precision and Recall). We also design the model serving architecture: the model is deployed as a microservice (e.g., using TensorFlow Serving, ONNX Runtime) that scores each transaction in real time, with a latency budget of < 10ms (p95).

LEARNING OBJECTIVES

1. Define the Feature Engineering Pipeline—converting raw transaction data into features: (1) Aggregate Features (sum, count, average of payments in the last hour), (2) Time-based Features (hour of day, day of week), (3) Graph Features (degree centrality of the payer and payee), (4) Device Features (fingerprint hash, IP geolocation), and (5) Consent Features (consent age, remaining days). We will prove that the feature vector has 50+ dimensions.

2. Implement Supervised Learning (XGBoost and Random Forest) —deriving the XGBoost objective function: L(θ) = Σ_l(y_i, ŷ_i) + Σ Ω(f_k), where l is the logistic loss, Ω is the regularization term. We will prove that the gradient boosting algorithm iteratively fits trees to the negative gradient of the loss function. We will also derive the Random Forest algorithm: training B decision trees on bootstrapped samples and averaging their predictions.

3. Implement Unsupervised Learning (Isolation Forest and Autoencoders) —deriving the Isolation Forest algorithm: randomly partition the feature space until each data point is isolated; the number of partitions (path length) is the anomaly score. We will prove that anomalies (fraud) have shorter path lengths. We will also derive the Autoencoder: a neural network that compresses the input to a lower-dimensional representation (latent space) and reconstructs it; the reconstruction error is the anomaly score. We will prove that the Autoencoder learns to reconstruct normal transactions well but fails to reconstruct fraudulent transactions.

4. Address the Class Imbalance—applying SMOTE (Synthetic Minority Over-sampling) to generate synthetic fraud samples: for each fraud sample, find its k-nearest neighbours (k=5), generate a new sample along the line segment between the sample and a random neighbour. We will also implement cost-sensitive learning by assigning a higher weight to fraud samples in the loss function: weight = (1 - p) / p where p is the fraud rate.

5. Evaluate the Model Performance—plotting the Precision-Recall curve and calculating the F1-score. We will prove that for an imbalanced dataset, the F1-score is a better metric than accuracy.

6. Design the Model Serving Architecture—deploying the trained model as a microservice with a latency budget of < 10ms (p95). We will use ONNX Runtime (optimized inference) or TensorFlow Serving. We will design the feature store (Redis) that pre-computes the features for each transaction, reducing the inference time.

7. Implement Model Retraining and Drift Detection—defining a retraining schedule (daily or weekly), and implementing model drift detection: monitoring the distribution of the model’s predictions (the AUC over a sliding window), and triggering a retraining when the AUC drops below a threshold (e.g., 0.05 drop).

PART 1: FEATURE ENGINEERING — The 50+ Dimensional Feature Vector

Feature engineering is the process of transforming raw transaction data into a structured feature vector that the ML model can consume.

Raw Transaction Data:

• amount, timestamp, payee_id, payer_id, device_fingerprint, consent_id, ip_address, user_agent.

Engineered Features (Categories) :

Total Features: ~50 dimensions.

Feature Normalization:
All numerical features are standardised to have mean 0 and standard deviation 1. Categorical features are one-hot encoded.

PART 2: SUPERVISED LEARNING — XGBoost and Random Forest

2.1 XGBoost (Extreme Gradient Boosting)

XGBoost is a gradient-boosted decision tree algorithm that sequentially adds trees to correct the errors of the previous trees.

Objective Function:
L(θ) = Σ_i l(y_i, ŷ_i) + Σ_k Ω(f_k)

Where:

• l is the loss function (for binary classification, the logistic loss: l = -[y_i log(ŷ_i) + (1 - y_i) log(1 - ŷ_i)]).

• Ω(f_k) = γ T + (λ / 2) ||w||² is the regularization term (penalises complex trees).

Gradient Boosting:
At each iteration t, we add a new tree f_t that fits the negative gradient of the loss with respect to the previous prediction.

g_i = ∂L / ∂ŷ_{i}^{(t-1)} (gradient)
h_i = ∂²L / ∂(ŷ_{i}^{(t-1)})² (Hessian)

The optimal weight for a leaf is:
w_j = - G_j / (H_j + λ)

Where G_j = Σ g_i and H_j = Σ h_i over the leaf.

Training:

• We train XGBoost on historical fraud data (10,000 fraud samples, 100,000 legitimate samples).

• We use 5-fold cross-validation to tune the hyper-parameters: max_depth = 6, learning_rate = 0.1, n_estimators = 100.

2.2 Random Forest

Random Forest is an ensemble of decision trees, each trained on a bootstrapped sample (bagging) and a random subset of features.

Training:

• For B trees (B=100), sample n observations with replacement (bootstrap).

• At each node, randomly select m features (out of 50) and choose the best split.

• The final prediction is the average of the B trees’ predictions.

Performance:

• XGBoost: F1-score = 0.85 (Precision = 0.90, Recall = 0.80).

• Random Forest: F1-score = 0.82 (Precision = 0.88, Recall = 0.77).

PART 3: UNSUPERVISED LEARNING — Isolation Forest and Autoencoders

Unsupervised learning is used when labelled data is not available (or the fraud pattern is novel and not in the training set).

3.1 Isolation Forest

Isolation Forest works on the principle that anomalies are “few and different” and are easier to isolate.

Algorithm:

1. Randomly choose a feature.

2. Randomly choose a split value between the min and max of that feature.

3. Recursively partition the data until each point is isolated.

4. The anomaly score is the average path length (number of partitions) to isolate a point.

Mathematics:
The expected path length for a point x is E[h(x)]. The anomaly score is:

s(x) = 2^{-E[h(x)] / c(n)}

where c(n) is the average path length for a dataset of size n.

Interpretation: Fraud (anomalies) have shorter path lengths (isolated early), so s(x) is closer to 1. Normal points have longer path lengths, so s(x) is closer to 0.

Performance: Isolation Forest achieves an F1-score of 0.78 (Precision = 0.85, Recall = 0.72).

3.2 Autoencoders

An Autoencoder is a neural network that compresses the input to a lower-dimensional representation (the latent space) and then reconstructs it.

Architecture:

• Encoder: h = f(W_e x + b_e) (compresses to 10 dimensions).

• Decoder: x' = f(W_d h + b_d) (reconstructs the input).

• Loss: L = ||x - x'||² (mean squared error).

Anomaly Score:
The reconstruction error R = ||x - x'||². If a transaction is anomalous (fraud), the autoencoder cannot reconstruct it well (because it was not trained on fraud patterns). The reconstruction error is high.

Performance: The Autoencoder achieves an F1-score of 0.76 (Precision = 0.83, Recall = 0.70).

PART 4: CLASS IMBALANCE — SMOTE and Cost-Sensitive Learning

Fraud transactions are rare (< 1% of all transactions). Training a classifier on an imbalanced dataset leads to a model that predicts “legitimate” for almost all transactions (high accuracy but low recall).

SMOTE (Synthetic Minority Over-sampling) :

1. For each fraud sample, find its k-nearest neighbours (k=5).

2. Randomly select one neighbour.

3. Generate a new synthetic sample: x_new = x_i + rand(0,1) × (x_neighbour - x_i).

4. Repeat until the fraud class has 50% of the dataset size.

Cost-Sensitive Learning:
Assign a higher weight to fraud samples in the loss function:
weight = (1 - p) / p where p is the fraud rate.
For p = 0.01, weight = 99. This forces the model to pay more attention to fraud samples.

PART 5: MODEL EVALUATION — Precision-Recall Curve and F1-Score

Since the dataset is imbalanced, we use the Precision-Recall curve instead of the ROC curve.

Precision: TP / (TP + FP) (how many of the flagged transactions are actually fraud).
Recall: TP / (TP + FN) (how many fraud transactions are flagged).

F1-Score: 2 × (Precision × Recall) / (Precision + Recall).

Performance:

Ensemble:
We combine XGBoost and Isolation Forest by averaging their scores:
Score = 0.6 × Score_XGB + 0.4 × Score_IF

Conclusion: The ensemble model achieves an F1-score of 0.87, which is excellent for fraud detection.

PART 6: MODEL SERVING ARCHITECTURE — Real-Time Scoring (< 10ms)

The trained model must score each transaction in real time (< 10ms).

Architecture:

+-----------------------------------------------------------------------+
|               REAL-TIME ML SCORING ARCHITECTURE                        |
+-----------------------------------------------------------------------+
|                                                                        |
|  API Gateway (Request)                                                |
|          |                                                            |
|          v                                                            |
|  Feature Store (Redis)                                                |
|  +------------------------------------------------------------------+  |
|  |  • Pre-computes aggregate features (sum, count, average).        |  |
|  |  • TTL: 5 minutes.                                                |  |
|  |  • Latency: 1ms (Redis GET).                                    |  |
|  +------------------------------------------------------------------+  |
|          |                                                            |
|          v                                                            |
|  Model Service (ONNX Runtime / TensorFlow Serving)                    |
|  +------------------------------------------------------------------+  |
|  |  • Loads the trained model (XGBoost, Isolation Forest).          |  |
|  |  • Scores the feature vector.                                    |  |
|  |  • Latency: 5ms (XGBoost) + 3ms (Isolation).                    |  |
|  +------------------------------------------------------------------+  |
|          |                                                            |
|          v                                                            |
|  Decision Engine                                                      |
|  +------------------------------------------------------------------+  |
|  |  • Combines model score with rule-based alerts (Lesson 8.3).    |  |
|  |  • Outputs: Allow, Challenge, Block.                             |  |
|  |  • Latency: 0.5ms.                                               |  |
|  +------------------------------------------------------------------+  |
|                                                                        |
|  Total Latency (p95): 1 + 8 + 0.5 = 9.5ms < 10ms.                    |
+-----------------------------------------------------------------------+

ONNX Runtime: A high-performance inference engine that can run XGBoost and Isolation Forest models (converted from Python) with low latency.

PART 7: MODEL DRIFT AND RETRAINING

Fraud patterns evolve over time. The model must be retrained regularly.

Model Drift Detection:
We monitor the AUC (Area Under the ROC Curve) over a sliding window of 7 days. If the AUC drops by more than 0.05, we trigger a retraining.

Retraining Cadence:

• Daily retraining (incremental: update the model with the new day’s data).

• Weekly full retraining (re-train from scratch).

Retraining Pipeline:

1. Extract the last 90 days of data.

2. Re-run feature engineering.

3. Re-train the model (XGBoost + Isolation Forest).

4. Validate on a hold-out set (last 7 days).

5. If the F1-score improves, deploy the new model.

6. Rollback if the new model underperforms (monitoring).

CLOSING — THE MACHINE LEARNING FRONTIER

Machine learning transforms fraud detection from a reactive (rules-based) to a proactive (predictive) system. The ensemble model (XGBoost + Isolation Forest) achieves an F1-score of 0.87, catching 83% of fraud transactions while maintaining 92% precision. The real-time scoring pipeline adds < 10ms of latency, keeping the system responsive.

Operational Risk: If the model is not retrained, it becomes stale and its performance degrades. The drift detection system ensures that the model is retrained when performance drops. The model’s predictions must be interpretable (for regulatory purposes). XGBoost provides feature importance scores, which can be used to explain why a transaction was flagged.

Key Takeaways:

• Feature Engineering: 50+ features from transaction, time, graph, device, consent, and velocity.

• Supervised Learning: XGBoost (F1=0.85), Random Forest (F1=0.82).

• Unsupervised Learning: Isolation Forest (F1=0.78), Autoencoder (F1=0.76).

• Ensemble: XGBoost + Isolation Forest (F1=0.87).

• Class Imbalance: SMOTE and cost-sensitive learning.

• Serving: ONNX Runtime, < 10ms latency.

• Retraining: Daily incremental, weekly full, drift detection.

Transition to Lesson 8.5: With the ML fraud detection pipeline in place, we now turn to the Fraud Decision Engine and Real-Time Orchestration. Lesson 8.5 teaches you how to combine the rules engine (Lesson 8.3), the ML model (Lesson 8.4), the anomaly detection (Lesson 8.1), and the device fingerprinting (Lesson 8.2) into a single, unified fraud decision engine that outputs a final action (Allow, Challenge, Block). We will quantify the overall fraud detection latency and prove that the total end-to-end latency (from request to decision) is under 30ms.