SECTION 1: LEARNING OBJECTIVES

By the end of this lesson, you will be able to:

  1. Handle missing data in banking datasets using appropriate imputation methods.

  2. Scale and normalize financial data for machine learning models.

  3. Encode categorical variables (customer segments, transaction types) for modeling.

  4. Create meaningful features from raw banking data (financial ratios, risk indicators).

  5. Apply feature engineering techniques for time series and transactional data.

  6. Select features using statistical methods and domain knowledge.

  7. Understand the impact of preprocessing on model performance in banking.

SECTION 2: HANDLING MISSING DATA

2.1 Understanding Missing Data Mechanisms

Before handling missing data, understand WHY data is missing:

 
 
Mechanism Description Banking Example
MCAR Missing Completely at Random A server glitch caused random records to be dropped
MAR Missing at Random (depends on other variables) High-income customers less likely to report income
MNAR Missing Not at Random (depends on the missing value) Customers with bad credit scores hide their scores

2.2 Methods for Handling Missing Data

python
# ============= MISSING DATA HANDLING =============

import pandas as pd
import numpy as np
from sklearn.impute import SimpleImputer, KNNImputer
from sklearn.experimental import enable_iterative_imputer
from sklearn.impute import IterativeImputer

# Create a sample dataset with missing values
np.random.seed(42)
sample_data = pd.DataFrame({
    'customer_id': range(1, 101),
    'age': np.random.randint(18, 80, 100),
    'annual_income': np.random.normal(80000, 30000, 100).clip(20000, 200000),
    'credit_score': np.random.normal(700, 50, 100).clip(500, 850),
    'dti_ratio': np.random.uniform(0.1, 0.5, 100),
    'tenure_months': np.random.randint(1, 240, 100)
})

# Introduce missing values
sample_data.loc[np.random.choice(sample_data.index, 15, replace=False), 'credit_score'] = np.nan
sample_data.loc[np.random.choice(sample_data.index, 10, replace=False), 'annual_income'] = np.nan
sample_data.loc[np.random.choice(sample_data.index, 8, replace=False), 'dti_ratio'] = np.nan

print("Original Data with Missing Values:")
print(sample_data.isnull().sum())

class MissingDataHandler:
    """Handle missing data using various methods."""
    
    def __init__(self, data):
        self.data = data.copy()
        self.missing_columns = data.columns[data.isnull().any()].tolist()
    
    def drop_missing(self, threshold=0.5):
        """Drop columns with > threshold missing values."""
        missing_pct = self.data.isnull().mean()
        cols_to_drop = missing_pct[missing_pct > threshold].index.tolist()
        if cols_to_drop:
            print(f"Dropping columns: {cols_to_drop}")
            return self.data.drop(columns=cols_to_drop)
        return self.data
    
    def impute_constant(self, value=0):
        """Impute with a constant value."""
        for col in self.missing_columns:
            self.data[col] = self.data[col].fillna(value)
        return self.data
    
    def impute_mean(self):
        """Impute with column mean."""
        for col in self.missing_columns:
            if pd.api.types.is_numeric_dtype(self.data[col]):
                self.data[col] = self.data[col].fillna(self.data[col].mean())
        return self.data
    
    def impute_median(self):
        """Impute with column median."""
        for col in self.missing_columns:
            if pd.api.types.is_numeric_dtype(self.data[col]):
                self.data[col] = self.data[col].fillna(self.data[col].median())
        return self.data
    
    def impute_forward_fill(self):
        """Forward fill (for time series data)."""
        self.data = self.data.fillna(method='ffill')
        return self.data
    
    def impute_interpolation(self):
        """Interpolation (for time series data)."""
        self.data = self.data.interpolate(method='linear')
        return self.data
    
    def impute_knn(self, n_neighbors=5):
        """KNN imputation."""
        numeric_cols = self.data.select_dtypes(include=[np.number]).columns
        imputer = KNNImputer(n_neighbors=n_neighbors)
        self.data[numeric_cols] = imputer.fit_transform(self.data[numeric_cols])
        return self.data
    
    def impute_iterative(self):
        """Iterative imputation (MICE)."""
        numeric_cols = self.data.select_dtypes(include=[np.number]).columns
        imputer = IterativeImputer(random_state=42)
        self.data[numeric_cols] = imputer.fit_transform(self.data[numeric_cols])
        return self.data
    
    def compare_methods(self):
        """Compare different imputation methods."""
        results = {}
        
        # Original missing counts
        original_missing = self.data.isnull().sum().sum()
        
        # Test methods
        methods = {
            'Mean': self.impute_mean,
            'Median': self.impute_median,
            'KNN (k=3)': lambda: self.impute_knn(3),
            'KNN (k=5)': lambda: self.impute_knn(5),
            'Iterative': self.impute_iterative
        }
        
        for name, method in methods.items():
            # Reset data
            test_data = self.data.copy()
            handler = MissingDataHandler(test_data)
            result = handler.__getattribute__(method.__name__ if hasattr(method, '__name__') else 'impute_knn')()
            results[name] = result.isnull().sum().sum()
        
        print("\n" + "="*60)
        print("COMPARISON OF IMPUTATION METHODS")
        print("="*60)
        print(f"Original missing values: {original_missing}")
        for name, missing_after in results.items():
            print(f"{name}: {missing_after} missing values remaining")
        
        return results

# Demonstrate missing data handling
handler = MissingDataHandler(sample_data)
print("\nMissing Data Summary:")
print(handler.data.isnull().sum())

# Try different methods
handler.compare_methods()

# Use a specific method
cleaned_data = handler.impute_median()
print("\nCleaned Data (Median Imputation):")
print(cleaned_data.isnull().sum())

2.3 Choosing the Right Imputation Method

python
# ============= GUIDELINES FOR IMPUTATION =============

def guide_imputation_choice(data):
    """Guide the choice of imputation method."""
    missing_pct = data.isnull().mean()
    numeric_cols = data.select_dtypes(include=[np.number]).columns
    
    guidance = []
    
    # Check missing percentage
    if missing_pct.max() > 0.5:
        guidance.append("⚠️ Columns with >50% missing may be better dropped")
    
    # Check data type
    for col in data.columns:
        if data[col].isnull().any():
            if pd.api.types.is_numeric_dtype(data[col]):
                guidance.append(f"📊 {col}: Numeric - Consider mean/median imputation")
            elif pd.api.types.is_categorical_dtype(data[col]):
                guidance.append(f"📋 {col}: Categorical - Consider mode imputation")
            else:
                guidance.append(f"📝 {col}: Text/Object - Consider flagging missing values")
    
    # For time series data
    if 'date' in data.columns or 'timestamp' in data.columns:
        guidance.append("⏰ Time series data detected - Consider forward fill or interpolation")
    
    print("\n" + "="*60)
    print("IMPUTATION GUIDANCE")
    print("="*60)
    for g in guidance:
        print(g)
    
    return guidance

# Get guidance
guide_imputation_choice(sample_data)

SECTION 3: FEATURE SCALING & NORMALIZATION

3.1 Why Scale Financial Data?

Many machine learning algorithms require features to be on the same scale:

 
 
Algorithm Scaling Required? Why
Logistic Regression Yes Gradient descent converges faster
SVM Yes Distance-based algorithm
K-Nearest Neighbors Yes Distance-based
Decision Trees No Splits are scale-invariant
Random Forest No Splits are scale-invariant
Neural Networks Yes Gradient descent

3.2 Scaling Methods

python
# ============= FEATURE SCALING =============

from sklearn.preprocessing import StandardScaler, MinMaxScaler, RobustScaler

# Create sample data
np.random.seed(42)
scaling_data = pd.DataFrame({
    'customer_age': np.random.normal(45, 15, 1000).clip(18, 80),
    'annual_income': np.random.lognormal(10.5, 0.6, 1000).clip(20000, 250000),
    'credit_score': np.random.normal(700, 50, 1000).clip(500, 850),
    'transaction_count': np.random.poisson(20, 1000).clip(0, 80)
})

print("Original Data Statistics:")
print(scaling_data.describe())

class FeatureScaler:
    """Scale features using different methods."""
    
    def __init__(self, data):
        self.data = data.copy()
        self.scalers = {}
    
    def standard_scale(self, columns=None):
        """Standardization (z-score): mean=0, std=1."""
        if columns is None:
            columns = self.data.select_dtypes(include=[np.number]).columns
        
        scaler = StandardScaler()
        self.data[columns] = scaler.fit_transform(self.data[columns])
        self.scalers['standard'] = scaler
        
        print(f"Standardized {len(columns)} columns")
        return self.data
    
    def minmax_scale(self, columns=None, feature_range=(0, 1)):
        """Min-Max scaling: scale to [0, 1] range."""
        if columns is None:
            columns = self.data.select_dtypes(include=[np.number]).columns
        
        scaler = MinMaxScaler(feature_range=feature_range)
        self.data[columns] = scaler.fit_transform(self.data[columns])
        self.scalers['minmax'] = scaler
        
        print(f"Min-Max scaled {len(columns)} columns")
        return self.data
    
    def robust_scale(self, columns=None):
        """Robust scaling: uses median and IQR (robust to outliers)."""
        if columns is None:
            columns = self.data.select_dtypes(include=[np.number]).columns
        
        scaler = RobustScaler()
        self.data[columns] = scaler.fit_transform(self.data[columns])
        self.scalers['robust'] = scaler
        
        print(f"Robust scaled {len(columns)} columns")
        return self.data
    
    def compare_scaling(self):
        """Compare scaling methods."""
        columns = self.data.select_dtypes(include=[np.number]).columns
        results = {}
        
        for col in columns:
            results[col] = {
                'original': self.data[col].describe()
            }
            
            # Test different scalers
            for method in ['standard', 'minmax', 'robust']:
                test_data = self.data.copy()
                scaler_obj = FeatureScaler(test_data)
                getattr(scaler_obj, f'{method}_scale')([col])
                results[col][method] = scaler_obj.data[col].describe()
        
        # Print comparison for first column
        first_col = columns[0]
        print("\n" + "="*60)
        print(f"SCALING COMPARISON: {first_col}")
        print("="*60)
        
        for method, stats in results[first_col].items():
            print(f"\n{method.upper()}:")
            print(f"  Mean: {stats['mean']:.4f}")
            print(f"  Std: {stats['std']:.4f}")
            print(f"  Min: {stats['min']:.4f}")
            print(f"  Max: {stats['max']:.4f}")
        
        return results

# Demonstrate scaling
scaler = FeatureScaler(scaling_data)

# Try different scaling methods
print("\n" + "="*60)
print("FEATURE SCALING DEMONSTRATION")
print("="*60)

# Original
print("\nOriginal Statistics (Annual Income):")
print(scaling_data['annual_income'].describe())

# Standard scale
standard_scaled = scaler.standard_scale(['annual_income'])
print("\nStandardized (Annual Income):")
print(standard_scaled['annual_income'].describe())

# Min-Max scale
scaler = FeatureScaler(scaling_data)
minmax_scaled = scaler.minmax_scale(['annual_income'])
print("\nMin-Max Scaled (Annual Income):")
print(minmax_scaled['annual_income'].describe())

# Compare scaling methods
scaler.compare_scaling()

SECTION 4: ENCODING CATEGORICAL VARIABLES

4.1 Types of Categorical Data

 
 
Type Description Banking Example
Nominal No inherent order State (CA, NY, TX), Transaction type
Ordinal Has inherent order Customer segment (Premium > Standard > Basic)
Binary Two categories Is_active (True/False), Has_loan (Yes/No)

4.2 Encoding Methods

python
# ============= ENCODING CATEGORICAL VARIABLES =============

from sklearn.preprocessing import LabelEncoder, OneHotEncoder
import pandas as pd

# Create sample categorical data
categorical_data = pd.DataFrame({
    'customer_id': range(1, 101),
    'segment': np.random.choice(['Premium', 'Standard', 'Basic'], 100, p=[0.2, 0.5, 0.3]),
    'state': np.random.choice(['CA', 'NY', 'TX', 'FL', 'IL'], 100),
    'has_credit_card': np.random.choice([0, 1], 100, p=[0.3, 0.7]),
    'risk_tier': np.random.choice(['Low', 'Medium', 'High'], 100, p=[0.4, 0.4, 0.2])
})

print("Sample Categorical Data:")
print(categorical_data.head())

class CategoricalEncoder:
    """Encode categorical variables."""
    
    def __init__(self, data):
        self.data = data.copy()
        self.encoders = {}
    
    def label_encode(self, column):
        """Label encoding (integer encoding)."""
        encoder = LabelEncoder()
        self.data[f'{column}_encoded'] = encoder.fit_transform(self.data[column])
        self.encoders[column] = encoder
        
        print(f"Label encoded: {column}")
        print(f"  Mapping: {dict(zip(encoder.classes_, encoder.transform(encoder.classes_)))}")
        return self.data
    
    def one_hot_encode(self, column):
        """One-hot encoding (create binary columns)."""
        one_hot = pd.get_dummies(self.data[column], prefix=column)
        self.data = pd.concat([self.data, one_hot], axis=1)
        self.data = self.data.drop(column, axis=1)
        
        print(f"One-hot encoded: {column}")
        print(f"  Created {len(one_hot.columns)} columns: {list(one_hot.columns)}")
        return self.data
    
    def ordinal_encode(self, column, ordering):
        """Ordinal encoding (for ordered categories)."""
        mapping = {category: rank for rank, category in enumerate(ordering)}
        self.data[f'{column}_ordinal'] = self.data[column].map(mapping)
        self.encoders[column] = mapping
        
        print(f"Ordinal encoded: {column}")
        print(f"  Mapping: {mapping}")
        return self.data
    
    def binary_encode(self, column, positive_value=1, negative_value=0):
        """Binary encoding (for binary categories)."""
        self.data[f'{column}_binary'] = self.data[column].map(
            {self.data[column].unique()[0]: positive_value,
             self.data[column].unique()[1]: negative_value}
        )
        
        print(f"Binary encoded: {column}")
        return self.data

# Demonstrate encoding
encoder = CategoricalEncoder(categorical_data)

# Label encode segment
encoder.label_encode('segment')

# One-hot encode state
encoder.one_hot_encode('state')

# Ordinal encode risk tier
encoder.ordinal_encode('risk_tier', ['Low', 'Medium', 'High'])

# Binary encode has_credit_card
encoder.binary_encode('has_credit_card')

print("\nEncoded Data:")
print(encoder.data.head())

# Summary of encoding
print("\n" + "="*60)
print("ENCODING SUMMARY")
print("="*60)

encoded_cols = [col for col in encoder.data.columns if col not in categorical_data.columns]
print(f"New columns created: {len(encoded_cols)}")
print(f"Columns: {encoded_cols}")

SECTION 5: FEATURE ENGINEERING FOR BANKING

5.1 Creating Financial Features

python
# ============= FEATURE ENGINEERING =============

# Create sample banking data
np.random.seed(42)
n_customers = 500

banking_data = pd.DataFrame({
    'customer_id': range(1, n_customers + 1),
    'monthly_income': np.random.normal(6000, 2000, n_customers).clip(1500, 20000),
    'monthly_expenses': np.random.normal(4000, 1500, n_customers).clip(500, 15000),
    'credit_card_balance': np.random.normal(5000, 3000, n_customers).clip(0, 20000),
    'credit_card_limit': np.random.normal(15000, 5000, n_customers).clip(5000, 40000),
    'loan_balance': np.random.normal(100000, 50000, n_customers).clip(0, 300000),
    'savings_balance': np.random.normal(30000, 20000, n_customers).clip(0, 100000),
    'tenure_months': np.random.randint(1, 240, n_customers),
    'num_transactions': np.random.poisson(20, n_customers).clip(0, 80),
    'num_late_payments': np.random.poisson(0.5, n_customers).clip(0, 10)
})

class FinancialFeatureEngineer:
    """Create financial features from raw banking data."""
    
    def __init__(self, data):
        self.data = data.copy()
    
    def create_ratio_features(self):
        """Create financial ratios."""
        self.data['savings_rate'] = self.data['savings_balance'] / self.data['monthly_income']
        self.data['debt_to_income_ratio'] = (self.data['credit_card_balance'] + self.data['loan_balance']) / self.data['monthly_income']
        self.data['credit_utilization'] = self.data['credit_card_balance'] / self.data['credit_card_limit']
        self.data['expense_to_income'] = self.data['monthly_expenses'] / self.data['monthly_income']
        self.data['savings_to_debt'] = self.data['savings_balance'] / (self.data['credit_card_balance'] + self.data['loan_balance'] + 1)
        
        print("Created ratio features:")
        print(f"  • savings_rate: {self.data['savings_rate'].describe()}")
        print(f"  • debt_to_income_ratio: {self.data['debt_to_income_ratio'].describe()}")
        print(f"  • credit_utilization: {self.data['credit_utilization'].describe()}")
        
        return self.data
    
    def create_risk_features(self):
        """Create risk indicators."""
        # Risk flags
        self.data['high_utilization'] = (self.data['credit_utilization'] > 0.8).astype(int)
        self.data['high_debt'] = (self.data['debt_to_income_ratio'] > 0.43).astype(int)
        self.data['low_savings'] = (self.data['savings_rate'] < 0.1).astype(int)
        self.data['has_late_payments'] = (self.data['num_late_payments'] > 0).astype(int)
        
        # Composite risk score
        self.data['risk_score'] = (
            self.data['high_utilization'] * 3 +
            self.data['high_debt'] * 4 +
            self.data['low_savings'] * 2 +
            self.data['has_late_payments'] * 3
        )
        
        print("Created risk features:")
        print(f"  • high_utilization: {self.data['high_utilization'].sum()} customers")
        print(f"  • high_debt: {self.data['high_debt'].sum()} customers")
        print(f"  • risk_score: {self.data['risk_score'].describe()}")
        
        return self.data
    
    def create_behavioral_features(self):
        """Create behavioral features."""
        # Tenure categories
        self.data['tenure_category'] = pd.cut(
            self.data['tenure_months'],
            bins=[0, 12, 36, 120, float('inf')],
            labels=['New', 'Medium', 'Long', 'Veteran']
        )
        
        # Transaction intensity
        self.data['transaction_intensity'] = self.data['num_transactions'] / (self.data['tenure_months'] / 12)
        self.data['transaction_intensity_category'] = pd.cut(
            self.data['transaction_intensity'],
            bins=[0, 10, 30, float('inf')],
            labels=['Low', 'Medium', 'High']
        )
        
        # Total balance
        self.data['total_balance'] = (
            self.data['savings_balance'] - 
            self.data['credit_card_balance'] - 
            self.data['loan_balance']
        )
        
        self.data['positive_balance'] = (self.data['total_balance'] > 0).astype(int)
        
        print("Created behavioral features:")
        print(f"  • tenure_category: {self.data['tenure_category'].value_counts().to_dict()}")
        print(f"  • transaction_intensity_category: {self.data['transaction_intensity_category'].value_counts().to_dict()}")
        print(f"  • positive_balance: {self.data['positive_balance'].sum()} customers")
        
        return self.data
    
    def create_interaction_features(self):
        """Create interaction features."""
        # Income × Tenure (loyalty signal)
        self.data['income_x_tenure'] = self.data['monthly_income'] * self.data['tenure_months'] / 1000
        
        # Balance × Utilization (risk signal)
        self.data['balance_x_utilization'] = self.data['credit_card_balance'] * self.data['credit_utilization']
        
        # Savings × Income (wealth building signal)
        self.data['savings_x_income'] = self.data['savings_balance'] * self.data['monthly_income'] / 10000
        
        print("Created interaction features:")
        print(f"  • income_x_tenure: {self.data['income_x_tenure'].describe()}")
        print(f"  • balance_x_utilization: {self.data['balance_x_utilization'].describe()}")
        
        return self.data
    
    def engineer_features(self):
        """Run all feature engineering steps."""
        self.create_ratio_features()
        self.create_risk_features()
        self.create_behavioral_features()
        self.create_interaction_features()
        
        print("\n" + "="*60)
        print("FEATURE ENGINEERING COMPLETE")
        print("="*60)
        print(f"Original features: 9")
        print(f"New features: {len(self.data.columns) - 9}")
        print(f"Total features: {len(self.data.columns)}")
        
        return self.data

# Demonstrate feature engineering
engineer = FinancialFeatureEngineer(banking_data)
engineered_data = engineer.engineer_features()

print("\nFeature Engineering Results:")
print(f"\nSample Data with New Features:")
print(engineered_data.head())

print(f"\nAll Features ({len(engineered_data.columns)} total):")
for col in engineered_data.columns:
    print(f"  • {col}")

5.2 Feature Engineering for Time Series Data

python
# ============= TIME SERIES FEATURE ENGINEERING =============

def create_time_series_features(transactions):
    """Create features from transaction time series."""
    
    # Ensure date is datetime
    transactions['date'] = pd.to_datetime(transactions['date'])
    
    # Time-based features
    transactions['hour'] = transactions['date'].dt.hour
    transactions['day_of_week'] = transactions['date'].dt.dayofweek
    transactions['is_weekend'] = transactions['day_of_week'].isin([5, 6]).astype(int)
    transactions['month'] = transactions['date'].dt.month
    transactions['quarter'] = transactions['date'].dt.quarter
    transactions['year'] = transactions['date'].dt.year
    
    # Lag features
    transactions = transactions.sort_values(['customer_id', 'date'])
    
    # Create lag features (previous transactions)
    transactions['prev_amount'] = transactions.groupby('customer_id')['amount'].shift(1)
    transactions['amount_change'] = transactions['amount'] - transactions['prev_amount']
    transactions['pct_change'] = transactions['amount'] / transactions['prev_amount'] - 1
    
    # Rolling statistics
    transactions['rolling_avg_3'] = transactions.groupby('customer_id')['amount'].rolling(3).mean().reset_index(0, drop=True)
    transactions['rolling_std_3'] = transactions.groupby('customer_id')['amount'].rolling(3).std().reset_index(0, drop=True)
    transactions['rolling_max_3'] = transactions.groupby('customer_id')['amount'].rolling(3).max().reset_index(0, drop=True)
    
    # Count features
    transactions['transaction_count_cum'] = transactions.groupby('customer_id').cumcount() + 1
    
    print("Created time series features:")
    print(f"  • Hour: {transactions['hour'].nunique()} unique values")
    print(f"  • Day of week: {transactions['day_of_week'].nunique()} unique values")
    print(f"  • Lag features: prev_amount, amount_change, pct_change")
    print(f"  • Rolling stats: rolling_avg_3, rolling_std_3, rolling_max_3")
    
    return transactions

# Create sample transaction data
n_transactions = 1000
sample_transactions = pd.DataFrame({
    'transaction_id': range(1, n_transactions + 1),
    'customer_id': np.random.randint(1, 51, n_transactions),
    'date': [datetime.now() - timedelta(days=np.random.randint(0, 365)) for _ in range(n_transactions)],
    'amount': np.random.lognormal(4, 1.5, n_transactions)
})

# Create time series features
enhanced_transactions = create_time_series_features(sample_transactions)
print("\nSample Time Series Features:")
print(enhanced_transactions.head())

SECTION 6: FEATURE SELECTION

6.1 Feature Selection Methods

python
# ============= FEATURE SELECTION =============

from sklearn.feature_selection import SelectKBest, f_regression, mutual_info_regression
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import train_test_split

# Prepare data for feature selection
# Create a target variable (e.g., total spending)
np.random.seed(42)
target_data = engineered_data.copy()
target_data['total_spending'] = (
    target_data['monthly_expenses'] * 12 +
    target_data['credit_card_balance'] * 0.1 +
    target_data['num_transactions'] * 50 +
    np.random.normal(0, 1000, len(target_data))
)

# Separate features and target
feature_cols = [col for col in target_data.columns if col not in ['customer_id', 'total_spending']]
X = target_data[feature_cols]
y = target_data['total_spending']

# For categorical features, we need to encode them for selection
# Handle categorical columns
categorical_cols = X.select_dtypes(include=['object', 'category']).columns
X_encoded = pd.get_dummies(X, columns=categorical_cols, drop_first=True)

class FeatureSelector:
    """Select important features using different methods."""
    
    def __init__(self, X, y):
        self.X = X
        self.y = y
        self.results = {}
    
    def filter_method(self, k=10):
        """Filter method: Select top k features based on statistical tests."""
        selector = SelectKBest(score_func=f_regression, k=k)
        selector.fit(self.X, self.y)
        
        # Get scores
        scores = pd.DataFrame({
            'feature': self.X.columns,
            'score': selector.scores_
        }).sort_values('score', ascending=False)
        
        selected = scores.head(k)['feature'].tolist()
        self.results['filter'] = {
            'selected': selected,
            'scores': scores
        }
        
        print(f"Filter Method: Selected {len(selected)} features")
        print(f"  Top features: {selected[:5]}")
        return selected
    
    def wrapper_method(self, n_features=10):
        """Wrapper method: Forward selection using model performance."""
        from sklearn.feature_selection import RFE
        from sklearn.linear_model import LinearRegression
        
        estimator = LinearRegression()
        selector = RFE(estimator, n_features_to_select=n_features)
        selector.fit(self.X, self.y)
        
        selected = self.X.columns[selector.support_].tolist()
        rankings = pd.DataFrame({
            'feature': self.X.columns,
            'ranking': selector.ranking_
        }).sort_values('ranking')
        
        self.results['wrapper'] = {
            'selected': selected,
            'rankings': rankings
        }
        
        print(f"Wrapper Method (RFE): Selected {len(selected)} features")
        print(f"  Selected features: {selected[:5]}")
        return selected
    
    def embedded_method(self, n_features=10):
        """Embedded method: Feature importance from Random Forest."""
        rf = RandomForestRegressor(n_estimators=100, random_state=42)
        rf.fit(self.X, self.y)
        
        importances = pd.DataFrame({
            'feature': self.X.columns,
            'importance': rf.feature_importances_
        }).sort_values('importance', ascending=False)
        
        selected = importances.head(n_features)['feature'].tolist()
        
        self.results['embedded'] = {
            'selected': selected,
            'importances': importances
        }
        
        print(f"Embedded Method (Random Forest): Selected {len(selected)} features")
        print(f"  Top features: {selected[:5]}")
        print(f"  Feature importances:")
        print(importances.head(10))
        return selected
    
    def compare_methods(self):
        """Compare feature selection methods."""
        print("\n" + "="*60)
        print("FEATURE SELECTION COMPARISON")
        print("="*60)
        
        # Run all methods
        filter_selected = self.filter_method(k=10)
        wrapper_selected = self.wrapper_method(n_features=10)
        embedded_selected = self.embedded_method(n_features=10)
        
        # Find common features
        common = set(filter_selected) & set(wrapper_selected) & set(embedded_selected)
        
        print(f"\nCommon features across all methods: {len(common)}")
        print(f"  {common}")
        
        # Union of all selected
        union = set(filter_selected) | set(wrapper_selected) | set(embedded_selected)
        print(f"Unique features across all methods: {len(union)}")
        
        return common, union

# Run feature selection
selector = FeatureSelector(X_encoded, y)
common, union = selector.compare_methods()

SECTION 7: BUSINESS RISK & FINANCIAL IMPACT

7.1 Impact of Poor Preprocessing

 
 
Issue Impact Example
Missing Data Biased models Credit scoring model underestimates risk for missing data
Outliers Skewed predictions Fraud model flags too many false positives
Incorrect Scaling Poor model convergence Neural network fails to learn
Poor Encoding Lost information Ordinal categories encoded as nominal
Missing Features Lost predictive power Customer churn model misses key signals

7.2 Regulatory Considerations

 
 
Regulation Preprocessing Requirement
SR 11-7 Must document feature engineering decisions
Fair Lending Features must not discriminate
GDPR Must explain how features are created

SECTION 8: SUMMARY FOR THE DATA PRACTITIONER

8.1 The 1-Minute Elevator Pitch

“Data preprocessing transforms raw banking data into features that machine learning models can use effectively. We handle missing data using imputation, scale features for consistent ranges, encode categorical variables, and engineer meaningful financial features like debt-to-income ratios and credit utilization. Feature selection identifies the most predictive variables. Proper preprocessing is essential for building accurate, compliant models for credit scoring, fraud detection, and customer analytics.”

8.2 Key Takeaways

  1. Missing data requires careful handling based on why it’s missing.

  2. Imputation methods include mean, median, KNN, and iterative approaches.

  3. Scaling is required for distance-based algorithms.

  4. Encoding transforms categorical data into numeric form.

  5. Feature engineering creates domain-specific features from raw data.

  6. Financial ratios (DTI, utilization, savings rate) are powerful predictors.

  7. Risk indicators help identify potentially problematic customers.

  8. Time series features capture patterns in transaction data.

  9. Feature selection identifies the most predictive variables.

  10. Documentation is essential for regulatory compliance.

8.3 Recommended Next Steps

  1. Practice preprocessing on a real banking dataset

  2. Implement your own feature engineering pipeline

  3. Learn about automated feature engineering (FeatureTools)

  4. Document your preprocessing decisions for regulators

 

 
 
 
 
PreviousNext