+------------------+     +------------------+     +------------------+     +------------------+     +------------------+
|   Business       |     |   Data           |     |   Data           |     |   Modelling      |     |   Evaluation     |
|   Understanding  | --> |   Understanding  | --> |   Preparation    | --> |                  | --> |                  |
+------------------+     +------------------+     +------------------+     +------------------+     +------------------+
                                                                                                                 |
                                                                                                                 v
+------------------+     +------------------+     +------------------+     +------------------+     +------------------+
|   Deployment     | <-- |   Model          |     |   Evaluation     |     |   Modelling      |     |   Evaluation     |
|                  |     |   Monitoring     |     |   (Regulatory)   |     |   (Tuning)       |     |   (Final)        |
+------------------+     +------------------+     +------------------+     +------------------+     +------------------+

Sprint 1           Sprint 2           Sprint 3           Sprint 4
┌──────────┐       ┌──────────┐       ┌──────────┐       ┌──────────┐
│ Data     │       │ Feature  │       │ Baseline │       │ Model    │
│ Ingestion│ ----> │ Engineering| ----> │ Model    │ ----> │ Tuning   │
└──────────┘       └──────────┘       └──────────┘       └──────────┘
      │                  │                  │                  │
      v                  v                  v                  v
┌──────────┐       ┌──────────┐       ┌──────────┐       ┌──────────┐
│ Data     │       │ EDA &    │       │ Model    │       │ Validation│
│ Quality  │       │ Insights │       │ Evaluation│       │ & Deployment
└──────────┘       └──────────┘       └──────────┘       └──────────┘

# ===================================================================
# MODULE 8, LESSON 2: PROJECT MANAGEMENT FOR DATA SCIENCE
# ===================================================================

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from datetime import datetime, timedelta
import warnings
warnings.filterwarnings('ignore')

# Set style
sns.set_style("whitegrid")
np.random.seed(42)

print("="*70)
print("PROJECT MANAGEMENT AND DELIVERY FOR FINANCIAL DATA SCIENCE")
print("="*70)

# ----------------------------------------------------------------
# PART A: PROJECT BACKLOG (AGILE)
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART A: Project Backlog (Agile)")
print("-"*60)

# Define user stories and tasks for a credit risk modelling project
backlog = pd.DataFrame({
    'Story ID': ['US-01', 'US-02', 'US-03', 'US-04', 'US-05', 'US-06', 'US-07', 'US-08'],
    'User Story': [
        'As a credit officer, I want to see a dashboard of loan applications',
        'As a risk manager, I want to predict default probabilities',
        'As a data scientist, I want to clean and prepare loan data',
        'As an ML engineer, I want to deploy the default prediction model',
        'As a regulator, I want to validate the model and its explainability',
        'As a product owner, I want to test the model with A/B testing',
        'As a business user, I want to generate reports and insights',
        'As a compliance officer, I want to audit model decisions'
    ],
    'Priority': ['High', 'High', 'High', 'Medium', 'High', 'Medium', 'Low', 'Medium'],
    'Story Points': [3, 5, 8, 5, 3, 3, 2, 5],
    'Status': ['To Do', 'In Progress', 'In Progress', 'To Do', 'To Do', 'To Do', 'To Do', 'To Do']
})

print("Project Backlog:")
print(backlog.to_string(index=False))

# ----------------------------------------------------------------
# PART B: SPRINT PLANNING
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART B: Sprint Planning")
print("-"*60)

# Define sprints
sprints = pd.DataFrame({
    'Sprint': ['Sprint 1', 'Sprint 2', 'Sprint 3', 'Sprint 4'],
    'Start': ['2024-01-08', '2024-01-22', '2024-02-05', '2024-02-19'],
    'End': ['2024-01-19', '2024-02-02', '2024-02-16', '2024-03-01'],
    'Stories': ['US-01, US-02', 'US-03, US-04', 'US-05, US-06', 'US-07, US-08'],
    'Story Points': ['8', '13', '6', '7'],
    'Velocity': ['8', '13', '6', '7'],
    'Objective': [
        'Set up infrastructure and data pipeline',
        'Build and validate initial model',
        'Deploy and test model',
        'Optimise and finalise'
    ]
})

print("\nSprint Plan:")
print(sprints.to_string(index=False))

# ----------------------------------------------------------------
# PART C: PROJECT TIMELINE (GANTT CHART)
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART C: Project Timeline - Gantt Chart")
print("-"*60)

def create_gantt_chart():
    """Create a simple Gantt chart for the project."""
    
    tasks = [
        {'Task': 'Business Understanding', 'Start': 0, 'Duration': 5},
        {'Task': 'Data Discovery', 'Start': 3, 'Duration': 10},
        {'Task': 'Data Preparation', 'Start': 10, 'Duration': 15},
        {'Task': 'Feature Engineering', 'Start': 15, 'Duration': 10},
        {'Task': 'Model Development', 'Start': 20, 'Duration': 15},
        {'Task': 'Model Tuning', 'Start': 30, 'Duration': 10},
        {'Task': 'Model Validation', 'Start': 35, 'Duration': 10},
        {'Task': 'Deployment', 'Start': 42, 'Duration': 10},
        {'Task': 'Monitoring Setup', 'Start': 48, 'Duration': 8},
        {'Task': 'Documentation', 'Start': 30, 'Duration': 20},
    ]
    
    fig, ax = plt.subplots(figsize=(14, 6))
    
    # Create bars
    for i, task in enumerate(tasks):
        y = i
        start = task['Start']
        duration = task['Duration']
        ax.barh(y, duration, left=start, height=0.6, 
                color='steelblue', edgecolor='black', alpha=0.8)
        ax.text(start + duration/2, y, task['Task'], 
                ha='center', va='center', fontsize=9, color='white', fontweight='bold')
    
    ax.set_yticks(range(len(tasks)))
    ax.set_yticklabels([t['Task'] for t in tasks])
    ax.set_xlabel('Days')
    ax.set_title('Credit Risk Modelling Project – Gantt Chart', fontsize=14)
    ax.grid(True, alpha=0.3, axis='x')
    
    # Mark milestones
    milestones = [5, 20, 35, 45, 55]
    for m in milestones:
        ax.axvline(x=m, color='red', linestyle='--', alpha=0.5, linewidth=0.5)
    
    plt.tight_layout()
    plt.savefig('project_gantt_chart.png', dpi=300, bbox_inches='tight')
    plt.show()

create_gantt_chart()
print("Gantt chart saved as 'project_gantt_chart.png'")

# ----------------------------------------------------------------
# PART D: RISK REGISTER
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART D: Project Risk Register")
print("-"*60)

risk_register = pd.DataFrame({
    'Risk ID': ['R1', 'R2', 'R3', 'R4', 'R5', 'R6', 'R7'],
    'Description': [
        'Data quality issues (missing values, outliers)',
        'Model does not achieve target AUC',
        'Deployment delay due to IT constraints',
        'Regulatory rejection of model',
        'Scope creep (additional features requested)',
        'Loss of key team member',
        'Integration issues with existing systems'
    ],
    'Probability': ['High', 'Medium', 'Medium', 'Medium', 'High', 'Low', 'Medium'],
    'Impact': ['High', 'High', 'Medium', 'High', 'Medium', 'High', 'Medium'],
    'Risk Score': [16, 12, 9, 12, 12, 8, 9],
    'Mitigation': [
        'Data profiling, imputation, validation',
        'Ensemble methods, feature engineering',
        'Early engagement with IT, MLOps automation',
        'Engage compliance team from start, document everything',
        'Clear scope, change control process',
        'Cross-training, documentation',
        'API-first design, early integration testing'
    ],
    'Owner': ['Data Lead', 'DS Lead', 'ML Engineer', 'Compliance Lead', 'Product Owner', 'Team Lead', 'Tech Lead'],
    'Status': ['Active', 'Active', 'Active', 'Active', 'Active', 'Inactive', 'Active']
})

print("Project Risk Register:")
print(risk_register.to_string(index=False))

# ----------------------------------------------------------------
# PART E: STAKEHOLDER ANALYSIS
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART E: Stakeholder Analysis")
print("-"*60)

stakeholders = pd.DataFrame({
    'Stakeholder': ['Head of Retail Banking', 'Chief Risk Officer', 'IT Director', 
                   'Compliance Officer', 'Data Governance Lead', 'Head of Analytics'],
    'Interest': ['High', 'High', 'Medium', 'High', 'Medium', 'High'],
    'Influence': ['High', 'High', 'High', 'High', 'Medium', 'Medium'],
    'Power/Interest Grid': ['High/High', 'High/High', 'Medium/High', 'High/High', 'Medium/Medium', 'Medium/Medium'],
    'Engagement Strategy': [
        'Regular updates, demos, involve in steering committee',
        'Approval and sign-off, regular risk reporting',
        'Technical reviews, integration planning',
        'Regular compliance checkpoints, documentation review',
        'Data access approvals, data quality reporting',
        'Sprint reviews, progress updates'
    ]
})

print("Stakeholder Analysis:")
print(stakeholders.to_string(index=False))

# ----------------------------------------------------------------
# PART F: COMMUNICATION PLAN
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART F: Communication Plan")
print("-"*60)

comm_plan = pd.DataFrame({
    'Frequency': ['Daily', 'Weekly', 'Bi-Weekly', 'Monthly', 'Quarterly'],
    'Format': ['Stand-up', 'Status Report', 'Sprint Review', 'Steering Committee', 'Executive Summary'],
    'Audience': ['Team', 'Project Team, Stakeholders', 'Stakeholders', 'Executives', 'C-Suite'],
    'Content': ['Progress, blockers, daily plan', 'Progress against milestones, risks', 
                'Showcase progress, collect feedback', 'KPIs, strategic alignment, risks', 
                'Business impact, ROI, next steps'],
    'Responsible': ['Team Lead', 'Project Manager', 'Data Scientist', 'Project Manager', 'PM/Sponsor']
})

print("Communication Plan:")
print(comm_plan.to_string(index=False))

# ----------------------------------------------------------------
# PART G: PROJECT SUCCESS CRITERIA
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART G: Project Success Criteria")
print("-"*60)

success_criteria = pd.DataFrame({
    'Category': ['Model Performance', 'Business Impact', 'Timeline', 'Budget', 'Quality', 'Compliance', 'User Satisfaction'],
    'Metric': [
        'AUC > 0.80, KS > 0.30, Calibration p > 0.05',
        'Fraud reduction > 15%, Cost savings > $5M/year',
        'Project delivered within 6 months',
        'Within 10% of $200,000 budget',
        'Data quality > 95%, No critical bugs',
        'Pass regulatory review, Full documentation',
        'NPS > 40, Adoption rate > 80%'
    ],
    'Target': [
        'AUC = 0.85, KS = 0.38',
        'Fraud reduction = 18%',
        '6 months (on track)',
        '$195,000 (under budget)',
        '100% data quality, 0 critical bugs',
        'Approved, complete documentation',
        'NPS = 45, Adoption = 85%'
    ],
    'Status': ['On Track', 'At Risk', 'On Track', 'On Track', 'On Track', 'At Risk', 'On Track']
})

print("Project Success Criteria:")
print(success_criteria.to_string(index=False))

# ----------------------------------------------------------------
# PART H: PROJECT GOVERNANCE
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART H: Project Governance")
print("-"*60)

print("""
Project Governance Structure:

+----------------------------------+
|      Executive Steering          |
|      Committee (Quarterly)       |
|  - CEO, CDAO, CRO, Head of Risk  |
+----------------------------------+
                 |
+----------------v---------------+
|      Project Steering          |
|      Committee (Monthly)       |
|  - Sponsors, PM, Tech Lead     |
+--------------------------------+
                 |
+----------------v---------------+
|      Project Team (Weekly)     |
|  - Data Scientist, Engineer,   |
|    Analyst, Product Manager    |
+--------------------------------+

Key Governance Artefacts:
  - Project Charter
  - Business Case
  - Project Plan (Gantt)
  - Risk Register
  - Weekly Status Reports
  - Sprint Reports
  - Model Validation Report
  - Deployment Plan
  - User Documentation

Decision-Making Framework:
  - Escalation: Issues escalated to Project Steering Committee
  - Change Control: Formal process for scope changes
  - Approval Gates: Model validation, deployment, go-live
  - Regulatory Checkpoints: Compliance review at key phases
""")

# ----------------------------------------------------------------
# PART I: SUMMARY AND RECOMMENDATIONS
# ----------------------------------------------------------------

print("\n" + "="*70)
print("PART I: Summary and Recommendations")
print("="*70)

print("""
Project Management for Data Science – Key Takeaways:

1. Project Lifecycle: Use CRISP-DM or Agile frameworks.
2. Scoping: Define clear problem, objectives, success criteria, and constraints.
3. Planning: Estimate effort, plan resources, and set milestones.
4. Stakeholders: Identify, analyse, and engage regularly.
5. Risk Management: Identify, assess, and mitigate risks proactively.
6. Communication: Plan and execute regular communication with stakeholders.
7. Governance: Establish clear decision-making and escalation processes.
8. Success Criteria: Define and track metrics for success.

Recommendations:
  - Use Agile for iterative development and stakeholder feedback.
  - Build a detailed risk register and review it regularly.
  - Automate testing, validation, and deployment with MLOps.
  - Document everything – decisions, assumptions, and learnings.
  - Involve compliance and risk teams early.
  - Celebrate successes and learn from failures.
""")

print("="*70)
print("END OF LESSON 2 – MODULE 8")
print("="*70)