SECTION 1: LEARNING OBJECTIVES

By the end of this lesson, you will be able to:

  • Define blockchain scalability and identify the core challenges.

  • Differentiate between Layer 1 and Layer 2 scaling solutions.

  • Explain how payment channels (e.g., Lightning Network) work.

  • Describe rollups (Optimistic and ZK) and their mechanisms.

  • Understand sharding and its role in blockchain scaling.

  • Compare scalability approaches across major blockchains.

  • Implement a simple state channel simulation in Python.

  • Develop a decision framework for scalability choices.


SECTION 2: THE SCALABILITY PROBLEM

2.1 What is Scalability?

Scalability refers to a blockchain’s ability to handle increasing transaction volumes without compromising decentralisation or security.

The Blockchain Trilemma (revisited):

  • Decentralisation: many nodes, no central control.

  • Security: resistance to attacks and integrity of data.

  • Scalability: high transaction throughput and low latency.

Trade-off: improving one often degrades another.

2.2 Current Limitations

 
 
Blockchain TPS (approx) Block Time Finality
Bitcoin 7 10 min ~1 hour (6 conf)
Ethereum 15-30 12-15 sec ~1 min (finality)
Visa 24,000 instant

Causes:

  • Block size limits.

  • Block frequency constraints.

  • Node resource requirements.

  • Consensus overhead.


SECTION 3: LAYER 1 SCALING SOLUTIONS

3.1 Definition

Layer 1 refers to the base protocol itself. Scaling at this level involves changing the core protocol (e.g., block size, consensus, sharding).

3.2 Key Layer 1 Approaches

 
 
Approach Description Examples
Increasing Block Size Larger blocks = more transactions per block. Bitcoin Cash
Faster Block Times Reduce time between blocks. Ethereum (13s), Solana (400ms)
Consensus Changes PoW → PoS for better efficiency. Ethereum 2.0
Sharding Split chain into parallel shards. Ethereum 2.0, Zilliqa
DAG (Directed Acyclic Graph) No blocks; transaction references. Hedera, Nano

SECTION 4: LAYER 2 SCALING SOLUTIONS

4.1 Definition

Layer 2 solutions are built on top of the base chain to handle transactions off-chain while using the main chain for settlement and security.

4.2 Payment Channels

text
┌─────────────────────────────────────────────────────────────────────────────┐
│                    PAYMENT CHANNEL (e.g., Lightning)                       │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│  1. Open channel:                            Main chain                    │
│     Alice → Bob (deposit)                   ┌──────────────────────────┐   │
│     ┌─────────┐                             │ Funding transaction     │   │
│     │ Alice   │─────────────────────────────│    (multisig)          │   │
│     │ 10 BTC  │                             └──────────────────────────┘   │
│     └─────────┘                                                             │
│                                                                             │
│  2. Off-chain transactions (many, instant, low-fee):                      │
│     ┌─────┐    ┌─────┐    ┌─────┐                                        │
│     │ A→B │    │ B→C │    │ C→A │ ...                                    │
│     │ 1   │    │ 2   │    │ 0.5 │                                        │
│     └─────┘    └─────┘    └─────┘                                        │
│                                                                             │
│  3. Close channel:                          Main chain                    │
│     ┌──────────────────────────────┐      ┌──────────────────────────┐   │
│     │ Final state: Alice 8.5, Bob 1.5│───▶│ Settlement transaction   │   │
│     └──────────────────────────────┘      └──────────────────────────┘   │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Benefits:

  • Instant transactions.

  • Near-zero fees.

  • High throughput.

Drawbacks:

  • Requires collateral lock-up.

  • Not all use cases can be channelised.

  • Complexity of routing.

4.3 Rollups

Rollups execute transactions off-chain but post compressed data to the main chain.

 
 
Type Security Mechanism Pros Cons
Optimistic Rollup Assume valid unless challenged (fraud proofs) Lower cost, EVM-compatible Withdrawal delay (~7 days)
ZK-Rollup Zero-knowledge proofs (validity proofs) Faster finality, high privacy Computationally heavy, less EVM-compatible

Popular Rollups:

  • Optimistic: Arbitrum, Optimism.

  • ZK: ZkSync, StarkNet, Polygon zkEVM.

4.4 State Channels

Similar to payment channels but for arbitrary state (not just transfers). Used in gaming, predictions, etc.

4.5 Plasma

Child chains that report to the main chain using Merkle proofs. Requires fraud proofs.

4.6 Sidechains

Separate blockchains that are connected to the main chain via a bridge. They have their own consensus.


SECTION 5: SHARDING

5.1 Concept

Sharding partitions the blockchain state into multiple shards, each processing its own transactions in parallel.

text
┌─────────────────────────────────────────────────────────────────────────────┐
│                    SHARDING ARCHITECTURE                                    │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                             │
│                          ┌─────────────┐                                   │
│                          │   Beacon    │                                   │
│                          │    Chain    │  (Coordination & finality)        │
│                          └──────┬──────┘                                   │
│                                 │                                           │
│              ┌──────────────────┼──────────────────┐                      │
│              │                  │                  │                      │
│              v                  v                  v                      │
│   ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐        │
│   │    Shard 0       │ │    Shard 1       │ │    Shard 2       │        │
│   │   Tx: A→B        │ │   Tx: C→D        │ │   Tx: E→F        │        │
│   │   State: ...     │ │   State: ...     │ │   State: ...     │        │
│   └──────────────────┘ └──────────────────┘ └──────────────────┘        │
│                                                                             │
│  Each shard processes independently; cross-shard communication via         │
│  the beacon chain or cross-shard messages.                                 │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

5.2 Benefits and Challenges

 
 
Benefit Challenge
Linear scaling with shard count Cross-shard communication complexity
Lower node requirements Data availability issues
Higher throughput Security risk (1% attack)

SECTION 6: IMPLEMENTATION IN PYTHON

python
# ===================================================================
# MODULE 1, LESSON 6: BLOCKCHAIN SCALABILITY
# ===================================================================

import hashlib
import time
import random
from typing import List, Dict, Tuple
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import warnings
warnings.filterwarnings('ignore')

print("="*70)
print("BLOCKCHAIN SCALABILITY – LAYER 2 AND SHARDING")
print("="*70)

# ----------------------------------------------------------------
# PART A: PAYMENT CHANNEL SIMULATION
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART A: Payment Channel Simulation (State Channel)")
print("-"*60)

class PaymentChannel:
    def __init__(self, alice: str, bob: str, initial_deposit: int):
        self.alice = alice
        self.bob = bob
        self.balances = {alice: initial_deposit, bob: 0}
        self.state_version = 0
        self.closed = False
        self.transactions = []
    
    def send(self, sender: str, recipient: str, amount: int) -> bool:
        if self.closed:
            print("Channel is closed.")
            return False
        if sender not in self.balances or recipient not in self.balances:
            return False
        if self.balances[sender] < amount:
            print(f"Insufficient balance for {sender}.")
            return False
        
        self.balances[sender] -= amount
        self.balances[recipient] += amount
        self.state_version += 1
        tx = {'from': sender, 'to': recipient, 'amount': amount, 'version': self.state_version}
        self.transactions.append(tx)
        print(f"Off-chain tx: {sender}{recipient} {amount} (version {self.state_version})")
        return True
    
    def get_state_hash(self) -> str:
        # Simple hash of balances and version
        data = f"{self.alice}:{self.balances[self.alice]}|{self.bob}:{self.balances[self.bob]}|v{self.state_version}"
        return hashlib.sha256(data.encode()).hexdigest()
    
    def close(self) -> Dict:
        self.closed = True
        print(f"Channel closed. Final balances: Alice={self.balances[self.alice]}, Bob={self.balances[self.bob]}")
        return self.balances.copy()

# Simulate channel
channel = PaymentChannel("Alice", "Bob", 100)
print("Channel opened: Alice has 100, Bob has 0")

channel.send("Alice", "Bob", 30)
channel.send("Alice", "Bob", 20)
channel.send("Bob", "Alice", 10)

final_state = channel.close()
print(f"Final state hash: {channel.get_state_hash()[:16]}...")

# ----------------------------------------------------------------
# PART B: SIMPLE ROLLUP SIMULATION (OPTIMISTIC)
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART B: Optimistic Rollup Simulation")
print("-"*60)

class OptimisticRollup:
    def __init__(self, layer1_blocks: int = 10):
        self.l2_transactions = []
        self.batches = []
        self.state = {"Alice": 100, "Bob": 50, "Charlie": 0}
        self.l1_blocks = layer1_blocks
        self.challenge_period = 5  # blocks
        self.finalized = False
    
    def execute_tx(self, sender: str, recipient: str, amount: int):
        if self.state.get(sender, 0) < amount:
            print(f"Invalid tx: {sender} insufficient funds.")
            return False
        self.state[sender] -= amount
        self.state[recipient] = self.state.get(recipient, 0) + amount
        self.l2_transactions.append((sender, recipient, amount))
        return True
    
    def create_batch(self):
        if not self.l2_transactions:
            print("No transactions to batch.")
            return None
        batch = {
            'transactions': self.l2_transactions.copy(),
            'timestamp': time.time(),
            'state_root': self.compute_state_root(),
        }
        self.batches.append(batch)
        self.l2_transactions = []
        print(f"Batch created with {len(batch['transactions'])} txs, state root: {batch['state_root'][:16]}...")
        return batch
    
    def compute_state_root(self) -> str:
        # Hash of sorted state items
        state_str = "|".join(f"{k}:{v}" for k, v in sorted(self.state.items()))
        return hashlib.sha256(state_str.encode()).hexdigest()
    
    def challenge_batch(self, batch_index: int, proof_of_invalid_state: str) -> bool:
        # In practice, someone would submit a fraud proof
        if batch_index >= len(self.batches):
            return False
        # Simulate challenge success with random probability
        if random.random() < 0.3:  # 30% chance of fraud detection
            print(f"Fraud detected in batch {batch_index}! Rollback.")
            return True
        return False
    
    def finalize_batch(self, batch_index: int):
        if batch_index >= len(self.batches):
            return
        # After challenge period with no proof, finalize
        if not self.challenge_batch(batch_index, ""):
            print(f"Batch {batch_index} finalized.")
            self.finalized = True

# Simulate
rollup = OptimisticRollup()
rollup.execute_tx("Alice", "Bob", 20)
rollup.execute_tx("Bob", "Charlie", 10)
rollup.execute_tx("Alice", "Charlie", 5)
rollup.create_batch()

rollup.execute_tx("Bob", "Alice", 15)
rollup.create_batch()

print("State after rollup execution:")
for k, v in rollup.state.items():
    print(f"  {k}: {v}")

# ----------------------------------------------------------------
# PART C: SHARDING SIMULATION
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART C: Sharding Simulation (Parallel Processing)")
print("-"*60)

class Shard:
    def __init__(self, shard_id: int):
        self.shard_id = shard_id
        self.transactions = []
        self.state = {}
        self.blocks = []
    
    def process_tx(self, tx: Dict):
        # Simulate processing
        self.transactions.append(tx)
        # Update state
        sender = tx.get('from')
        recipient = tx.get('to')
        amount = tx.get('amount', 0)
        self.state[sender] = self.state.get(sender, 0) - amount
        self.state[recipient] = self.state.get(recipient, 0) + amount
    
    def produce_block(self) -> Dict:
        block = {
            'shard': self.shard_id,
            'transactions': self.transactions.copy(),
            'state_root': self.compute_state_root(),
            'timestamp': time.time()
        }
        self.blocks.append(block)
        self.transactions = []
        return block
    
    def compute_state_root(self) -> str:
        state_str = "|".join(f"{k}:{v}" for k, v in sorted(self.state.items()))
        return hashlib.sha256(state_str.encode()).hexdigest()
    
    def __repr__(self):
        return f"Shard({self.shard_id}, blocks={len(self.blocks)})"

class BlockchainWithSharding:
    def __init__(self, num_shards: int):
        self.shards = [Shard(i) for i in range(num_shards)]
        self.beacon_chain = []
    
    def get_shard_for_address(self, address: str) -> int:
        # Simple hash-based sharding
        return int(hashlib.sha256(address.encode()).hexdigest()[0], 16) % len(self.shards)
    
    def submit_tx(self, from_addr: str, to_addr: str, amount: int):
        shard_id = self.get_shard_for_address(from_addr)
        tx = {'from': from_addr, 'to': to_addr, 'amount': amount}
        self.shards[shard_id].process_tx(tx)
        print(f"Tx from {from_addr} to {to_addr} assigned to Shard {shard_id}")
    
    def produce_blocks(self):
        for shard in self.shards:
            block = shard.produce_block()
            # Add to beacon chain (simplified)
            self.beacon_chain.append({
                'shard_id': shard.shard_id,
                'block': block,
                'timestamp': time.time()
            })
        print("Blocks produced for all shards.")
    
    def get_metrics(self):
        total_txs = sum(len(s.blocks) for s in self.shards) * 2  # approx
        return {
            'num_shards': len(self.shards),
            'total_blocks': sum(len(s.blocks) for s in self.shards),
            'shard_metrics': [{
                'shard': s.shard_id,
                'blocks': len(s.blocks),
                'state_size': len(s.state)
            } for s in self.shards]
        }

# Simulate
num_shards = 3
chain = BlockchainWithSharding(num_shards)

# Generate transactions
addresses = [f"User{i}" for i in range(10)]
for _ in range(20):
    from_addr = random.choice(addresses)
    to_addr = random.choice([a for a in addresses if a != from_addr])
    amount = random.randint(1, 50)
    chain.submit_tx(from_addr, to_addr, amount)

chain.produce_blocks()

metrics = chain.get_metrics()
print("\nSharding Metrics:")
print(f"Number of shards: {metrics['num_shards']}")
print(f"Total blocks produced: {metrics['total_blocks']}")
for m in metrics['shard_metrics']:
    print(f"  Shard {m['shard']}: {m['blocks']} blocks, {m['state_size']} state entries")

# ----------------------------------------------------------------
# PART D: SCALABILITY SOLUTION COMPARISON
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART D: Scalability Solution Comparison")
print("-"*60)

scaling_compare = pd.DataFrame({
    'Solution': [
        'Layer 1: Block Size',
        'Layer 1: Sharding',
        'Layer 2: State Channels',
        'Layer 2: Optimistic Rollup',
        'Layer 2: ZK-Rollup',
        'Layer 2: Sidechain'
    ],
    'TPS (estimate)': [50, 1000, 10000, 2000, 3000, 1000],
    'Security': ['High', 'Medium', 'Medium', 'High', 'Very High', 'Low-Med'],
    'Finality': ['Slow', 'Medium', 'Instant', 'Delayed (7d)', 'Fast', 'Medium'],
    'Cost': ['Low', 'Low', 'Very Low', 'Low', 'Medium', 'Low'],
    'Complexity': ['Low', 'High', 'High', 'High', 'Very High', 'Medium']
})

print(scaling_compare.to_string(index=False))

# ----------------------------------------------------------------
# PART E: SCALABILITY METRICS VISUALISATION
# ----------------------------------------------------------------

print("\n" + "-"*60)
print("PART E: Scalability Metrics Visualisation")
print("-"*60)

# Simulate TPS growth with scaling solutions
solutions = ['Base Chain', '+ Layer 2', '+ Sharding', 'Full Scaling']
tps_values = [15, 2000, 5000, 15000]
latency = [10000, 100, 1000, 50]  # ms

fig, axes = plt.subplots(1, 2, figsize=(12, 4))

ax1 = axes[0]
ax1.bar(solutions, tps_values, color='teal', alpha=0.7)
ax1.set_ylabel('TPS (estimated)')
ax1.set_title('Throughput Improvement')
ax1.grid(True, alpha=0.3)

ax2 = axes[1]
ax2.bar(solutions, latency, color='orange', alpha=0.7)
ax2.set_ylabel('Latency (ms)')
ax2.set_title('Latency Reduction')
ax2.grid(True, alpha=0.3)

plt.tight_layout()
plt.savefig('scalability_metrics.png', dpi=300, bbox_inches='tight')
plt.show()
print("Scalability metrics chart saved as 'scalability_metrics.png'")

# ----------------------------------------------------------------
# PART F: SUMMARY AND RECOMMENDATIONS
# ----------------------------------------------------------------

print("\n" + "="*70)
print("PART F: Summary and Recommendations")
print("="*70)

print("""
Blockchain Scalability – Key Takeaways:

1. Scalability is a core challenge in blockchain (trilemma).
2. Layer 1: increases base chain capacity (block size, sharding).
3. Layer 2: moves computation off-chain while using main chain for security.
4. Payment channels enable instant, low-cost transfers.
5. Rollups (Optimistic and ZK) batch transactions and post compressed data.
6. Sharding partitions the state for parallel processing.
7. Choice depends on use case: speed, security, cost, complexity.

Recommendations:
  - Use Layer 2 for high-frequency transactions (e.g., payments).
  - Use rollups for scaling smart contract platforms.
  - Consider sharding for high-throughput, general-purpose blockchains.
  - Evaluate trade-offs between security and speed.
  - Stay updated on evolving solutions (e.g., ZK-EVMs, validium).
""")