1. LEARNING OBJECTIVES

By the end of this lesson, you will be able to:

  • Understand REST API architecture and HTTP methods (GET, POST, PUT, DELETE).

  • Authenticate with financial APIs using API keys, OAuth, and JWT.

  • Make API requests using the requests library.

  • Parse JSON responses from financial data APIs.

  • Implement webhooks to receive real-time financial data.

  • Build a simple API client for a financial data provider.

  • Apply asynchronous programming with asyncio and aiohttp.

  • Handle rate limiting and retry logic.

  • Build a production-grade financial data ingestion pipeline.


2. REST API ARCHITECTURE

2.1 What is REST?

REST (Representational State Transfer) is an architectural style for designing networked applications.

Key Principles:

  • Stateless: Each request contains all necessary information.

  • Client-Server: Separation of concerns.

  • Cacheable: Responses can be cached.

  • Uniform Interface: Consistent API design.

2.2 HTTP Methods

 
 
Method Purpose Financial Application
GET Retrieve data Get stock prices, account balance
POST Create new resource Place an order, create a subscription
PUT Update a resource Update account settings
DELETE Delete a resource Cancel an order, delete an API key
PATCH Partial update Update a specific field

2.3 HTTP Status Codes

 
 
Code Range Meaning Example
2xx Success 200 OK, 201 Created
3xx Redirection 301 Moved Permanently
4xx Client Error 400 Bad Request, 401 Unauthorized, 404 Not Found
5xx Server Error 500 Internal Server Error, 503 Service Unavailable

3. MAKING API REQUESTS WITH REQUESTS

3.1 Installation and Import

python
pip install requests
import requests
import json

3.2 Basic GET Request

python
# Simple GET request
url = "https://api.example.com/v1/stock/AAPL"
response = requests.get(url)

# Check status
if response.status_code == 200:
    data = response.json()  # Parse JSON
    print(data)
else:
    print(f"Error: {response.status_code}")
    print(response.text)

3.3 GET Request with Parameters

python
# Query parameters
params = {
    'symbol': 'AAPL',
    'start_date': '2024-01-01',
    'end_date': '2024-12-31',
    'interval': '1d'
}

response = requests.get(
    "https://api.example.com/v1/historical_prices",
    params=params
)

# URL becomes: https://api.example.com/v1/historical_prices?symbol=AAPL&start_date=2024-01-01&...

3.4 POST Request (Creating Resources)

python
# POST request with JSON body
order_data = {
    'symbol': 'AAPL',
    'side': 'BUY',
    'quantity': 100,
    'order_type': 'LIMIT',
    'limit_price': 150.50
}

headers = {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer YOUR_API_KEY'
}

response = requests.post(
    "https://api.example.com/v1/orders",
    json=order_data,
    headers=headers
)

if response.status_code == 201:
    order_id = response.json()['order_id']
    print(f"Order placed: {order_id}")

3.5 PUT and DELETE Requests

python
# PUT (Update)
update_data = {'limit_price': 151.00}
response = requests.put(
    f"https://api.example.com/v1/orders/{order_id}",
    json=update_data,
    headers=headers
)

# DELETE (Cancel)
response = requests.delete(
    f"https://api.example.com/v1/orders/{order_id}",
    headers=headers
)

4. API AUTHENTICATION

4.1 API Key Authentication

python
# API key in headers
headers = {
    'X-API-Key': 'YOUR_API_KEY',
    'Content-Type': 'application/json'
}

response = requests.get(url, headers=headers)

# API key in query string
params = {'apikey': 'YOUR_API_KEY'}
response = requests.get(url, params=params)

4.2 OAuth 2.0 Authentication

python
# Get access token
auth_data = {
    'grant_type': 'client_credentials',
    'client_id': 'CLIENT_ID',
    'client_secret': 'CLIENT_SECRET'
}

response = requests.post(
    "https://auth.example.com/oauth/token",
    data=auth_data
)

access_token = response.json()['access_token']

# Use token in subsequent requests
headers = {
    'Authorization': f'Bearer {access_token}',
    'Content-Type': 'application/json'
}

response = requests.get(
    "https://api.example.com/v1/data",
    headers=headers
)

4.3 JWT (JSON Web Token)

python
import jwt
import time

# Generate JWT
payload = {
    'sub': 'user_id_123',
    'iat': int(time.time()),
    'exp': int(time.time()) + 3600  # 1 hour expiry
}
token = jwt.encode(payload, 'SECRET_KEY', algorithm='HS256')

# Decode JWT
decoded = jwt.decode(token, 'SECRET_KEY', algorithms=['HS256'])

5. BUILDING A FINANCIAL API CLIENT

5.1 Complete Client Class

python
import requests
import time
from typing import Dict, List, Optional

class FinancialAPIClient:
    def __init__(self, api_key: str, base_url: str, timeout: int = 30):
        self.api_key = api_key
        self.base_url = base_url
        self.timeout = timeout
        self.session = requests.Session()
        self.session.headers.update({
            'X-API-Key': self.api_key,
            'Content-Type': 'application/json',
            'Accept': 'application/json'
        })
    
    def _request(self, method: str, endpoint: str, 
                 params: Optional[Dict] = None, 
                 data: Optional[Dict] = None) -> Dict:
        """Generic request handler with error handling."""
        url = f"{self.base_url}{endpoint}"
        
        try:
            response = self.session.request(
                method=method,
                url=url,
                params=params,
                json=data,
                timeout=self.timeout
            )
            response.raise_for_status()
            return response.json()
        except requests.exceptions.Timeout:
            raise Exception(f"Request to {url} timed out")
        except requests.exceptions.HTTPError as e:
            raise Exception(f"HTTP Error: {e.response.status_code} - {e.response.text}")
        except requests.exceptions.RequestException as e:
            raise Exception(f"Request failed: {str(e)}")
    
    def get_stock_price(self, symbol: str) -> Dict:
        """Get current stock price."""
        return self._request('GET', f'/stock/{symbol}/price')
    
    def get_historical_prices(self, symbol: str, start: str, end: str) -> Dict:
        """Get historical prices."""
        params = {
            'symbol': symbol,
            'start_date': start,
            'end_date': end
        }
        return self._request('GET', '/historical_prices', params=params)
    
    def place_order(self, symbol: str, side: str, quantity: int, 
                    order_type: str, limit_price: Optional[float] = None) -> Dict:
        """Place an order."""
        data = {
            'symbol': symbol,
            'side': side,
            'quantity': quantity,
            'order_type': order_type
        }
        if limit_price:
            data['limit_price'] = limit_price
        
        return self._request('POST', '/orders', data=data)
    
    def cancel_order(self, order_id: str) -> Dict:
        """Cancel an existing order."""
        return self._request('DELETE', f'/orders/{order_id}')
    
    def get_portfolio(self) -> Dict:
        """Get portfolio holdings."""
        return self._request('GET', '/portfolio')

5.2 Usage Example

python
# Initialize client
client = FinancialAPIClient(
    api_key="YOUR_API_KEY",
    base_url="https://api.example.com/v1"
)

# Get stock price
price = client.get_stock_price("AAPL")
print(f"AAPL: ${price['price']:.2f}")

# Get historical data
historical = client.get_historical_prices(
    symbol="AAPL",
    start="2024-01-01",
    end="2024-12-31"
)

# Place an order
order = client.place_order(
    symbol="AAPL",
    side="BUY",
    quantity=100,
    order_type="LIMIT",
    limit_price=150.50
)

# Get portfolio
portfolio = client.get_portfolio()

6. RATE LIMITING AND RETRY LOGIC

6.1 Rate Limiting Strategies

python
import time
from functools import wraps

def rate_limiter(calls_per_second=10):
    """Decorator to limit API call rate."""
    min_interval = 1.0 / calls_per_second
    last_called = [0.0]
    
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            elapsed = time.time() - last_called[0]
            if elapsed < min_interval:
                time.sleep(min_interval - elapsed)
            last_called[0] = time.time()
            return func(*args, **kwargs)
        return wrapper
    return decorator

# Usage
@rate_limiter(calls_per_second=5)
def make_api_request():
    response = requests.get("https://api.example.com/v1/data")
    return response.json()

6.2 Exponential Backoff Retry

python
def retry_with_backoff(max_retries=5, backoff_factor=2):
    """Decorator for retrying requests with exponential backoff."""
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            retries = 0
            delay = 1
            
            while retries < max_retries:
                try:
                    return func(*args, **kwargs)
                except Exception as e:
                    retries += 1
                    if retries >= max_retries:
                        raise
                    
                    print(f"Retry {retries}/{max_retries} after {delay}s delay")
                    time.sleep(delay)
                    delay *= backoff_factor
            
            return None
        return wrapper
    return decorator

# Usage
@retry_with_backoff(max_retries=5, backoff_factor=2)
def fetch_data():
    response = requests.get("https://api.example.com/v1/data")
    response.raise_for_status()
    return response.json()

6.3 Handling Rate Limit Headers

python
def rate_limited_request(url, headers=None):
    """Handle rate limit headers from response."""
    response = requests.get(url, headers=headers)
    
    # Check rate limit headers
    if 'X-RateLimit-Remaining' in response.headers:
        remaining = int(response.headers['X-RateLimit-Remaining'])
        reset = int(response.headers['X-RateLimit-Reset'])
        
        if remaining == 0:
            wait_time = reset - time.time()
            if wait_time > 0:
                time.sleep(wait_time)
    
    return response

7. WEBHOOKS

7.1 What are Webhooks?

Webhooks are HTTP callbacks that send real-time notifications when an event occurs. Instead of polling, the server pushes data to the client.

7.2 Building a Webhook Listener

python
from flask import Flask, request, jsonify
import hashlib
import hmac

app = Flask(__name__)

WEBHOOK_SECRET = "YOUR_WEBHOOK_SECRET"

@app.route('/webhook', methods=['POST'])
def handle_webhook():
    """Handle incoming webhook events."""
    try:
        # Verify signature
        signature = request.headers.get('X-Signature')
        payload = request.get_data()
        
        expected = hmac.new(
            WEBHOOK_SECRET.encode(),
            payload,
            hashlib.sha256
        ).hexdigest()
        
        if signature != expected:
            return jsonify({'error': 'Invalid signature'}), 401
        
        # Parse event
        event = request.get_json()
        event_type = event.get('event_type')
        
        # Process different event types
        if event_type == 'ORDER_FILLED':
            process_order_filled(event['data'])
        elif event_type == 'PRICE_UPDATE':
            process_price_update(event['data'])
        elif event_type == 'ACCOUNT_BALANCE':
            process_balance_update(event['data'])
        else:
            print(f"Unknown event type: {event_type}")
        
        return jsonify({'status': 'success'}), 200
        
    except Exception as e:
        print(f"Webhook error: {str(e)}")
        return jsonify({'error': str(e)}), 500

def process_order_filled(data):
    print(f"Order filled: {data['order_id']} at ${data['price']}")
    # Update database, send notification, etc.

def process_price_update(data):
    print(f"Price update: {data['symbol']} ${data['price']}")
    # Store in database, trigger alerts, etc.

def process_balance_update(data):
    print(f"Balance updated: ${data['balance']}")
    # Update portfolio, check margin, etc.

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

7.3 Webhook Security

python
def verify_webhook(payload, signature, secret):
    """Verify HMAC signature."""
    expected = hmac.new(
        secret.encode(),
        payload,
        hashlib.sha256
    ).hexdigest()
    return hmac.compare_digest(signature, expected)

def verify_timestamp(timestamp, max_age=300):
    """Prevent replay attacks."""
    return abs(time.time() - int(timestamp)) <= max_age

8. ASYNCHRONOUS PROGRAMMING

8.1 Why Async?

  • I/O Bound Tasks: API calls, database queries, file operations.

  • Concurrency: Handle multiple requests simultaneously.

  • Efficiency: No need for threading (reduces overhead).

8.2 Basic Asynchronous Code

python
import asyncio
import aiohttp
import time

async def fetch_price(session, symbol):
    """Asynchronously fetch stock price."""
    url = f"https://api.example.com/v1/stock/{symbol}"
    
    async with session.get(url) as response:
        data = await response.json()
        return symbol, data['price']

async def fetch_prices(symbols):
    """Fetch multiple prices concurrently."""
    async with aiohttp.ClientSession() as session:
        tasks = [fetch_price(session, symbol) for symbol in symbols]
        results = await asyncio.gather(*tasks)
        return dict(results)

# Usage
symbols = ['AAPL', 'GOOGL', 'MSFT', 'AMZN', 'META']
start = time.time()
prices = asyncio.run(fetch_prices(symbols))
print(f"Time: {time.time() - start:.2f}s")

8.3 Asynchronous API Client

python
import aiohttp
import asyncio
from typing import Dict, List

class AsyncFinancialAPIClient:
    def __init__(self, api_key: str, base_url: str):
        self.api_key = api_key
        self.base_url = base_url
        self.session = None
    
    async def __aenter__(self):
        self.session = aiohttp.ClientSession(
            headers={
                'X-API-Key': self.api_key,
                'Content-Type': 'application/json'
            }
        )
        return self
    
    async def __aexit__(self, exc_type, exc_val, exc_tb):
        await self.session.close()
    
    async def get_stock_price(self, symbol: str) -> float:
        url = f"{self.base_url}/stock/{symbol}"
        async with self.session.get(url) as response:
            data = await response.json()
            return data['price']
    
    async def get_multiple_prices(self, symbols: List[str]) -> Dict[str, float]:
        tasks = [self.get_stock_price(sym) for sym in symbols]
        results = await asyncio.gather(*tasks)
        return dict(zip(symbols, results))
    
    async def place_order(self, symbol: str, side: str, quantity: int, 
                          order_type: str, limit_price: float = None) -> Dict:
        data = {
            'symbol': symbol,
            'side': side,
            'quantity': quantity,
            'order_type': order_type
        }
        if limit_price:
            data['limit_price'] = limit_price
        
        url = f"{self.base_url}/orders"
        async with self.session.post(url, json=data) as response:
            return await response.json()

# Usage
async def main():
    async with AsyncFinancialAPIClient(
        api_key="YOUR_API_KEY",
        base_url="https://api.example.com/v1"
    ) as client:
        prices = await client.get_multiple_prices(['AAPL', 'GOOGL', 'MSFT'])
        print(prices)

asyncio.run(main())

8.4 Semaphore (Rate Limiting Async)

python
class AsyncRateLimiter:
    def __init__(self, calls_per_second=10):
        self.semaphore = asyncio.Semaphore(calls_per_second)
        self.min_interval = 1.0 / calls_per_second
        self._last_call = 0
    
    async def __aenter__(self):
        await self.semaphore.acquire()
        
        # Enforce minimum interval
        now = time.time()
        elapsed = now - self._last_call
        if elapsed < self.min_interval:
            await asyncio.sleep(self.min_interval - elapsed)
        
        self._last_call = time.time()
        return self
    
    async def __aexit__(self, *args):
        self.semaphore.release()

# Usage
async def rate_limited_request(session, url):
    async with AsyncRateLimiter(calls_per_second=5):
        async with session.get(url) as response:
            return await response.json()

9. BUILDING A DATA INGESTION PIPELINE

9.1 Complete Data Pipeline

python
import asyncio
import aiohttp
import pandas as pd
from typing import List, Dict
import logging

logging.basicConfig(level=logging.INFO)

class DataIngestionPipeline:
    def __init__(self, api_key: str, base_url: str):
        self.api_key = api_key
        self.base_url = base_url
        self.logger = logging.getLogger(__name__)
    
    async def fetch_historical_data(self, symbol: str, start: str, end: str) -> pd.DataFrame:
        """Fetch historical data for a single symbol."""
        url = f"{self.base_url}/historical/{symbol}"
        params = {'start': start, 'end': end}
        
        async with aiohttp.ClientSession() as session:
            async with session.get(url, params=params, 
                                 headers={'X-API-Key': self.api_key}) as response:
                data = await response.json()
                
                df = pd.DataFrame(data['prices'])
                df['Date'] = pd.to_datetime(df['Date'])
                df.set_index('Date', inplace=True)
                df['Symbol'] = symbol
                
                self.logger.info(f"Fetched {len(df)} rows for {symbol}")
                return df
    
    async def fetch_multiple(self, symbols: List[str], start: str, end: str) -> pd.DataFrame:
        """Fetch historical data for multiple symbols concurrently."""
        tasks = [self.fetch_historical_data(sym, start, end) for sym in symbols]
        results = await asyncio.gather(*tasks)
        return pd.concat(results, axis=0)
    
    def process_data(self, df: pd.DataFrame) -> pd.DataFrame:
        """Clean and process the data."""
        # Add returns
        df['Return'] = df.groupby('Symbol')['Close'].pct_change()
        
        # Add rolling statistics
        df['MA_50'] = df.groupby('Symbol')['Close'].rolling(50).mean().reset_index(0, drop=True)
        df['MA_200'] = df.groupby('Symbol')['Close'].rolling(200).mean().reset_index(0, drop=True)
        
        # Add volatility
        df['Volatility'] = df.groupby('Symbol')['Return'].rolling(20).std().reset_index(0, drop=True)
        
        # Drop NaN values
        df = df.dropna()
        
        return df
    
    async def run_pipeline(self, symbols: List[str], start: str, end: str) -> pd.DataFrame:
        """Run the complete pipeline."""
        self.logger.info(f"Starting data ingestion for {len(symbols)} symbols")
        
        # Fetch data
        raw_data = await self.fetch_multiple(symbols, start, end)
        self.logger.info(f"Fetched {len(raw_data)} rows")
        
        # Process data
        processed_data = self.process_data(raw_data)
        self.logger.info(f"Processed {len(processed_data)} rows")
        
        return processed_data

# Usage
async def main():
    pipeline = DataIngestionPipeline(
        api_key="YOUR_API_KEY",
        base_url="https://api.example.com/v1"
    )
    
    symbols = ['AAPL', 'GOOGL', 'MSFT', 'AMZN', 'META']
    data = await pipeline.run_pipeline(
        symbols=symbols,
        start='2024-01-01',
        end='2024-12-31'
    )
    
    print(data.head())
    data.to_csv('historical_prices.csv')

asyncio.run(main())

10. SUMMARY FOR THE FINANCE PRACTITIONER

  • REST APIs are the standard for financial data access. Use GET for data retrieval, POST for order placement.

  • Authentication is critical. Use API keys, OAuth, or JWT depending on the provider.

  • Rate Limiting must be respected. Implement exponential backoff and retry logic.

  • Webhooks provide real-time data delivery. Build secure endpoints with signature verification.

  • Asynchronous Programming improves performance for I/O-bound tasks. Use asyncio and aiohttp.

  • Data Pipelines should be modular, error-resistant, and well-documented.