API Infrastructure Documentation

Overview

This document describes the core API infrastructure implemented for the Customizable Finance Dashboard. The system supports real-time data visualization, dynamic API integration, intelligent caching, and rate limiting.

Architecture

Core Components

┌─────────────────────────────────────────────────────────────┐
│                    Widget Components                        │
│            (Cards, Tables, Charts)                          │
└─────────────────┬───────────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────────────────────┐
│              Universal Adapter                              │
│        (Maps API responses to widget data)                  │
└─────────────────┬───────────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────────────────────┐
│                  Fetcher Layer                              │
│     (Handles requests, caching, rate limiting)              │
└─────────────────┬───────────────────────────────────────────┘
                  │
          ┌───────┴───────┐
          ▼               ▼
    ┌──────────┐    ┌──────────┐
    │  Cache   │    │   Rate   │
    │  System  │    │ Limiter  │
    └──────────┘    └──────────┘
          │               │
          └───────┬───────┘
                  ▼
         ┌─────────────────┐
         │  External APIs  │
         └─────────────────┘

File Structure

1. Type Definitions (app/types/api.types.ts)

Defines all API-related types:

• Error Types: ApiError, RateLimitError, NetworkError, ValidationError
• Data Types: CardData, TableData, ChartData
• Configuration Types: FetcherOptions, RateLimitConfig, CacheEntry

2. JSON Path Utility (app/lib/utils/jsonPath.ts)

Purpose: Dynamic field selection from nested API responses

Key Functions:

• getByPath(obj, path): Extract values using dot notation
• setByPath(obj, path, value): Set values for testing
• getAllPaths(obj): Extract all available paths from JSON
• pathExists(obj, path): Validate path existence
• getPathType(obj, path): Determine value type at path

Supported Syntax:

// Simple nested access
getByPath(data, "user.name")

// Array index access
getByPath(data, "items[0].price")

// Wildcard array access
getByPath(data, "stocks[*]")

// Deep nested
getByPath(data, "data.results[0].metrics.price")

3. Validation Schemas (app/lib/schemas/validation.ts)

Purpose: Runtime validation using Zod

Schemas:

• widgetConfigSchema: Validates widget configuration
  • Uses discriminated union for type-specific validation
  • Ensures card/table/chart have correct mapping fields
• dashboardConfigSchema: Validates entire dashboard config
• columnMappingSchema: Validates table column definitions
• formattingSchema: Validates display formatting options

Type Safety:

type WidgetConfig = z.infer<typeof widgetConfigSchema>;
// Automatically inferred from schema

4. Rate Limiter (app/lib/api/rateLimiter.ts)

Purpose: Prevent API abuse using sliding window algorithm

Features:

• Per-endpoint tracking
• Configurable limits and windows
• Automatic cleanup of old timestamps
• Retry-after calculation

API:

// Check if request is allowed
rateLimiter.canMakeRequest(key, limit, windowMs)

// Record a request
rateLimiter.recordRequest(key)

// Get retry delay
rateLimiter.getRetryAfter(key, windowMs)

// Get current count
rateLimiter.getRequestCount(key, windowMs)

Default Configuration:

• 60 requests per 60 seconds (1 minute)
• Automatic cleanup every 5 minutes

5. Fetcher (app/lib/api/fetcher.ts)

Purpose: Unified API request handler with caching and error handling

Features:

In-Memory Cache (ApiCache)

• TTL-based expiration
• Automatic cleanup
• Per-URL caching
• Cache hit tracking

Fetch Functions

// Basic fetch with all features
fetchWidgetData<T>(url, options)

// Fetch with automatic retries
fetchWithRetry<T>(url, options)

Error Handling

• Automatic error normalization
• Retry logic with exponential backoff
• Timeout handling
• CORS error detection

React Query Integration

export const queryClient = new QueryClient({
  defaultOptions: {
    queries: {
      staleTime: 5000,
      gcTime: 300000,
      retry: 3,
      retryDelay: (attemptIndex) => 
        Math.min(1000 * 2 ** attemptIndex, 30000),
    },
  },
});

6. Universal Adapter (app/lib/adapters/universalAdapter.ts)

Purpose: Map arbitrary API responses to widget data formats

Adapters:

Card Adapter

UniversalAdapter.mapToCard(response, config)

• Extracts single value and label
• Auto-detects trend information (changePercent)
• Formats values based on configuration
• Returns: CardData

Table Adapter

UniversalAdapter.mapToTable(response, config)

• Extracts rows from any path
• Maps columns using JSON paths
• Supports pagination metadata
• Formats columns by type
• Returns: TableData

Chart Adapter

UniversalAdapter.mapToChart(response, config)

• Supports line and candlestick charts
• Extracts OHLCV data for candlesticks
• Auto-detects metadata (symbol, interval)
• Returns: ChartData

Value Formatting

Supports:

• Currency: $1,234.56
• Percentage: 12.34%
• Number: 1234.56
• Custom prefix/suffix

Usage Examples

Example 1: Card Widget Configuration

const cardConfig: WidgetConfig = {
  id: "uuid-here",
  type: "card",
  name: "Apple Stock Price",
  apiUrl: "https://api.example.com/quote/AAPL",
  refreshInterval: 60,
  mapping: {
    rootPath: "data",
    valuePath: "price",
    labelPath: "symbol"
  },
  formatting: {
    valueType: "currency",
    currencyCode: "USD",
    decimalPlaces: 2
  },
  layout: { x: 0, y: 0, w: 3, h: 2 },
  cache: { enabled: true, ttl: 300 },
  createdAt: new Date().toISOString(),
  updatedAt: new Date().toISOString()
};

Example 2: Table Widget with Filtering

const tableConfig: WidgetConfig = {
  id: "uuid-here",
  type: "table",
  name: "Top Stocks",
  apiUrl: "https://api.example.com/stocks/top",
  refreshInterval: 120,
  mapping: {
    rootPath: "data",
    rowsPath: "stocks",
    columns: [
      { id: "symbol", label: "Symbol", path: "ticker", type: "text" },
      { id: "price", label: "Price", path: "current_price", type: "currency" },
      { id: "change", label: "Change", path: "change_percent", type: "percentage" }
    ]
  },
  pagination: {
    enabled: true,
    pageSize: 10
  },
  layout: { x: 0, y: 2, w: 6, h: 4 },
  createdAt: new Date().toISOString(),
  updatedAt: new Date().toISOString()
};

Example 3: Candlestick Chart

const chartConfig: WidgetConfig = {
  id: "uuid-here",
  type: "chart",
  name: "AAPL Price History",
  apiUrl: "https://api.example.com/historical/AAPL",
  refreshInterval: 300,
  mapping: {
    rootPath: "data",
    dataPath: "timeSeries",
    xPath: "timestamp",
    yPath: "close",
    chartType: "candlestick",
    openPath: "open",
    highPath: "high",
    lowPath: "low",
    closePath: "close",
    volumePath: "volume"
  },
  layout: { x: 6, y: 0, w: 6, h: 6 },
  createdAt: new Date().toISOString(),
  updatedAt: new Date().toISOString()
};

Example 4: Fetching and Adapting Data

import { fetchWidgetData } from '@/app/lib/api/fetcher';
import { adaptResponse } from '@/app/lib/adapters/universalAdapter';
import { validateWidgetConfig } from '@/app/lib/schemas/validation';

async function loadWidget(config: unknown) {
  // 1. Validate configuration
  const validConfig = validateWidgetConfig(config);
  
  // 2. Fetch data with caching and rate limiting
  const response = await fetchWidgetData(validConfig.apiUrl, {
    cache: validConfig.cache,
    timeout: 30000
  });
  
  // 3. Adapt response to widget format
  const widgetData = adaptResponse(response.data, validConfig);
  
  return widgetData;
}

API Response Examples

Stock Quote API Response

{
  "data": {
    "symbol": "AAPL",
    "price": 178.25,
    "changePercent": 2.45,
    "timestamp": "2024-01-23T10:30:00Z"
  }
}

Mapping: rootPath: "data", valuePath: "price", labelPath: "symbol"

Stock List API Response

{
  "data": {
    "stocks": [
      { "ticker": "AAPL", "current_price": 178.25, "change_percent": 2.45 },
      { "ticker": "GOOGL", "current_price": 142.50, "change_percent": -1.20 }
    ],
    "total": 100
  }
}

Mapping: rootPath: "data", rowsPath: "stocks"

Historical Data API Response

{
  "data": {
    "symbol": "AAPL",
    "timeSeries": [
      {
        "timestamp": "2024-01-23T09:30:00Z",
        "open": 177.50,
        "high": 178.80,
        "low": 177.20,
        "close": 178.25,
        "volume": 1234567
      }
    ]
  }
}

Mapping: rootPath: "data", dataPath: "timeSeries"

Error Handling

Error Types and Recovery

try {
  const data = await fetchWidgetData(url);
} catch (error) {
  if (error instanceof RateLimitError) {
    // Wait and retry
    await new Promise(resolve => setTimeout(resolve, error.retryAfter));
    // Retry logic...
  } else if (error instanceof ApiError) {
    if (error.statusCode === 401) {
      // Handle authentication
    } else if (error.statusCode >= 500) {
      // Server error - retry
    }
  } else if (error instanceof NetworkError) {
    // Network issue - show offline message
  }
}

Performance Considerations

Caching Strategy

• Default TTL: 5 minutes (300 seconds)
• Configurable per widget
• Automatic cleanup prevents memory leaks

Rate Limiting

• Default: 60 requests/minute per endpoint
• Prevents API key quota exhaustion
• Calculates optimal retry timing

React Query Benefits

• Automatic background refetching
• Request deduplication
• Stale-while-revalidate pattern
• Optimistic updates support

Testing Utilities

Path Testing

import { getAllPaths } from '@/app/lib/utils/jsonPath';

const apiResponse = { /* ... */ };
const paths = getAllPaths(apiResponse);
// Returns: ["data.price", "data.symbol", "data.stocks[*].ticker", ...]

Cache Management

import { clearAllCaches, clearCacheForUrl } from '@/app/lib/api/fetcher';

// Clear everything
clearAllCaches();

// Clear specific URL
clearCacheForUrl('https://api.example.com/quote/AAPL');

Rate Limiter Management

import { rateLimiter } from '@/app/lib/api/rateLimiter';

// Check current usage
const count = rateLimiter.getRequestCount(key);

// Clear for testing
rateLimiter.clearKey(key);

Next Steps

With this infrastructure in place, you can now:

1. Create Widget Components that use these adapters
2. Implement Dashboard Grid with drag-and-drop
3. Build Configuration UI for widget setup
4. Add Real-time Updates using React Query
5. Implement State Management with Zustand

The foundation is solid and follows best practices for:

• Type safety (TypeScript + Zod)
• Performance (caching + rate limiting)
• Error handling (normalized errors)
• Flexibility (dynamic path mapping)