Exa Reliability Patterns

Overview

Production-grade reliability patterns for Exa integrations.

Prerequisites

Understanding of circuit breaker pattern
opossum or similar library installed
Queue infrastructure for DLQ
Caching layer for fallbacks

Circuit Breaker

import CircuitBreaker from 'opossum';

const exaBreaker = new CircuitBreaker(
  async (operation: () => Promise<any>) => operation(),
  {
    timeout: 30000,
    errorThresholdPercentage: 50,
    resetTimeout: 30000,
    volumeThreshold: 10,
  }
);

// Events
exaBreaker.on('open', () => {
  console.warn('Exa circuit OPEN - requests failing fast');
  alertOps('Exa circuit breaker opened');
});

exaBreaker.on('halfOpen', () => {
  console.info('Exa circuit HALF-OPEN - testing recovery');
});

exaBreaker.on('close', () => {
  console.info('Exa circuit CLOSED - normal operation');
});

// Usage
async function safeExaCall<T>(fn: () => Promise<T>): Promise<T> {
  return exaBreaker.fire(fn);
}

Idempotency Keys

import { v4 as uuidv4 } from 'uuid';
import crypto from 'crypto';

// Generate deterministic idempotency key from input
function generateIdempotencyKey(
  operation: string,
  params: Record<string, any>
): string {
  const data = JSON.stringify({ operation, params });
  return crypto.createHash('sha256').update(data).digest('hex');
}

// Or use random key with storage
class IdempotencyManager {
  private store: Map<string, { key: string; expiresAt: Date }> = new Map();

  getOrCreate(operationId: string): string {
    const existing = this.store.get(operationId);
    if (existing && existing.expiresAt > new Date()) {
      return existing.key;
    }

    const key = uuidv4();
    this.store.set(operationId, {
      key,
      expiresAt: new Date(Date.now() + 24 * 60 * 60 * 1000),
    });
    return key;
  }
}

Bulkhead Pattern

import PQueue from 'p-queue';

// Separate queues for different operations
const exaQueues = {
  critical: new PQueue({ concurrency: 10 }),
  normal: new PQueue({ concurrency: 5 }),
  bulk: new PQueue({ concurrency: 2 }),
};

async function prioritizedExaCall<T>(
  priority: 'critical' | 'normal' | 'bulk',
  fn: () => Promise<T>
): Promise<T> {
  return exaQueues[priority].add(fn);
}

// Usage
await prioritizedExaCall('critical', () =>
  exaClient.processPayment(order)
);

await prioritizedExaCall('bulk', () =>
  exaClient.syncCatalog(products)
);

Timeout Hierarchy

const TIMEOUT_CONFIG = {
  connect: 5000,      // Initial connection
  request: 30000,     // Standard requests
  upload: 120000,     // File uploads
  longPoll: 300000,   // Webhook long-polling
};

async function timedoutExaCall<T>(
  operation: 'connect' | 'request' | 'upload' | 'longPoll',
  fn: () => Promise<T>
): Promise<T> {
  const timeout = TIMEOUT_CONFIG[operation];

  return Promise.race([
    fn(),
    new Promise<never>((_, reject) =>
      setTimeout(() => reject(new Error(`Exa ${operation} timeout`)), timeout)
    ),
  ]);
}

Graceful Degradation

interface ExaFallback {
  enabled: boolean;
  data: any;
  staleness: 'fresh' | 'stale' | 'very_stale';
}

async function withExaFallback<T>(
  fn: () => Promise<T>,
  fallbackFn: () => Promise<T>
): Promise<{ data: T; fallback: boolean }> {
  try {
    const data = await fn();
    // Update cache for future fallback
    await updateFallbackCache(data);
    return { data, fallback: false };
  } catch (error) {
    console.warn('Exa failed, using fallback:', error.message);
    const data = await fallbackFn();
    return { data, fallback: true };
  }
}

Dead Letter Queue

interface DeadLetterEntry {
  id: string;
  operation: string;
  payload: any;
  error: string;
  attempts: number;
  lastAttempt: Date;
}

class ExaDeadLetterQueue {
  private queue: DeadLetterEntry[] = [];

  add(entry: Omit<DeadLetterEntry, 'id' | 'lastAttempt'>): void {
    this.queue.push({
      ...entry,
      id: uuidv4(),
      lastAttempt: new Date(),
    });
  }

  async processOne(): Promise<boolean> {
    const entry = this.queue.shift();
    if (!entry) return false;

    try {
      await exaClient[entry.operation](entry.payload);
      console.log(`DLQ: Successfully reprocessed ${entry.id}`);
      return true;
    } catch (error) {
      entry.attempts++;
      entry.lastAttempt = new Date();

      if (entry.attempts < 5) {
        this.queue.push(entry);
      } else {
        console.error(`DLQ: Giving up on ${entry.id} after 5 attempts`);
        await alertOnPermanentFailure(entry);
      }
      return false;
    }
  }
}

Health Check with Degraded State

type HealthStatus = 'healthy' | 'degraded' | 'unhealthy';

async function exaHealthCheck(): Promise<{
  status: HealthStatus;
  details: Record<string, any>;
}> {
  const checks = {
    api: await checkApiConnectivity(),
    circuitBreaker: exaBreaker.stats(),
    dlqSize: deadLetterQueue.size(),
  };

  const status: HealthStatus =
    !checks.api.connected ? 'unhealthy' :
    checks.circuitBreaker.state === 'open' ? 'degraded' :
    checks.dlqSize > 100 ? 'degraded' :
    'healthy';

  return { status, details: checks };
}

Instructions

Step 1: Implement Circuit Breaker

Wrap Exa calls with circuit breaker.

Step 2: Add Idempotency Keys

Generate deterministic keys for operations.

Step 3: Configure Bulkheads

Separate queues for different priorities.

Step 4: Set Up Dead Letter Queue

Handle permanent failures gracefully.

Output

Circuit breaker protecting Exa calls
Idempotency preventing duplicates
Bulkhead isolation implemented
DLQ for failed operations

Error Handling

| Issue | Cause | Solution | |-------|-------|----------| | Circuit stays open | Threshold too low | Adjust error percentage | | Duplicate operations | Missing idempotency | Add idempotency key | | Queue full | Rate too high | Increase concurrency | | DLQ growing | Persistent failures | Investigate root cause |

Examples

Quick Circuit Check

const state = exaBreaker.stats().state;
console.log('Exa circuit:', state);

Resources

Next Steps

For policy enforcement, see exa-policy-guardrails.