ERGENEKON Engine Docs

ERGENEKON is a deterministic record-and-replay debugger for distributed Node.js systems. It intercepts all I/O at runtime, stores a byte-perfect recording, and lets you replay it locally — without a network, database, or external dependencies.

Package Overview

⚡ Quick Start

Get from zero to your first recorded replay in under 2 minutes.

1. Install the packages

   npm install @ergenekon/probe @ergenekon/collector

2. Start the Collector
   

   The Collector runs as a separate HTTP server on port 4380.

   npx ergenekon-collector --port 4380 --storage ./recordings

3. Add the Probe to your Express app

   TypeScriptimport { ErgenekonProbe } from '@ergenekon/probe';
   import express from 'express';
   
   const app = express();
   const probe = new ErgenekonProbe({
     serviceName: 'checkout-service',
     collectorUrl: 'http://localhost:4380',
   });
   
   app.use(probe.middleware()); // Must be first middleware
   
   app.get('/api/orders', async (req, res) => {
     // Your existing code — untouched
     const orders = await db.query('SELECT * FROM orders');
     res.json(orders);
   });
   
   app.listen(3000);

4. Make a request — it's recorded automatically

   curl http://localhost:3000/api/orders

5. View and replay the session

   npx ergenekon sessions       # list recordings
   npx ergenekon replay <id>   # replay and verify

🔄 How It Works

ERGENEKON operates on a simple principle: intercept all sources of non-determinism at I/O boundaries, record them, and replay them exactly during debugging.

The Recording Pipeline

Incoming Request
     │
     ▼
 ┌──────────────────────────────────────────────────────┐
 │  PROBE (your service)                                │
 │                                                      │
 │  ① HTTP middleware wraps the request                 │
 │  ② All I/O interceptions installed:                  │
 │    • Date.now(), Math.random(), crypto.randomUUID()  │
 │    • http.request(), fetch()                         │
 │    • fs.readFile(), fs.writeFile()                   │
 │    • dns.lookup(), dns.resolve()                     │
 │    • pg.query(), mysql2.execute()                    │
 │    • setTimeout(), setInterval()                     │
 │  ③ Each call → ErgenekonEvent (ULID + HLC timestamp) │
 │  ④ Events buffered in SpillBuffer                    │
 └──────────────────────────────────┬───────────────────┘
                                    │ POST /sessions
                                    ▼
                        ┌─────────────────────┐
                        │  COLLECTOR SERVER   │
                        │                     │
                        │  Rate limiting      │
                        │  SHA-256 checksum   │
                        │  Durable fsync write│
                        │  REST API           │
                        └─────────┬───────────┘
                                  │
                    ┌─────────────▼──────────────┐
                    │  REPLAY ENGINE             │
                    │                            │
                    │  Loads recording session   │
                    │  Mocks all I/O from events │
                    │  Re-executes your code     │
                    │  Verifies byte-for-byte    │
                    └────────────────────────────┘

Event Types

⚙️ Probe Configuration

TypeScriptconst probe = new ErgenekonProbe({
  // REQUIRED
  serviceName:  'payment-service',

  // Collector endpoint (default: http://localhost:4380)
  collectorUrl: process.env.ERGENEKON_COLLECTOR_URL,

  // Disable in test environments
  enabled: process.env.NODE_ENV !== 'test',

  // Record every request (1.0 = 100%, 0.1 = 10%) — Pro tier
  samplingRate: 1.0,

  // Auto-redact these headers in all events
  redactHeaders: ['authorization', 'cookie', 'x-api-key'],

  // Max body size to capture (bytes) — default 512KB
  maxBodyBytes: 524288,

  // Advanced sampling config — Pro tier
  sampling: {
    strategy: 'reservoir',   // 'deterministic' | 'reservoir' | 'adaptive'
    rate: 0.1,              // sample 10% of requests
    reservoirSize: 100,     // keep 100 samples per minute
    alwaysRecord: [          // patterns that always record
      'POST /api/payments',
      'DELETE *',
    ],
  },
});

Full Configuration Reference

🎲 Smart Sampling

Pro Sampling lets you record a representative subset of requests — reducing overhead to near zero in high-traffic environments.

Strategies

TypeScriptconst probe = new ErgenekonProbe({
  serviceName: 'api-gateway',
  sampling: {
    strategy: 'adaptive',
    rate: 0.05,             // baseline 5%
    alwaysRecord: [        // always record these — regardless of rate
      'POST /api/checkout',
      'POST /api/auth/*',
      'DELETE *',
    ],
    neverRecord: [         // never record health checks / metrics
      'GET /health',
      'GET /metrics',
    ],
  },
});

🔒 PII Redaction

Pro ERGENEKON includes a deep redaction engine that recursively walks recorded payloads and scrubs sensitive fields before they are written to disk.

TypeScriptimport { redactDeep } from '@ergenekon/probe';

const redacted = redactDeep(payload, {
  // Redact these field names (case-insensitive)
  fieldNames: ['password', 'ssn', 'creditCard', 'apiKey'],

  // Redact nested paths (supports * wildcard)
  pathPatterns: ['user.*.secret', 'payment.card.*'],

  // Auto-detect emails, JWTs, credit card numbers, SSNs
  autoDetect: true,

  // Custom redactor function
  customRedactor: (field, value, path) => {
    if (path.includes('billing')) return '[BILLING_REDACTED]';
    return undefined; // undefined = use default behavior
  },

  replacement: '[REDACTED]',
  maxDepth: 20,
});

Auto-detected Patterns

• Credit card numbers (Luhn-valid 13–19 digit sequences)
• JSON Web Tokens (three-part base64url strings)
• Email addresses
• US Social Security Numbers (XXX-XX-XXXX format)
• Private keys (PEM header detection)

📡 Running the Collector

Standalone (Development)

# Start with defaults (port 4380, ./recordings directory)
npx @ergenekon/collector

# Custom port and storage path
npx @ergenekon/collector --port 4380 --storage /var/ergenekon/recordings

# With license
ERGENEKON_LICENSE_KEY='...' npx @ergenekon/collector

Programmatic (Node.js)

TypeScriptimport { CollectorServer } from '@ergenekon/collector';

const server = new CollectorServer({
  port: 4380,
  storageDir: '/var/ergenekon/recordings',
});

await server.start();
console.log('Collector listening on :4380');

Defense Layers

🌐 Collector REST API

All endpoints served on http://localhost:4380 (default).

List Sessions

GET /sessions

Response:
{
  "sessions": [
    {
      "id": "01HWXYZ...",
      "traceId": "abc123",
      "serviceName": "checkout-service",
      "startTime": "2026-04-15T18:00:00Z",
      "durationMs": 234,
      "eventCount": 47,
      "nodeVersion": "v22.0.0"
    }
  ],
  "total": 1
}

Get Session Detail

GET /sessions/01HWXYZ...

Response:
{
  "id": "01HWXYZ...",
  "traceId": "abc123",
  "events": [
    {
      "id": "01HWXYZ001",
      "type": "http_request_in",
      "timestamp": { "wallTime": 1712345678901, "logical": 0 },
      "data": { "method": "POST", "url": "/api/checkout", "headers": {...} }
    },
    ...
  ]
}

📊 Dashboard

The Time-Travel Debugger dashboard gives you a visual interface to browse recordings, inspect events, and debug issues.

Connect to Your Collector

Open the dashboard and enter your collector URL in the connection panel:

# Option 1: Use the UI
1. Open ergenekon-dashboard.vercel.app
2. Enter http://localhost:4380 in the Collector field
3. Click Connect

# Option 2: URL parameter
https://ergenekon-dashboard.vercel.app?collector=http://localhost:4380

# Option 3: Self-host the dashboard
cd dashboard-demo && npx serve .

Features

• Live Mode — Polls your collector every 5s for new recordings
• Session List — Browse all recorded sessions with method, path, status
• Event Timeline — Scrub through events with the timeline slider
• Event Inspector — View full request/response data, DB queries, etc.
• Keyboard Shortcuts — ← → navigate, Space play/pause, ? help
• Search — Filter sessions by path, method, or service name
• Auto-reconnect — Saves your collector URL in localStorage

💻 CLI Reference

npx ergenekon <command> [options]

# Or install globally:
npm install -g @ergenekon/cli
ergenekon <command>

Commands

Environment Variables for CLI

Examples

# List sessions from a remote Collector
ERGENEKON_COLLECTOR_URL=http://prod-collector:4380 ergenekon sessions

# Show color timeline of session events
ergenekon timeline 01HWXYZ...

# Export for offline debugging
ergenekon export 01HWXYZ... ./debug-session.prdx

# Replay and compare output
ergenekon replay 01HWXYZ...
# ✅ PERFECT REPLAY — BYTE-FOR-BYTE IDENTICAL
# ✓ Date.now() → 1712345678901  (deterministic)
# ✓ Math.random() → 0.73421847  (deterministic)
# ✓ Response body → identical    (byte-for-byte)

⏪ Replay Engine

The replay engine re-runs your application code against a recorded session, mocking all I/O with the captured values — so you can step through the exact execution path that caused a production bug.

Programmatic Replay

TypeScriptimport { MockLayer } from '@ergenekon/replay';
import { importSessionJSON } from '@ergenekon/core';

const session = await importSessionJSON('./session.json');
const mock = new MockLayer(session);

// Install mocks before running your code
mock.install();

try {
  // Run your handler with the recorded request
  await yourRequestHandler(session.events[0].data);
} finally {
  mock.uninstall();
  const report = mock.verificationReport();
  console.log(report); // pass/fail per event
}

🔑 License System

Licenses are Ed25519-signed JWT tokens validated entirely offline — no network call at startup or runtime.

License Discovery (auto, in order)

1. ERGENEKON_LICENSE_KEY env var — inline JSON token
2. ERGENEKON_LICENSE env var — path to license file
3. .ergenekon-license.json in the current working directory
4. ~/.ergenekon-license.json in the home directory

License File Format

JSON{
  "token": "eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9...",
  "tier": "pro",
  "licensee": "Acme Corp",
  "issuedAt": "2026-04-15T00:00:00Z",
  "expiresAt": "2027-04-15T00:00:00Z"
}

Setting via Environment (recommended for production)

ERGENEKON_LICENSE_KEY='{"token":"eyJ...","tier":"pro"}' node app.js

🐳 Docker Deployment

Docker Compose (Recommended)

docker-compose.ymlversion: '3.9'
services:
  ergenekon-collector:
    image: node:22-alpine
    working_dir: /app
    command: npx @ergenekon/collector --port 4380 --storage /data
    ports:
      - "4380:4380"
    volumes:
      - ergenekon-data:/data
    environment:
      - ERGENEKON_LICENSE_KEY=${ERGENEKON_LICENSE_KEY}
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:4380/health"]
      interval: 30s
      timeout: 5s
      retries: 3

volumes:
  ergenekon-data:

docker compose up -d
docker compose logs -f ergenekon-collector

Standalone Dockerfile

DockerfileFROM node:22-alpine
WORKDIR /app
RUN npm install -g @ergenekon/collector
EXPOSE 4380
VOLUME /data
CMD ["ergenekon-collector", "--port", "4380", "--storage", "/data"]

☸️ Kubernetes Deployment

collector-deployment.yamlapiVersion: apps/v1
kind: Deployment
metadata:
  name: ergenekon-collector
  namespace: observability
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ergenekon-collector
  template:
    metadata:
      labels:
        app: ergenekon-collector
    spec:
      containers:
        - name: collector
          image: node:22-alpine
          command: ["npx", "@ergenekon/collector"]
          args: ["--port", "4380", "--storage", "/data"]
          ports:
            - containerPort: 4380
          env:
            - name: ERGENEKON_LICENSE_KEY
              valueFrom:
                secretKeyRef:
                  name: ergenekon-license
                  key: key
          volumeMounts:
            - name: recordings
              mountPath: /data
          resources:
            requests:
              memory: "128Mi"
              cpu: "100m"
            limits:
              memory: "512Mi"
              cpu: "500m"
          livenessProbe:
            httpGet:
              path: /health
              port: 4380
            initialDelaySeconds: 5
            periodSeconds: 15
      volumes:
        - name: recordings
          persistentVolumeClaim:
            claimName: ergenekon-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: ergenekon-collector
  namespace: observability
spec:
  selector:
    app: ergenekon-collector
  ports:
    - port: 4380
      targetPort: 4380

Secret for License Key

kubectl create secret generic ergenekon-license \
  --from-literal=key='{"token":"eyJ...","tier":"enterprise"}' \
  -n observability

🔧 Environment Variables