Scaling & Deployment

The reference backends run beautifully as a single process. To run more than one instance — for availability or throughput — there is exactly one thing you must change: the two pieces of in-process state have to move to shared storage. Everything else (per-request client, service layer, verifier) is already stateless. This page is about that move, plus the deployment mechanics that make horizontal scaling safe.

The one thing that isn't stateless

By design, the API key is per-request and the SDK client is disposable, so there's no connection state to share. Only three things live in process memory:

Move the message store to Redis/DB

The store is written against a small interface — record on send, advance forward-only on webhook, read by id. Keep that surface identical and swap the backing map for Redis or your database. The forward-only, idempotent logic doesn't change.

The in-memory version (abridged):

// store/message-store.ts — the surface to preserve
export class MessageStore {
  record(input: { id: string; status?: string; /* … */ }): StoredMessage { /* … */ }
  updateStatus(id: string, next: string): StoredMessage | null { /* forward-only */ }
  get(id: string): StoredMessage | null { /* … */ }
}

A Redis-backed implementation keeps the same methods. The one new requirement is that the read-rank-compare-write in updateStatus has to be atomic — two instances handling concurrent webhooks must not both read the old status, both decide their event is a valid forward move, and both write, each unaware of the other. A Lua script runs atomically inside Redis (Redis executes it as a single, uninterruptible operation), so it's the natural tool here:

// The same RANK/TERMINAL rules from the in-memory version, expressed in Lua so
// Redis evaluates read-compare-write as one atomic step, not three round trips.
const FORWARD_ONLY_LUA = `
  local RANK = { QUEUED=0, ROUTED=1, SENT=2, DELIVERED=3, FAILED=3, READ=4 }
  local TERMINAL = { READ=true, FAILED=true }
  local key, next_status, updated_at = KEYS[1], ARGV[1], ARGV[2]

  local current = redis.call('HGET', key, 'status')
  if not current then return nil end                        -- unknown id
  if TERMINAL[current] then return current end               -- locked
  if RANK[next_status] == nil or RANK[next_status] <= RANK[current] then
    return current                                            -- no regress / dup
  end

  redis.call('HSET', key, 'status', next_status, 'updatedAt', updated_at)
  return next_status
`;

// store/redis-message-store.ts — same interface, shared backing
export class RedisMessageStore {
  constructor(private readonly redis: Redis) {}

  async record(input: { id: string; status?: string; /* … */ }) {
    const key = `msg:${input.id}`;
    await this.redis.hset(key, {
      status: input.status ?? 'QUEUED',
      createdAt: new Date().toISOString(),
      /* … */
    });
    await this.redis.expire(key, 60 * 60 * 24 * 7); // TTL, like the in-memory eviction
  }

  // Forward-only advance across instances — see FORWARD_ONLY_LUA above.
  async updateStatus(id: string, next: string) {
    return this.redis.eval(FORWARD_ONLY_LUA, 1, `msg:${id}`, next, new Date().toISOString());
  }

  async get(id: string) {
    const h = await this.redis.hgetall(`msg:${id}`);
    return Object.keys(h).length ? h : null;
  }
}

Make the signing secret readable by every instance

The in-memory secret store already flags this in its own comment:

// store/webhook-secret-store.ts
/*
 * Production note: a real multi-instance service would load the secret per
 * delivery from its own datastore (keyed by X-Webhook-ID / endpoint), not from
 * process memory.
 */

When a customer registers or rotates an endpoint (POST /api/webhooks, POST /api/webhooks/:id/rotate-secret), persist the returned secret to shared storage instead of a process-local Set. The receiver then loads the current secret(s) for that endpoint per delivery.

• Store secrets encrypted at rest, keyed by webhook config id (X-Webhook-ID).
• Never log them — see Security.
• During rotation, keep the old and new secret valid together. Sent may send space-separated signatures in X-Webhook-Signature; accept if any matches a current secret, then retire the old one. See Endpoint management.

Idempotency across instances

Acknowledge-fast plus idempotent processing already handles retries on one instance. Across instances the dedupe check must hit a shared store — otherwise the same event processed on A and B both look "new."

// Derive the key from the payload (X-Webhook-ID is NOT per-event).
const dedupeKey = `evt:${payload.message_id}:${payload.message_status}`;

// SET NX = "claim it only if unseen." Atomic and cross-instance.
const claimed = await redis.set(dedupeKey, '1', 'EX', 86_400, 'NX');
if (!claimed) return res.status(200).json({ received: true }); // already handled

Graceful shutdown

On deploy or scale-down your platform sends SIGTERM. Stop accepting new connections, let in-flight requests finish, then exit — with a hard timeout so a stuck request can't hang the rollout.

// src/server.ts
const shutdown = (signal: string) => {
  logger.info({ signal }, 'Shutting down...');
  server.close(() => {           // stop accepting, drain in-flight
    logger.info('Server closed');
    process.exit(0);
  });
  setTimeout(() => {             // hard cap so a stuck request can't block forever
    logger.error('Forced shutdown');
    process.exit(1);
  }, 10_000);
};

process.on('SIGTERM', () => shutdown('SIGTERM'));
process.on('SIGINT', () => shutdown('SIGINT'));

# ASGI servers (uvicorn/gunicorn) handle SIGTERM draining for you: they stop
# accepting, wait for in-flight requests, then exit. Tune the grace period, and
# use FastAPI's lifespan to release shared resources (Redis/DB pools) on shutdown.
#   uvicorn app.main:app --timeout-graceful-shutdown 10

Health, readiness, and liveness

A load balancer and an orchestrator ask different questions. Expose distinct endpoints so a temporary dependency blip doesn't get your pod killed.

// src/controllers/health.controller.ts
router.get('/', (_req, res) =>
  res.status(200).json({
    status: 'ok',
    backend: 'typescript',
    framework: 'express',
    uptime: process.uptime(),
    // …timestamp, version elided
    // No persistent Sent connection to report — the client is per-request.
    services: { sentdm: 'ready' },
  }),
);

// /live is a pure process check — never touches a dependency.
router.get('/live', (_req, res) => res.status(200).json({ alive: true }));

// /ready actually checks the shared dependency. A timed-out or failed PING
// means "don't route traffic here" — remove from rotation, don't restart.
router.get('/ready', async (_req, res) => {
  try {
    await redis.ping();
    res.status(200).json({ ready: true });
  } catch {
    res.status(503).json({ ready: false });
  }
});

# src/app/routers/health.py

@router.get("/health")
async def health_check():
    return {"status": "ok", "backend": "python", "framework": "fastapi"}

@router.get("/health/live")
async def liveness():
    return {"status": "alive"}   # pure process check — never touches a dependency

@router.get("/health/ready")
async def readiness():
    try:
        await redis_client.ping()
        return {"status": "ready"}
    except Exception:
        return JSONResponse({"status": "not_ready"}, status_code=503)

Containerization notes

• Config comes from the environment, not the image (the "twelve-factor app" convention). No SENT_DM_API_KEY — the key arrives per request. Config that is env-based (log level, rate-limit window, Redis/DB URL) comes from the environment; nothing secret is baked into the image.
• Logs to stdout as JSON; the platform collects them. See Observability.
• One process per container, scaled by replica count behind a load balancer.
• Honor SIGTERM (above). Run as a non-root user; multi-stage build for a lean image.
• Probes wired to /live and /ready.

Next steps