Instrumenting Logs, Metrics & Traces

When a message doesn't arrive, you need to answer one question fast: where in the two flows did it stop? Good observability makes that a single log query. The rules are the usual ones (structured logs, a correlation ID on every request, sensible levels), with one hard constraint specific to this integration: never log the API key, the signing secret, or message payloads.

Samples on this page are shown in TypeScript and Python; the patterns are framework-agnostic and transfer directly to the other supported SDK languages.

Structured logging

Log JSON, not free text. In TypeScript, pino is a good default: JSON in production, pretty-printed in development, with a base that stamps every line with the pid and environment.

// src/config/logger.ts
import pino from 'pino';
import { env } from './env';

export const logger = pino({
  level: env.LOG_LEVEL,
  transport:
    env.NODE_ENV === 'development'
      ? { target: 'pino-pretty', options: { colorize: true } }
      : undefined, // JSON to stdout in production
  base: { pid: process.pid, env: env.NODE_ENV },
});
# src/app/utils/logging.py
import logging

from pythonjsonlogger import jsonlogger  # pip install python-json-logger

def configure_logging(level: str = "INFO") -> None:
    handler = logging.StreamHandler()  # JSON to stdout — no log files
    handler.setFormatter(
        jsonlogger.JsonFormatter("%(asctime)s %(name)s %(levelname)s %(message)s")
    )
    logging.basicConfig(
        level=getattr(logging, level.upper(), logging.INFO),
        handlers=[handler],
    )
    # Every `extra={...}` field now lands as a queryable JSON key.

Write logs to stdout as JSON and let your platform (the container runtime, a log shipper, or your host) collect them. Don't manage log files inside the app. A stateless service shouldn't own local state, including logs. See Scaling & deployment.

Correlation: a request ID on every request

Attach a unique ID to each inbound request, echo it back in an X-Request-Id header, and include it in every log line for that request. That single field lets you reconstruct a whole request from scattered log lines, and lets a client quote it in a support ticket.

// src/middleware/request-logger.ts
export function requestLogger(req, res, next) {
  const start = Date.now();
  const requestId = randomUUID();
  res.setHeader('X-Request-Id', requestId);

  logger.trace(
    { requestId, method: req.method, path: req.path, ip: req.ip },
    'Incoming request',
  );

  res.on('finish', () => {
    const logData = {
      requestId,
      method: req.method,
      path: req.path,
      statusCode: res.statusCode,
      duration: `${Date.now() - start}ms`,
    };
    // Level chosen from the outcome — see below.
    if (res.statusCode >= 500) logger.error(logData, 'Request failed');
    else if (res.statusCode >= 400) logger.warn(logData, 'Request failed');
    else logger.debug(logData, 'Request completed');
  });

  next();
}
# src/app/middleware.py
class RequestIdMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request, call_next):
        request_id = request.headers.get("X-Request-ID", str(uuid.uuid4()))
        request.state.request_id = request_id
        response = await call_next(request)
        response.headers["X-Request-ID"] = request_id
        return response

class LoggingMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request, call_next):
        start = time.time()
        request_id = getattr(request.state, "request_id", "unknown")
        response = await call_next(request)
        logger.info("Request completed", extra={
            "request_id": request_id,
            "method": request.method,
            "path": request.url.path,
            "status_code": response.status_code,
            "duration_ms": round((time.time() - start) * 1000, 2),
        })
        return response

Honor an inbound X-Request-Id if the caller already set one (as shown in the preceding Python snippet), so the ID flows across service boundaries. Generate a fresh one only when none is present.

What to log at each level

Pick the level from the outcome, not the code path. The preceding request logger is the template: 2xxdebug, 4xxwarn, 5xxerror.

LevelUse forExample
trace / debugNormal request lifecycle, health checks."Incoming request," "Request completed"
infoBusiness milestones worth keeping."Message sent" (with message_id), "Webhook endpoint registered"
warnHandled problems / client errors.4xx responses, an unverified webhook rejected 401
errorUnhandled failures needing attention.5xx responses, SDK call threw
fatalProcess-ending conditions.Uncaught exception, unhandled rejection

Logging webhook processing (without secrets)

The receiver is where observability and security collide. Log enough to trace a delivery, and nothing that leaks.

// Safe: identifiers and outcomes only.
logger.info(
  {
    requestId,
    webhookId: req.headers['x-webhook-id'], // config UUID, not per-event
    eventType: req.body.event,              // e.g. "message.delivered"
    messageId: payload.message_id,
    status: payload.message_status,
  },
  'Webhook processed',
);

// NEVER: logger.info(req.headers)  → leaks X-Webhook-Signature + secret material
// NEVER: logger.debug(req.body)    → leaks recipient numbers + message content
logger.info("Webhook processed", extra={
    "request_id": request_id,
    "webhook_id": request.headers.get("x-webhook-id"),
    "event_type": payload["event"],
    "message_id": payload["payload"]["message_id"],
    "status": payload["payload"]["message_status"],
})
# Never log request.headers or the raw body.

X-Webhook-ID is the webhook configuration UUID, not a per-event ID. It's the same across every delivery from one endpoint. It's useful context, but derive your idempotency/dedupe key from the payload (message_id + message_status), not from this header. See Status tracking.

Metrics and tracing

Logs tell you what happened to one request; metrics tell you the shape of the system. The go-live checklist gates on these baseline metrics, so emit at least:

  • Counters: messages sent, webhooks received, webhooks rejected (bad signature / stale timestamp), send failures.
  • Histograms: SDK call latency, webhook processing duration, request duration (you already compute it in the request logger).
  • Gauges: if you offload webhook work to a queue, its depth.

Distributed tracing, unlike the baseline metrics, is optional: skip it until you're actually debugging a cross-service latency problem the preceding logs and metrics can't answer. At that point, instrument with OpenTelemetry, a vendor-neutral standard for recording timed operations (spans) and stitching them into one end-to-end trace per request, even across services. Both flows here are natural spans: wrap the messages.send SDK call in a span on the outbound path, and start a span in the receiver keyed by the correlation ID on the inbound path. Carry the same request/correlation ID as baggage (OpenTelemetry's term for a small piece of context attached to a trace and automatically propagated to every span within it), so a log line and a span can be correlated back to each other.

Keep instrumentation in middleware and the service layer, not in controllers. The layered architecture means one span around the service method covers every route that calls it.

Next steps

On this page