Voice Agents SDK for TypeScript

@sentdm/voice-agents lets a Node process serve calls as a Sent voice agent: it connects to Sent with your account API key, receives each call as a session, and gives your code the ways to act on it. Speech, heartbeats, reconnects and resumes are the SDK's; your code sees calls. The agent record itself, its tools, transfer targets and fallback, is set up in Voice agents; what the SDK exchanges with Sent is the voice agent protocol, which the SDK implements at version 0.2.0 and passes the protocol's conformance scenarios for before every release.

The protocol is in Preview and so is this SDK. Within sent.voice.v1 Sent only adds frames and fields, so an agent built today keeps working.

Installation

npm install @sentdm/voice-agents

Node 20.19 or later. The one dependency is ws. The package is ES modules; require() works on the Node versions it supports.

Quick Start

Create the agent on Sent

Create a voice agent with POST /v3/voice/agents and bind a number to it, as Voice agents describes. Note its agent_id. Choose a text session, where Sent runs speech-to-text and text-to-speech, or an audio session, where your agent handles raw audio.

Serve calls

import { Agent, Session, UserSaid } from '@sentdm/voice-agents';

class Support extends Agent {
  override async onSessionStart(session: Session) {
    session.say('Hi, this is Acme support. How can I help?');
  }

  override async onUserSaid(session: Session, said: UserSaid) {
    if (said.text.includes('person')) {
      session.say('Transferring you now.');
      await session.transfer('+14155550100');
    } else {
      session.say(`You said ${said.text}`);
    }
  }
}

await new Support({ apiKey: process.env.SENT_API_KEY!, agentId: 'agent_...' }).serve();

serve() connects to Sent and serves calls until SIGINT or SIGTERM; run() does the same without the signal handlers and stop() ends it. Keep the API key on the server the agent runs on, never in a client.

Handlers

Override the ones you need. Each session's handlers run in order on their own chain, so sessions never wait for one another, and a slow handler delays its own session only.

Handler	When
`onSessionStart(session)`	A call reached the agent. In a text session, `say()` to speak; an audio session waits for audio
`onUserSaid(session, said)`	Text sessions: the caller said something. `said.correction` is true when it replaces the previous utterance; `clear()` and answer again
`onAudio(session, samples)`	Audio sessions: the caller's audio, a `Buffer` of 16-bit little-endian mono PCM at `session.sampleRate`
`onToolResult(session, result)`	Sent answered a tool call. The awaiting promise gets the same result
`onSessionEnd(session, end)`	The call left the agent; `end.reason` says why
`onSessionLost(session)`	The connection was lost and the session could not be resumed; Sent ends the call with `agent_unreachable`
`onError(error)`	Sent refused a frame; the connection stays open

The session

Method	What it does
`say(text, { last, interruptible })`	Text sessions: speak to the caller. Stream a reply in pieces with `last: false` and finish with `last: true`
`clear()`	Drop the speech or audio the agent sent that Sent has not played yet
`sendAudio(samples)`	Audio sessions: play 16-bit mono PCM at the session's sample rate; 20 ms per call, at most 200 ms
`transfer(to)`	Move the call to a number, a user of your app, or another agent; resolves with Sent's `ToolResult`
`hangup()`	End the call, after the speech queued before it
`sendDtmf(digits)`	Play digits on the call: `0` to `9`, `*`, `#`, `A` to `D`, and `w` to wait half a second
`hold(held = true)`	Put the caller on hold, or take them off it

session.from, to, direction, sessionType, language, context (what your backend passed when it placed an outbound call) and sampleRate carry what the call started with. A tool refused or failed resolves with ok: false and an error.code such as tool_not_allowed or target_not_allowed; the agent's tools and transfer targets settings decide what it may do.

Resilience

The SDK answers Sent's heartbeats, opens a new connection when Sent drains one, reconnects with backoff when a connection is lost, and resumes the sessions that can move so the callers stay on the line. It keeps the last 32 frames it sent per session and sends again the ones Sent did not read. A session that cannot be resumed reaches onSessionLost. Nothing in your handlers needs to know.

Frames

@sentdm/voice-agents/frames holds every frame of the protocol as a type, with decode() and encode() that check the protocol's rules. It is generated from the protocol and regenerated on every protocol release.

Other languages

The Python SDK is the same agent in Python; .NET, Go, Rust and Java follow, each on the same protocol and the same conformance scenarios. Any language with a WebSocket client can build an agent on the protocol directly.

On this page