Voice Agents SDK for Python

sentdm-voice-agents lets a Python process serve calls as a Sent voice agent: it connects to Sent with your account API key, receives each call as a session, and gives your code the ways to act on it. Speech, heartbeats, reconnects and resumes are the SDK's; your code sees calls. The agent record itself, its tools, transfer targets and fallback, is set up in Voice agents; what the SDK exchanges with Sent is the voice agent protocol, which the SDK implements at version 0.2.0 and passes the protocol's conformance scenarios for before every release.

The protocol is in Preview and so is this SDK. Within sent.voice.v1 Sent only adds frames and fields, so an agent built today keeps working.

Installation

pip install sentdm-voice-agents

Python 3.11 or later. The one dependency is websockets.

Quick Start

Create the agent on Sent

Create a voice agent with POST /v3/voice/agents and bind a number to it, as Voice agents describes. Note its agent_id. Choose a text session, where Sent runs speech-to-text and text-to-speech, or an audio session, where your agent handles raw audio.

Serve calls

from sentdm_voice_agents import Agent, Session

class Support(Agent):
    async def on_session_start(self, session: Session) -> None:
        await session.say("Hi, this is Acme support. How can I help?")

    async def on_user_said(self, session: Session, said) -> None:
        if "person" in said.text:
            await session.say("Transferring you now.")
            await session.transfer("+14155550100")
        else:
            await session.say(f"You said {said.text}")

Support(api_key="sk_live_...", agent_id="agent_...").serve()

serve() connects to Sent and serves calls until SIGINT or SIGTERM; await agent.run() does the same inside your own event loop. Keep the API key on the server the agent runs on, never in a client.

Handlers

Override the ones you need. Each session's handlers run in order on their own task, so sessions never wait for one another, and a slow handler delays its own session only.

Handler	When
`on_session_start(session)`	A call reached the agent. In a text session, `say()` to speak; an audio session waits for audio
`on_user_said(session, said)`	Text sessions: the caller said something. `said.correction` is true when it replaces the previous utterance; send `clear()` and answer again
`on_audio(session, samples)`	Audio sessions: the caller's audio, 16-bit little-endian mono PCM at `session.sample_rate`
`on_tool_result(session, result)`	Sent answered a tool call. The awaiting call gets the same result
`on_session_end(session, end)`	The call left the agent; `end.reason` says why
`on_session_lost(session)`	The connection was lost and the session could not be resumed; Sent ends the call with `agent_unreachable`
`on_error(error)`	Sent refused a frame; the connection stays open

The session

Method	What it does
`say(text, last=True, interruptible=True)`	Text sessions: speak to the caller. Stream a reply in pieces with `last=False` and finish with `last=True`
`clear()`	Drop the speech or audio the agent sent that Sent has not played yet
`send_audio(samples)`	Audio sessions: play 16-bit mono PCM at the session's sample rate; 20 ms per call, at most 200 ms
`transfer(to)`	Move the call to a number, a user of your app, or another agent; returns Sent's `ToolResult`
`hangup()`	End the call, after the speech queued before it
`send_dtmf(digits)`	Play digits on the call: `0` to `9`, `*`, `#`, `A` to `D`, and `w` to wait half a second
`hold(held=True)`	Put the caller on hold, or take them off it

session.from_, to, direction, session_type, language, context (what your backend passed when it placed an outbound call) and sample_rate carry what the call started with. A tool refused or failed answers ok=False with an error.code such as tool_not_allowed or target_not_allowed; the agent's tools and transfer targets settings decide what it may do.

Resilience

The SDK answers Sent's heartbeats, opens a new connection when Sent drains one, reconnects with backoff when a connection is lost, and resumes the sessions that can move so the callers stay on the line. It keeps the last 32 frames it sent per session and sends again the ones Sent did not read. A session that cannot be resumed reaches on_session_lost. Nothing in your handlers needs to know.

Frames

sentdm_voice_agents.frames holds every frame of the protocol as a type, with decode() and encode() that check the protocol's rules. It is generated from the protocol and regenerated on every protocol release.

Other languages

The TypeScript SDK is the same agent on Node; .NET, Go, Rust and Java follow, each on the same protocol and the same conformance scenarios. Any language with a WebSocket client can build an agent on the protocol directly; the protocol repository's sample agents show how.

On this page