Voice agent protocol (Preview)

An agent session is one phone call while one agent is on it. Sent and the agent exchange frames over a WebSocket: JSON text messages, one frame per message, and binary messages for audio. This page is the reference for those frames. How to get connected, either way round, is in Connect a voice agent.

Every frame has a type. Every frame about a session carries its session_id. Ids follow the REST API's: a session is vs_..., a call is call_..., the same id GET /v3/voice/calls/{id} reads, and an agent is agent_....

Frames

Order of frames

• Sent's first frame on a connection is connection.ready.
• For each session, Sent's first frame is session.start and its last is session.end. After session.end the agent sends nothing more for that session.
• On a connection a session is resumed on, Sent's first frame for it is session.resumed.
• ping, pong, connection.draining and error belong to the connection and can come at any time.
• Frames for one session arrive in the order they were sent, also across a resume. Frames for different sessions can interleave.

Limits

• user.said and agent.say carry at most 4,096 characters of text.
• A text frame is at most 64 KB. Sent refuses a larger one with invalid_frame.
• An outbound call's context is at most 4 KB as JSON.
• Each side keeps the last 32 frames it counted for a session, to send again when the session resumes. A session whose connection is lost has window_ms to resume, 15 seconds.

Connection frames

connection.ready

First on a connection the agent opened, and first on a connection Sent opened: the connection is authenticated, and sessions follow.

{ "type": "connection.ready", "protocol_version": "0.2.0", "heartbeat_interval_ms": 5000, "max_sessions": 50 }

connection.draining

No new sessions start on this connection. Open a new one now, while the sessions already on this one go on, and send a session.resume on it for each session, with the session's latest token and your received count. Sent closes the drained connection with code 1001 once no session is left on it. No fields.

{ "type": "connection.draining" }

ping and pong

Sent sends ping on every connection at heartbeat_interval_ms. Answer each one with pong carrying the same ts, milliseconds since the Unix epoch. Three unanswered pings close the connection with code 4008. Three intervals without a ping mean the connection is dead: open a new one and resume the sessions there.

{ "type": "ping", "ts": 1792411215000 }

{ "type": "pong", "ts": 1792411215000 }

error

A frame the agent sent wasn't accepted and had no effect. The connection stays open.

{
  "type": "error",
  "error": { "code": "frame_not_allowed", "message": "agent.say is not allowed in an audio session." },
  "session_id": "vs_7Hq2Lm9xQa",
  "frame_type": "agent.say"
}

Session frames

session.start

First for every session: a call has reached the agent.

{
  "type": "session.start",
  "session_id": "vs_7Hq2Lm9xQa",
  "call_id": "call_9f2ab000-0000-4000-8000-000000000001",
  "agent_id": "agent_9f2ab000-0000-4000-8000-000000000001",
  "direction": "inbound",
  "from": { "kind": "number", "value": "+14155550100" },
  "to": { "kind": "number", "value": "+14155550199" },
  "session_type": "text",
  "language": "en-US",
  "max_duration_ms": 3600000,
  "resume": { "token": "vsr_example_token_000000000000000001", "window_ms": 15000 },
  "started_at": "2026-10-23T12:00:00Z"
}

session.end

Last for every session: the call has left the agent.

{ "type": "session.end", "session_id": "vs_7Hq2Lm9xQa", "reason": "transferred", "duration_ms": 48210 }

End reasons

Text sessions

In a text session Sent runs speech-to-text and text-to-speech. When the agent has a greeting setting, Sent speaks it right after session.start. Otherwise the agent can speak first or wait for the caller. While interruptible is true, the default, the caller speaking stops the agent's speech and drops the speech still queued.

user.said

One complete utterance from the caller, once Sent has transcribed it. There are no interim partial results.

{ "type": "user.said", "session_id": "vs_7Hq2Lm9xQa", "text": "I need to talk to a person" }

agent.say

Speak this to the caller. Spoken in the order it arrives.

{ "type": "agent.say", "session_id": "vs_7Hq2Lm9xQa", "text": "Let me check your order", "last": false, "interruptible": false }

Audio sessions

In an audio session the agent runs its own speech. Sent sends the caller's audio and plays the audio the agent sends, as binary messages of 16-bit little-endian mono PCM at the sample_rate in session.start's audio. user.said and agent.say aren't allowed and answer frame_not_allowed.

Audio messages

Each binary message carries audio for one session:

The 2-byte header keeps the samples aligned, so a reader can view them as 16-bit integers without copying. Send 20 ms of audio per message, which is 640 bytes at 16,000 Hz, and no more than 200 ms. Sent sends the caller's audio for the whole session, silence included, and plays the agent's audio in the order it arrives. Audio a session has no room for is dropped, because late audio is worthless. The sample agents include one that plays the caller's audio back, the quickest way to hear a session work.

agent.clear

Drops the speech or audio the agent sent that Sent hasn't played yet, in either session type: when the caller starts talking over the agent, or after a correction. No fields beyond session_id.

{ "type": "agent.clear", "session_id": "vs_7Hq2Lm9xQa" }

Tools

The agent acts on the call with tool.call. Sent answers every tool.call with exactly one tool.result carrying the same call_ref. A tool runs after the speech or audio the agent queued before it, so an agent can say goodbye and then hang up. If the caller interrupts, the queued speech is dropped and the tool runs straight away. To act at once, send agent.clear before the tool.call.

The agent's tools setting decides which tools it may use, and its transfer_targets setting decides where it may transfer. See Tools and transfer targets.

tool.call

{
  "type": "tool.call",
  "session_id": "vs_7Hq2Lm9xQa",
  "call_ref": "t1",
  "name": "transfer",
  "arguments": { "to": { "kind": "number", "value": "+12125550100" } }
}

{ "type": "tool.call", "session_id": "vs_7Hq2Lm9xQa", "call_ref": "t2", "name": "send_dtmf", "arguments": { "digits": "1w2#" } }

tool.result

{ "type": "tool.result", "session_id": "vs_7Hq2Lm9xQa", "call_ref": "t1", "ok": true }

{
  "type": "tool.result",
  "session_id": "vs_7Hq2Lm9xQa",
  "call_ref": "t1",
  "ok": false,
  "error": { "code": "target_not_allowed", "message": "The agent may not transfer to this number." }
}

{
  "type": "tool.result",
  "session_id": "vs_7Hq2Lm9xQa",
  "call_ref": "t1",
  "ok": false,
  "error": { "code": "failed", "message": "The transfer did not connect." }
}

Tool error codes

Resume frames

A session whose session.start carries resume can move to a new connection after its connection is lost or while it drains. The flow, what happens while a session is detached, and which frames count are in Resume a session after a lost connection.

session.resume

Sent by the agent on a new connection, after connection.ready.

{ "type": "session.resume", "session_id": "vs_7Hq2Lm9xQa", "token": "vsr_example_token_000000000000000001", "received": 4 }

A refused session.resume answers error with connection_full or resume_failed.

session.resumed

The session is on this connection. Sent's frames the agent didn't read follow, in order.

{
  "type": "session.resumed",
  "session_id": "vs_7Hq2Lm9xQa",
  "received": 2,
  "resume": { "token": "vsr_example_token_000000000000000002", "window_ms": 15000 },
  "audio": { "stream": 3 }
}

Parties

from, to and a transfer's to are parties, in the shape of the REST API's call parties and of an agent's transfer_targets.

Sent may add kinds. Answer a kind you don't know as you would an anonymous caller.

Error codes

When Sent can't accept a frame, it sends error and ignores the frame. The connection stays open.

Close codes

A close with 1000 from the agent, or with 4003 from Sent, ends the connection's sessions. Any other close detaches them, and they wait window_ms for a resume.

A text session, worked

An inbound call to a number bound to a text agent that connects in. Sent's frames are marked Sent, the agent's agent. The connection was opened earlier and connection.ready already arrived.

// Sent: a call has reached the agent
{ "type": "session.start", "session_id": "vs_7Hq2Lm9xQa", "call_id": "call_9f2ab000-0000-4000-8000-000000000001", "agent_id": "agent_9f2ab000-0000-4000-8000-000000000001", "direction": "inbound", "from": { "kind": "number", "value": "+14155550100" }, "to": { "kind": "number", "value": "+14155550199" }, "session_type": "text", "language": "en-US", "max_duration_ms": 3600000, "resume": { "token": "vsr_example_token_000000000000000001", "window_ms": 15000 }, "started_at": "2026-10-23T12:00:00Z" }

// agent: the agent speaks first, because it has no greeting setting
{ "type": "agent.say", "session_id": "vs_7Hq2Lm9xQa", "text": "Hi, this is Acme support. How can I help?" }

// Sent: a heartbeat, which the agent answers
{ "type": "ping", "ts": 1792411215000 }
{ "type": "pong", "ts": 1792411215000 }

// Sent: the caller's first complete utterance
{ "type": "user.said", "session_id": "vs_7Hq2Lm9xQa", "text": "I need to talk to a person" }

// agent: a reply in two pieces, so speech starts before the reply is complete
{ "type": "agent.say", "session_id": "vs_7Hq2Lm9xQa", "text": "Of course.", "last": false }
{ "type": "agent.say", "session_id": "vs_7Hq2Lm9xQa", "text": "I'll put you through to the support line now." }

// agent: the transfer to a number in transfer_targets runs once the goodbye has played
{ "type": "tool.call", "session_id": "vs_7Hq2Lm9xQa", "call_ref": "t1", "name": "transfer", "arguments": { "to": { "kind": "number", "value": "+12125550100" } } }

// Sent: the number answered and the caller was bridged to it
{ "type": "tool.result", "session_id": "vs_7Hq2Lm9xQa", "call_ref": "t1", "ok": true }

// Sent: the call has left the agent
{ "type": "session.end", "session_id": "vs_7Hq2Lm9xQa", "reason": "transferred", "duration_ms": 48210 }

The call goes on between the caller and the support line, and ends as one call: call.completed with the call's whole duration, and GET /v3/voice/calls/{id} with mode agent and the agent's agent_id. Had the number not answered, the tool.result would have been ok: false with failed and the message The transfer did not connect., and the session would have gone on with the agent.

Compatibility and versions

Within sent.voice.v1, Sent only adds: fields, frame types, tools, party kinds, session.end reasons and error codes. So that an agent built against this page keeps working:

• Ignore fields and frame types you don't know.
• Treat a session.end reason you don't know as failed, and an error code you don't know as a failure.
• Answer a party kind you don't know as you would an anonymous caller.
• Count a frame type you don't know towards received when it carries your session's session_id.

connection.ready gives the protocol_version Sent speaks. Releases of the protocol are tags of the repository, such as v0.2.0. While the protocol is in Preview, a 0.x release can still change frames under sent.voice.v1. From 1.0, release N.x is sent.voice.vN: a change that would break an agent gets a new major release and a new subprotocol, and Sent serves both for a deprecation period.