El socket se abre en /realtime?session=rts_… con la sesión que devuelve POST /agents/v1/realtime/sessions; la clave secreta nunca viaja por él. Cada mensaje es un objeto JSON con type.
El cliente envía
Turns (text or audio), instruction changes and the end of the conversation.
session.update
Change the agent’s instructions mid-conversation.
{
"type": "object",
"required": [
"type"
],
"additionalProperties": false,
"properties": {
"type": {
"const": "session.update"
},
"instructions": {
"type": "string",
"maxLength": 500
}
}
}input_text.send
A complete user turn, as text.
{
"type": "object",
"required": [
"type",
"text"
],
"additionalProperties": false,
"properties": {
"type": {
"const": "input_text.send"
},
"text": {
"type": "string",
"minLength": 1,
"maxLength": 4000
}
}
}input_audio.append
A chunk of the user’s audio.
{
"type": "object",
"required": [
"type",
"audio"
],
"additionalProperties": false,
"properties": {
"type": {
"const": "input_audio.append"
},
"audio": {
"type": "string",
"contentEncoding": "base64",
"description": "PCM 16-bit little-endian, 16 kHz, mono."
}
}
}input_audio.commit
The user stopped talking; the agent answers what it heard.
{
"type": "object",
"required": [
"type"
],
"additionalProperties": false,
"properties": {
"type": {
"const": "input_audio.commit"
}
}
}session.end
Hang up.
{
"type": "object",
"required": [
"type"
],
"additionalProperties": false,
"properties": {
"type": {
"const": "session.end"
}
}
}El servidor envía
The session start, the agent’s answer in deltas, running usage, errors and the end.
session.started
{
"type": "object",
"required": [
"type",
"session"
],
"properties": {
"type": {
"const": "session.started"
},
"session": {
"type": "object",
"required": [
"id",
"workspace",
"cell",
"livemode",
"input_audio_format",
"max_duration_seconds"
],
"properties": {
"id": {
"type": "string"
},
"workspace": {
"type": "string"
},
"cell": {
"type": "string"
},
"livemode": {
"type": "boolean"
},
"input_audio_format": {
"type": "string",
"const": "pcm16_16khz_mono"
},
"max_duration_seconds": {
"type": "integer"
},
"served_by": {
"type": "string",
"description": "The service and language that answered (agents-runtime/python)."
}
}
}
}
}response.text.delta
The next piece of the agent’s answer.
{
"type": "object",
"required": [
"type",
"response_id",
"delta"
],
"properties": {
"type": {
"const": "response.text.delta"
},
"response_id": {
"type": "string"
},
"delta": {
"type": "string"
}
}
}response.done
{
"type": "object",
"required": [
"type",
"response_id",
"text"
],
"properties": {
"type": {
"const": "response.done"
},
"response_id": {
"type": "string"
},
"text": {
"type": "string"
}
}
}usage.updated
Running totals, after every turn.
{
"type": "object",
"required": [
"type",
"elapsed_seconds",
"audio_seconds",
"turns"
],
"properties": {
"type": {
"const": "usage.updated"
},
"elapsed_seconds": {
"type": "number"
},
"audio_seconds": {
"type": "number"
},
"turns": {
"type": "integer"
}
}
}session.ended
{
"type": "object",
"required": [
"type",
"reason",
"billed_minutes"
],
"properties": {
"type": {
"const": "session.ended"
},
"reason": {
"type": "string",
"enum": [
"client_ended",
"max_duration",
"client_disconnected"
]
},
"billed_minutes": {
"type": "integer",
"description": "0 in test mode."
},
"usage_record_id": {
"type": [
"string",
"null"
],
"description": "The billing record (contracts/internal/proto/qomni/billing/v1), when billed."
}
}
}error
{
"type": "object",
"required": [
"type",
"error"
],
"properties": {
"type": {
"const": "error"
},
"error": {
"type": "object",
"required": [
"code",
"message"
],
"properties": {
"code": {
"type": "string",
"description": "invalid_message, unknown_message_type, session_expired…"
},
"message": {
"type": "string"
}
}
}
}
}