# Streaming Chat API Contract This document defines the server-sent events (SSE) contract for chat completions and Quick Questions, and the WebSocket contract for the persistent Agent room. Endpoint: - `POST /v1/chat-completions/stream` - `POST /v1/chats/:chatId/stream/attach` - `POST /v1/quick-questions/stream` Transport: - HTTP response uses `Content-Type: text/event-stream; charset=utf-8` - Events are emitted in SSE format (`event: ...`, `data: ...`) - Request body is JSON - Chat completion request bodies support the same inline attachment schema and limits documented in `docs/api/rest.md`. Quick Questions accept text only. Authentication: - Same as REST endpoints (`Authorization: Bearer ` when token mode is enabled) ## Chat Completion Request Body ```json { "chatId": "optional-chat-id", "persist": true, "clientRequestId": "optional-client-generated-id", "provider": "openai|anthropic|xai|gemini|hermes-agent", "model": "string", "messages": [ { "role": "system|user|assistant|tool", "content": "string", "name": "optional", "attachments": [ { "kind": "image", "id": "attachment-id", "filename": "photo.jpg", "mimeType": "image/jpeg", "sizeBytes": 12345, "dataUrl": "data:image/jpeg;base64,..." }, { "kind": "text", "id": "attachment-id", "filename": "notes.md", "mimeType": "text/markdown", "sizeBytes": 4567, "text": "# Notes\\n...", "truncated": false } ] } ], "additionalSystemPrompt": "optional one-off system prompt", "enabledTools": ["web_search", "fetch_url"], "temperature": 0.2, "maxTokens": 256 } ``` Notes: - `persist` defaults to `true`. - If `persist` is `true` and `chatId` is omitted, backend creates a new chat. - If `chatId` is provided, backend validates it exists. - If `persist` is `false`, `chatId` must be omitted. Backend does not create a chat and does not persist input messages, tool-call messages, assistant output, or `LlmCall` metadata. - `clientRequestId` is optional and is only valid for a persisted stream with a `chatId`. Clients should generate one stable, unique value per user submission and reuse it when retrying a disconnected request. - A retry with the same `chatId` and `clientRequestId` attaches to and replays the matching active stream. If that submission already completed, the endpoint replays `meta` and `done` without invoking the provider again. This makes retrying the initial streaming `POST` idempotent. - `clientRequestId` values may be up to 128 characters. The server stores the value in `metadata.clientRequestId` on the submitted user message and completed assistant message. - For persisted streams, backend stores only new non-assistant input history rows to avoid duplicates. - `additionalSystemPrompt`, when present directly or loaded from stored chat settings, is prepended to the provider request as a `system` message and is not inserted into the persisted chat transcript by this endpoint. - `enabledTools` limits Sybil-managed tools for this request. When omitted for a saved chat, the stored chat setting is used; otherwise all available tools are enabled by default. An empty array disables Sybil-managed tools. - `maxTokens` is optional. For `anthropic`, when omitted the backend requests the selected model's maximum output token limit from Anthropic's Models API and uses that as `max_tokens`; if the model limit cannot be loaded, the fallback is 128000. For other providers, omitted `maxTokens` is not sent as an explicit cap. - Attachments are optional and are persisted under `message.metadata.attachments` on stored user messages when `persist` is `true`. Persisted chat streams with a `chatId` are backend-owned active runs: - Once started, the backend keeps the stream running even if the HTTP client disconnects or refreshes. - The backend reserves the active run before persisting submitted messages, preventing a root or fork deletion from interleaving with stream setup. - While running, `GET /v1/active-runs` includes the `chatId`. - Starting a second persisted stream for the same active `chatId` returns `409`, unless its `clientRequestId` matches the active submission, in which case the existing stream is replayed. - Starting a persisted stream while that chat family is being deleted returns `409 { "message": "chat family deletion already in progress" }`. - Clients can reattach with `POST /v1/chats/:chatId/stream/attach`. ## Attach Endpoint `POST /v1/chats/:chatId/stream/attach` - Body: none. - Response uses the same `text/event-stream` transport and event names as `POST /v1/chat-completions/stream`. - Replays buffered events for the active in-memory stream, then emits new events until `done` or `error`. - Returns `404 { "message": "active chat stream not found" }` if no stream is currently active for that chat. - Authentication is the same as all other API endpoints. This endpoint is intended for clients that restored an active `chatId` from `GET /v1/active-runs`, especially after browser refresh. Replayed `delta` events may include text that was originally emitted before the client attached. ## Quick Question Endpoint `POST /v1/quick-questions/stream` Request body: ```json { "provider": "openai|anthropic|xai|gemini|hermes-agent", "model": "string", "question": "What is the capital of France?", "enabledTools": ["web_search", "fetch_url"], "userLocation": "optional city, region, country", "temperature": 0.2, "maxTokens": 256 } ``` Behavior notes: - `provider` and `model` are required. Web and native clients default Quick Question to `provider: "openai"` and `model: "chat-latest"` when no Quick Question selection has been saved. Saved Quick Question choices take precedence over this default and are separate from regular chat preferences. - `question` is required, trimmed by the server, and must not be empty. - The server prepends a Quick Question system prompt that asks for a succinct, direct, self-contained answer without follow-up questions. Clients do not send or maintain this prompt. - Quick Questions are always non-persistent. The endpoint does not create a chat or store messages, tool-call logs, assistant output, or `LlmCall` metadata. - The response uses the same `meta`, `tool_call`, `delta`, `done`, and `error` SSE events as chat completion streams. The `meta` event has `chatId: null` and `callId: null`. - `enabledTools`, `temperature`, and `maxTokens` are optional and behave as they do for chat completion streams. When `enabledTools` is omitted, all available Sybil-managed tools are enabled by default. - User location is inferred from the same request headers as chat completion streams when `userLocation` is omitted. ## Event Stream Contract Event order: 1. Exactly one `meta` 2. Zero or more `tool_call` 3. Zero or more `delta` 4. Exactly one terminal event: `done` or `error` Each tool invocation can emit multiple `tool_call` events with the same `toolCallId`. The backend emits `status: "initiated"` before the tool starts executing, then emits `status: "completed"` or `status: "failed"` when execution finishes. Clients should upsert by `toolCallId` instead of appending each event. ### `meta` ```json { "type": "meta", "chatId": "chat-id-or-null", "callId": "llm-call-id-or-null", "provider": "openai", "model": "gpt-4.1-mini" } ``` For `persist: false` streams, `chatId` and `callId` are `null`. ### `delta` ```json { "type": "delta", "text": "next chunk" } ``` `text` may contain partial words, punctuation, or whitespace. ### `tool_call` ```json { "toolCallId": "call_123", "name": "web_search", "status": "initiated", "summary": "Searching web for 'latest CPI release'.", "args": { "query": "latest CPI release" }, "startedAt": "2026-03-02T10:00:00.000Z" } ``` Terminal tool-call event: ```json { "toolCallId": "call_123", "name": "web_search", "status": "completed", "summary": "Performed web search for 'latest CPI release'.", "args": { "query": "latest CPI release" }, "startedAt": "2026-03-02T10:00:00.000Z", "completedAt": "2026-03-02T10:00:00.820Z", "durationMs": 820, "resultPreview": "{\"ok\":true,...}" } ``` `status` is one of `initiated`, `completed`, or `failed`. `completedAt` and `durationMs` are only present on terminal events. `error` is present on failed terminal events; `resultPreview` is present on terminal events when available. ### `done` ```json { "type": "done", "text": "full assistant response", "usage": { "inputTokens": 123, "outputTokens": 456, "totalTokens": 579 } } ``` `usage` may be omitted when provider does not expose final token accounting for stream mode. ### `error` ```json { "type": "error", "message": "provider timeout" } ``` ## Provider Streaming Behavior - `openai`: backend uses OpenAI's Responses API and may execute internal function tool calls (`web_search`, `fetch_url`, optional `codex_exec`, and optional `shell_exec`) before producing final text. - `anthropic`: backend uses Anthropic's Messages API and may execute the same internal tools with `tool_use`/`tool_result` content blocks before producing final text. - `xai`: backend uses xAI's OpenAI-compatible Chat Completions API and may execute the same internal tool calls before producing final text. - `gemini`: backend uses Google's native Gemini `streamGenerateContent` API and may execute the same internal tool calls before producing final text. - `fetch_url` sends browser-like navigation headers for outbound URL requests to reduce false 403s from sites that reject generic server clients. - `hermes-agent`: backend uses the configured Hermes Agent OpenAI-compatible Chat Completions API. Sybil does not add its own tool definitions for this provider; Hermes Agent handles its own tools server-side. Custom Hermes stream events are normalized away unless they produce text deltas in this SSE contract. - `openai`: image attachments are sent as Responses `input_image` items; text attachments are sent as `input_text` items. - `gemini`: image attachments are sent as native Gemini `inlineData` parts; text attachments are inlined as text parts. - `xai` and `hermes-agent`: image attachments are sent as Chat Completions content parts; text attachments are inlined as text parts. - `openai`: Responses calls that can enter the server-managed tool loop use `store: true` so reasoning and function-call items can be passed between tool rounds. - `anthropic`: streamed via event stream; emits `delta` from `content_block_delta` with `text_delta`, and emits normalized `tool_call` SSE events when Anthropic `tool_use` blocks are executed. Image attachments are sent as base64 `image` blocks and text attachments are appended as `text` blocks. - `web_search` uses `CHAT_WEB_SEARCH_ENGINE`: `exa` (default), `brave` (requires `BRAVE_SEARCH_API_KEY`), or `searxng` (requires `SEARXNG_BASE_URL`; the instance must allow `format=json`). This only affects chat-mode tool calls, not search-mode endpoints. - Brave searches are queued and evenly paced according to the shortest window in Brave's `X-RateLimit-Policy` response header. The backend also honors `X-RateLimit-Remaining`/`X-RateLimit-Reset` and retries `429` responses up to three times with reset-aware exponential backoff; quota resets beyond the bounded retry window fail immediately. - `codex_exec` is available only when `CHAT_CODEX_TOOL_ENABLED=true`. It SSHes to `CHAT_CODEX_REMOTE_HOST`, creates/uses `CHAT_CODEX_REMOTE_WORKDIR`, and runs `codex exec --dangerously-bypass-approvals-and-sandbox --skip-git-repo-check ` there with SSH stdin closed. Prefer `CHAT_CODEX_SSH_KEY_PATH` with a read-only mounted private key; `CHAT_CODEX_SSH_PRIVATE_KEY_B64` is also supported. - `shell_exec` is available only when `CHAT_SHELL_TOOL_ENABLED=true`. It uses the same devbox SSH configuration, starts in `CHAT_CODEX_REMOTE_WORKDIR`, and runs non-interactive shell commands there with SSH stdin closed, not inside the Sybil server container. - `CHAT_MAX_TOOL_ROUNDS` controls how many model/tool result cycles may occur before the backend returns a tool-call limit message; default is 100. Tool-enabled streaming notes (`openai`/`anthropic`/`xai`/`gemini`): - Stream still emits standard `meta`, `delta`, `done|error` events. - Stream may emit `tool_call` events while tool calls are executed. - `delta` events carry assistant text and are emitted incrementally for normal text rounds. The backend may buffer model-native text briefly while determining whether a provider round contains tool calls. - OpenAI Responses stream events are normalized by the backend into this SSE contract; clients do not consume OpenAI's raw Responses stream event names. ## Persistence + Consistency Model Backend database remains source of truth. For persisted streams: - Client may optimistically render accumulated `delta` text. - Backend emits initiated tool-call events without persisting them. - Backend persists each completed or failed tool call as a `tool` message before emitting its terminal `tool_call` SSE event, so chat detail refreshes can show completed tool calls while the assistant response is still running. On successful persisted completion: - Backend persists assistant `Message` and updates `LlmCall` usage/latency in a transaction. - Backend then emits `done`. On persisted failure: - Backend records call error and emits `error`. For `persist: false` streams: - Client may render the same `meta`, `tool_call`, `delta`, and terminal events. - Backend does not write any chat, message, tool-call log, assistant output, or call metadata rows. - `done.text` is the canonical assistant text if the client later imports the result into a saved chat. Client recommendation (for iOS/web): 1. Render deltas in real time for UX. 2. On `done`, refresh chat detail from REST (`GET /v1/chats/:chatId`) and use DB-backed data as canonical. 3. On `error`, preserve user input and show retry affordance. ## SSE Parsing Rules - Concatenate multiple `data:` lines with newline before JSON parse. - Event completes on blank line. - Ignore unknown event names for forward compatibility. ## Example Stream ```text event: meta data: {"type":"meta","chatId":"c1","callId":"k1","provider":"openai","model":"gpt-4.1-mini"} event: delta data: {"type":"delta","text":"Hello"} event: delta data: {"type":"delta","text":" world"} event: done data: {"type":"done","text":"Hello world"} ``` ## Sybil Agent WebSocket `GET /v1/agent/ws` upgrades to a continuous WebSocket room subscription. Behind the web proxy use `/api/v1/agent/ws`, with `ws:` or `wss:` matching the page. It remains connected after each assistant turn and receives later background results. Sending messages uses `POST /v1/agent/messages`. The browser must send this first frame within five seconds: ```json { "type": "auth", "token": "Sybil token", "after": "43" } ``` In open mode, use `token: null`. Omit `after` for recent history; on reconnect supply the last successfully merged cursor. Invalid credentials close with 4401, malformed initial frames with 4400. Malformed upgrade URLs and query parameters are rejected with HTTP 400; unknown upgrade paths return HTTP 404. Never put tokens in URLs. Sybil adds its private Sybil Agent credential only to the upstream Authorization header; browser credentials are not forwarded to the agent. Native `pi-room.v1` frames are forwarded after authentication. This protocol identifier is retained for compatibility with existing clients: ```json { "type": "ready", "protocol": "pi-room.v1", "conversationId": "1", "model": "..." } ``` ```json { "type": "messages", "conversationId": "1", "messages": [{ "id": "outbound:44", "role": "assistant", "content": "The result is ready.", "createdAt": "2026-10-14T18:00:00.000Z", "metadata": { "kind": "message", "entryId": 44, "status": "complete" } }], "cursor": "44", "beforeCursor": "44", "hasMore": false } ``` ```json { "type": "activity", "conversationId": "1", "activity": { "state": "thinking", "isThinking": true, "queued": 0, "backgroundWorkers": 0, "tools": [], "queuedMessages": [] } } ``` Activity is forwarded from Sybil Agent and also describes tools, queued messages, durable waiting, errors, and workers. Use `isThinking` for the thinking indicator and `backgroundWorkers` for ongoing delegated work; no model reasoning is exposed. Tools arrive as `role: "tool"` messages with `metadata.kind: "tool_call"`, matching Sybil's tool renderer. REST pages and socket messages are upserts keyed by stable `id`. Save each returned cursor after applying its entire frame, even when it contains no visible messages. Clients may send `{ "type": "ping" }` (receiving `{ "type": "pong" }`) or `{ "type": "subscribe", "after": "44" }` to change the replay cursor. Transport ping/pong heartbeats detect dead connections. Other writes belong on REST. Errors use `{ "type": "error", "error": "..." }`. Upstream loss closes with 1011 or 1012; slow consumers may receive 1013. Reconnect with backoff and the last merged cursor. Missing agent configuration closes with 4503. The proxy closes upstream sockets when browsers leave or Sybil shuts down. History remains durable in Sybil Agent. Shared read position is separate from delivery: update `GET`/`PUT /v1/agent/read-state` only when messages have actually been viewed.