Sybil-2/docs/api/streaming-chat.md

Streaming Chat API Contract

This document defines the server-sent events (SSE) contract for chat completions and Quick Questions, and the WebSocket contract for the persistent Agent room.

Endpoint:

• POST /v1/chat-completions/stream
• POST /v1/chats/:chatId/stream/attach
• POST /v1/quick-questions/stream

Transport:

• HTTP response uses Content-Type: text/event-stream; charset=utf-8
• Events are emitted in SSE format (event: ..., data: ...)
• Request body is JSON
• Chat completion request bodies support the same inline attachment schema and limits documented in docs/api/rest.md. Quick Questions accept text only.

Authentication:

• Same as REST endpoints (Authorization: Bearer <token> when token mode is enabled)
• The web client checks GET /v1/auth/session before opening streams. An initial missing-token response prompts sign-in without an error; an incorrect supplied token is an authentication failure. The web sign-in form has no tokenless bypass; open development servers still authenticate automatically. Stream authorization requirements are unchanged.

Chat Completion Request Body

{
  "chatId": "optional-chat-id",
  "persist": true,
  "clientRequestId": "optional-client-generated-id",
  "provider": "openai|anthropic|xai|gemini|hermes-agent",
  "model": "string",
  "messages": [
    {
      "role": "system|user|assistant|tool",
      "content": "string",
      "name": "optional",
      "attachments": [
        {
          "kind": "image",
          "id": "attachment-id",
          "filename": "photo.jpg",
          "mimeType": "image/jpeg",
          "sizeBytes": 12345,
          "dataUrl": "data:image/jpeg;base64,..."
        },
        {
          "kind": "text",
          "id": "attachment-id",
          "filename": "notes.md",
          "mimeType": "text/markdown",
          "sizeBytes": 4567,
          "text": "# Notes\\n...",
          "truncated": false
        }
      ]
    }
  ],
  "additionalSystemPrompt": "optional one-off system prompt",
  "enabledTools": ["web_search", "fetch_url"],
  "temperature": 0.2,
  "maxTokens": 256
}

Notes:

• persist defaults to true.
• If persist is true and chatId is omitted, backend creates a new chat.
• If chatId is provided, backend validates it exists.
• If persist is false, chatId must be omitted. Backend does not create a chat and does not persist input messages, tool-call messages, assistant output, or LlmCall metadata.
• clientRequestId is optional and is only valid for a persisted stream with a chatId. Clients should generate one stable, unique value per user submission and reuse it when retrying a disconnected request.
• A retry with the same chatId and clientRequestId attaches to and replays the matching active stream. If that submission already completed, the endpoint replays meta and done without invoking the provider again. This makes retrying the initial streaming POST idempotent.
• clientRequestId values may be up to 128 characters. The server stores the value in metadata.clientRequestId on the submitted user message and completed assistant message.
• For persisted streams, backend stores only new non-assistant input history rows to avoid duplicates.
• additionalSystemPrompt, when present directly or loaded from stored chat settings, is prepended to the provider request as a system message and is not inserted into the persisted chat transcript by this endpoint.
• enabledTools limits Sybil-managed tools for this request. When omitted for a saved chat, the stored chat setting is used; otherwise all available tools are enabled by default. An empty array disables Sybil-managed tools.
• maxTokens is optional. For anthropic, when omitted the backend requests the selected model's maximum output token limit from Anthropic's Models API and uses that as max_tokens; if the model limit cannot be loaded, the fallback is 128000. For other providers, omitted maxTokens is not sent as an explicit cap.
• Attachments are optional and are persisted under message.metadata.attachments on stored user messages when persist is true.

Persisted chat streams with a chatId are backend-owned active runs:

• Once started, the backend keeps the stream running even if the HTTP client disconnects or refreshes.
• The backend reserves the active run before persisting submitted messages, preventing a root or fork deletion from interleaving with stream setup.
• While running, GET /v1/active-runs includes the chatId.
• Starting a second persisted stream for the same active chatId returns 409, unless its clientRequestId matches the active submission, in which case the existing stream is replayed.
• Starting a persisted stream while that chat family is being deleted returns 409 { "message": "chat family deletion already in progress" }.
• Clients can reattach with POST /v1/chats/:chatId/stream/attach.

Attach Endpoint

POST /v1/chats/:chatId/stream/attach

• Body: none.
• Response uses the same text/event-stream transport and event names as POST /v1/chat-completions/stream.
• Replays buffered events for the active in-memory stream, then emits new events until done or error.
• Returns 404 { "message": "active chat stream not found" } if no stream is currently active for that chat.
• Authentication is the same as all other API endpoints.

This endpoint is intended for clients that restored an active chatId from GET /v1/active-runs, especially after browser refresh. Replayed delta events may include text that was originally emitted before the client attached.

Quick Question Endpoint

POST /v1/quick-questions/stream

Request body:

{
  "provider": "openai|anthropic|xai|gemini|hermes-agent",
  "model": "string",
  "question": "What is the capital of France?",
  "enabledTools": ["web_search", "fetch_url"],
  "userLocation": "optional city, region, country",
  "temperature": 0.2,
  "maxTokens": 256
}

Behavior notes:

• provider and model are required. Web and native clients default Quick Question to provider: "openai" and model: "chat-latest" when no Quick Question selection has been saved. Saved Quick Question choices take precedence over this default and are separate from regular chat preferences.
• question is required, trimmed by the server, and must not be empty.
• The server prepends a Quick Question system prompt that asks for a succinct, direct, self-contained answer without follow-up questions. Clients do not send or maintain this prompt.
• Quick Questions are always non-persistent. The endpoint does not create a chat or store messages, tool-call logs, assistant output, or LlmCall metadata.
• The response uses the same meta, tool_call, delta, done, and error SSE events as chat completion streams. The meta event has chatId: null and callId: null.
• enabledTools, temperature, and maxTokens are optional and behave as they do for chat completion streams. When enabledTools is omitted, all available Sybil-managed tools are enabled by default.
• User location is inferred from the same request headers as chat completion streams when userLocation is omitted.

Event Stream Contract

Event order:

1. Exactly one meta
2. Zero or more tool_call
3. Zero or more delta
4. Exactly one terminal event: done or error

Each tool invocation can emit multiple tool_call events with the same toolCallId. The backend emits status: "initiated" before the tool starts executing, then emits status: "completed" or status: "failed" when execution finishes. Clients should upsert by toolCallId instead of appending each event.

meta

{
  "type": "meta",
  "chatId": "chat-id-or-null",
  "callId": "llm-call-id-or-null",
  "provider": "openai",
  "model": "gpt-4.1-mini"
}

For persist: false streams, chatId and callId are null.

delta

{ "type": "delta", "text": "next chunk" }

text may contain partial words, punctuation, or whitespace.

tool_call

{
  "toolCallId": "call_123",
  "name": "web_search",
  "status": "initiated",
  "summary": "Searching web for 'latest CPI release'.",
  "args": { "query": "latest CPI release" },
  "startedAt": "2026-03-02T10:00:00.000Z"
}

Terminal tool-call event:

{
  "toolCallId": "call_123",
  "name": "web_search",
  "status": "completed",
  "summary": "Performed web search for 'latest CPI release'.",
  "args": { "query": "latest CPI release" },
  "startedAt": "2026-03-02T10:00:00.000Z",
  "completedAt": "2026-03-02T10:00:00.820Z",
  "durationMs": 820,
  "resultPreview": "{\"ok\":true,...}"
}

status is one of initiated, completed, or failed. completedAt and durationMs are only present on terminal events. error is present on failed terminal events; resultPreview is present on terminal events when available.

done

{
  "type": "done",
  "text": "full assistant response",
  "usage": {
    "inputTokens": 123,
    "outputTokens": 456,
    "totalTokens": 579
  }
}

usage may be omitted when provider does not expose final token accounting for stream mode.

error

{ "type": "error", "message": "provider timeout" }

Provider Streaming Behavior

• openai: backend uses OpenAI's Responses API and may execute internal function tool calls (web_search, fetch_url, optional codex_exec, and optional shell_exec) before producing final text.
• anthropic: backend uses Anthropic's Messages API and may execute the same internal tools with tool_use/tool_result content blocks before producing final text.
• xai: backend uses xAI's OpenAI-compatible Chat Completions API and may execute the same internal tool calls before producing final text.
• gemini: backend uses Google's native Gemini streamGenerateContent API and may execute the same internal tool calls before producing final text.
• fetch_url sends browser-like navigation headers for outbound URL requests to reduce false 403s from sites that reject generic server clients.
• hermes-agent: backend uses the configured Hermes Agent OpenAI-compatible Chat Completions API. Sybil does not add its own tool definitions for this provider; Hermes Agent handles its own tools server-side. Custom Hermes stream events are normalized away unless they produce text deltas in this SSE contract.
• openai: image attachments are sent as Responses input_image items; text attachments are sent as input_text items.
• gemini: image attachments are sent as native Gemini inlineData parts; text attachments are inlined as text parts.
• xai and hermes-agent: image attachments are sent as Chat Completions content parts; text attachments are inlined as text parts.
• openai: Responses calls that can enter the server-managed tool loop use store: true so reasoning and function-call items can be passed between tool rounds.
• anthropic: streamed via event stream; emits delta from content_block_delta with text_delta, and emits normalized tool_call SSE events when Anthropic tool_use blocks are executed. Image attachments are sent as base64 image blocks and text attachments are appended as text blocks.
• web_search uses CHAT_WEB_SEARCH_ENGINE: exa (default), brave (requires BRAVE_SEARCH_API_KEY), or searxng (requires SEARXNG_BASE_URL; the instance must allow format=json). This only affects chat-mode tool calls, not search-mode endpoints.
• Brave searches are queued and evenly paced according to the shortest window in Brave's X-RateLimit-Policy response header. The backend also honors X-RateLimit-Remaining/X-RateLimit-Reset and retries 429 responses up to three times with reset-aware exponential backoff; quota resets beyond the bounded retry window fail immediately.
• codex_exec is available only when CHAT_CODEX_TOOL_ENABLED=true. It SSHes to CHAT_CODEX_REMOTE_HOST, creates/uses CHAT_CODEX_REMOTE_WORKDIR, and runs codex exec --dangerously-bypass-approvals-and-sandbox --skip-git-repo-check <non-interactive wrapped prompt> there with SSH stdin closed. Prefer CHAT_CODEX_SSH_KEY_PATH with a read-only mounted private key; CHAT_CODEX_SSH_PRIVATE_KEY_B64 is also supported.
• shell_exec is available only when CHAT_SHELL_TOOL_ENABLED=true. It uses the same devbox SSH configuration, starts in CHAT_CODEX_REMOTE_WORKDIR, and runs non-interactive shell commands there with SSH stdin closed, not inside the Sybil server container.
• CHAT_MAX_TOOL_ROUNDS controls how many model/tool result cycles may occur before the backend returns a tool-call limit message; default is 100.

Tool-enabled streaming notes (openai/anthropic/xai/gemini):

• Stream still emits standard meta, delta, done|error events.
• Stream may emit tool_call events while tool calls are executed.
• delta events carry assistant text and are emitted incrementally for normal text rounds. The backend may buffer model-native text briefly while determining whether a provider round contains tool calls.
• OpenAI Responses stream events are normalized by the backend into this SSE contract; clients do not consume OpenAI's raw Responses stream event names.

Persistence + Consistency Model

Backend database remains source of truth.

For persisted streams:

• Client may optimistically render accumulated delta text.
• Backend emits initiated tool-call events without persisting them.
• Backend persists each completed or failed tool call as a tool message before emitting its terminal tool_call SSE event, so chat detail refreshes can show completed tool calls while the assistant response is still running.

On successful persisted completion:

• Backend persists assistant Message and updates LlmCall usage/latency in a transaction.
• Backend then emits done.

On persisted failure:

• Backend records call error and emits error.

For persist: false streams:

• Client may render the same meta, tool_call, delta, and terminal events.
• Backend does not write any chat, message, tool-call log, assistant output, or call metadata rows.
• done.text is the canonical assistant text if the client later imports the result into a saved chat.

Client recommendation (for iOS/web):

1. Render deltas in real time for UX.
2. On done, refresh chat detail from REST (GET /v1/chats/:chatId) and use DB-backed data as canonical.
3. On error, preserve user input and show retry affordance.

SSE Parsing Rules

• Concatenate multiple data: lines with newline before JSON parse.
• Event completes on blank line.
• Ignore unknown event names for forward compatibility.

Example Stream

event: meta
data: {"type":"meta","chatId":"c1","callId":"k1","provider":"openai","model":"gpt-4.1-mini"}

event: delta
data: {"type":"delta","text":"Hello"}

event: delta
data: {"type":"delta","text":" world"}

event: done
data: {"type":"done","text":"Hello world"}

Sybil Agent WebSocket

GET /v1/agent/ws upgrades to a continuous WebSocket room subscription. Behind the web proxy use /api/v1/agent/ws, with ws: or wss: matching the page. It remains connected after each assistant turn and receives later background results. Sending messages uses POST /v1/agent/messages.

The browser must send this first frame within five seconds:

{ "type": "auth", "token": "Sybil token", "after": "43" }

In open mode, use token: null. Omit after for recent history; on reconnect supply the last successfully merged cursor. Invalid credentials close with 4401, malformed initial frames with 4400. Malformed upgrade URLs and query parameters are rejected with HTTP 400; unknown upgrade paths return HTTP 404. Never put tokens in URLs. Sybil adds its private Sybil Agent credential only to the upstream Authorization header; browser credentials are not forwarded to the agent.

Native pi-room.v1 frames are forwarded after authentication. This protocol identifier is retained for compatibility with existing clients:

{ "type": "ready", "protocol": "pi-room.v1", "conversationId": "1", "model": "..." }

{
  "type": "messages",
  "conversationId": "1",
  "messages": [{ "id": "outbound:44", "role": "assistant", "content": "The result is ready.", "createdAt": "2026-10-14T18:00:00.000Z", "metadata": { "kind": "message", "entryId": 44, "status": "complete" } }],
  "cursor": "44",
  "beforeCursor": "44",
  "hasMore": false
}

{ "type": "activity", "conversationId": "1", "activity": { "state": "thinking", "isThinking": true, "queued": 0, "backgroundWorkers": 0, "tools": [], "queuedMessages": [] } }

Activity is forwarded from Sybil Agent and also describes tools, queued messages, durable waiting, errors, and workers. Use isThinking for the thinking indicator and backgroundWorkers for ongoing delegated work; no model reasoning is exposed. Tools arrive as role: "tool" messages with metadata.kind: "tool_call", matching Sybil's tool renderer. REST pages and socket messages are upserts keyed by stable id. Save each returned cursor after applying its entire frame, even when it contains no visible messages.

Clients may send { "type": "ping" } (receiving { "type": "pong" }) or { "type": "subscribe", "after": "44" } to change the replay cursor. Transport ping/pong heartbeats detect dead connections. Other writes belong on REST. Errors use { "type": "error", "error": "..." }. Upstream loss closes with 1011 or 1012; slow consumers may receive 1013. Reconnect with backoff and the last merged cursor. Missing agent configuration closes with 4503.

The proxy closes upstream sockets when browsers leave or Sybil shuts down. History remains durable in Sybil Agent. Shared read position is separate from delivery: update GET/PUT /v1/agent/read-state only when messages have actually been viewed.

Agent attachment history, queued activity and replay

Uploads and attached sends use REST, as specified in rest.md. WebSocket authentication, event types, stable message IDs, cursor semantics and raw tool rows are unchanged. Capability discovery is optional GET /v1/agent → capabilities.attachments.version: 1; absence disables new attachment sends. Upgrade/authenticate the native runtime and backend before advertising it.

Both messages.messages[] and activity.activity.queuedMessages[] carry attachment descriptors under metadata.agentAttachments, distinct from ordinary chat's inline metadata.attachments:

{
  "id": "inbound:sybil:client-stable-id",
  "role": "user",
  "content": "Read these notes",
  "createdAt": "2026-10-14T18:00:00.000Z",
  "metadata": {
    "kind": "message",
    "channel": "sybil",
    "status": "queued",
    "agentAttachments": [{
      "id": "att_12345678-1234-4234-8234-123456789abc",
      "kind": "text",
      "filename": "notes.txt",
      "mimeType": "text/plain",
      "sizeBytes": 12,
      "sha256": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
      "workspacePath": "/data/workspace/uploads/att_12345678-1234-4234-8234-123456789abc/notes.txt"
    }]
  }
}

workspacePath is optional, authoritative runtime text for display/copy, never a download URL or send parameter. Only attachments:[{id}] goes into POST messages. Committed/replayed messages retain the same ordered descriptors and original caption; file-only content is Attached: notes.txt.. Internal model manifests/previews and native base64 are excluded from public user message content. Root/worker native inputs and raw authorized model endpoints may contain the real image blocks; native token auth must protect all of them.

Merge queued and committed messages by stable message ID. A matching ID alone does not confirm an attachment-bearing optimistic message: compare the ordered metadata.agentAttachments[].id against its local IDs as well. Retain descriptors, error and retry identity on a mismatch or uncertain receipt; show the error inline for touch clients. Matching canonical echoes can acknowledge delivery even if HTTP was lost. Preserve raw tool rows/cursors in client room state and existing transcript filtering. Old clients can ignore the additive metadata and still see a meaningful caption.

Retrieve bytes using authenticated GET /v1/agent/attachments/:id/content; fetch with an Authorization header and use/revoke browser blob URLs for previews. Never add credentials to URLs or use a public file route. Persist completed descriptors and pending upload IDs with the composer/outbox; navigation cannot silently turn an attached draft into a text-only send. Incomplete reloaded uploads require verified file reselection or explicit removal, per the REST contract.

Attached input admission and its request binding are atomic. Replay verifies the durable accepted payload, including the ordered attachment IDs, before acknowledging it. Queued attachment metadata survives restart even if placement did not run before the crash; normal queue recovery places the original input without duplicate charge or delivery. Worker preparation saturation retries through durable Reporter checkpoints; permanent admission failures appear through the existing worker-report path.