Sybil-2/docs/api/streaming-chat.md

Streaming Chat API Contract

This document defines the server-sent events (SSE) contract for chat completions and Quick Questions, and the WebSocket contract for the persistent Agent room.

Endpoint:

• POST /v1/chat-completions/stream
• POST /v1/chats/:chatId/stream/attach
• POST /v1/quick-questions/stream

Transport:

• HTTP response uses Content-Type: text/event-stream; charset=utf-8
• Events are emitted in SSE format (event: ..., data: ...)
• Request body is JSON
• Chat completion request bodies support the same inline attachment schema and limits documented in docs/api/rest.md. Quick Questions accept text only.

Authentication:

• Same as REST endpoints (Authorization: Bearer <token> when token mode is enabled)
• The web client checks GET /v1/auth/session before opening streams. An initial missing-token response prompts sign-in without an error; an incorrect supplied token is an authentication failure. The web sign-in form has no tokenless bypass; open development servers still authenticate automatically. Stream authorization requirements are unchanged.

Chat Completion Request Body

{
  "chatId": "optional-chat-id",
  "persist": true,
  "clientRequestId": "optional-client-generated-id",
  "provider": "openai|anthropic|xai|gemini|hermes-agent",
  "model": "string",
  "messages": [
    {
      "role": "system|user|assistant|tool",
      "content": "string",
      "name": "optional",
      "attachments": [
        {
          "kind": "image",
          "id": "attachment-id",
          "filename": "photo.jpg",
          "mimeType": "image/jpeg",
          "sizeBytes": 12345,
          "dataUrl": "data:image/jpeg;base64,..."
        },
        {
          "kind": "text",
          "id": "attachment-id",
          "filename": "notes.md",
          "mimeType": "text/markdown",
          "sizeBytes": 4567,
          "text": "# Notes\\n...",
          "truncated": false
        }
      ]
    }
  ],
  "additionalSystemPrompt": "optional one-off system prompt",
  "enabledTools": ["web_search", "fetch_url"],
  "temperature": 0.2,
  "maxTokens": 256
}

Notes:

• persist defaults to true.
• If persist is true and chatId is omitted, backend creates a new chat.
• If chatId is provided, backend validates it exists.
• If persist is false, chatId must be omitted. Backend does not create a chat and does not persist input messages, tool-call messages, assistant output, or LlmCall metadata.
• clientRequestId is optional and is only valid for a persisted stream with a chatId. Clients should generate one stable, unique value per user submission and reuse it when retrying a disconnected request.
• A retry with the same chatId and clientRequestId attaches to and replays the matching active stream. If that submission already completed, the endpoint replays meta and done without invoking the provider again. This makes retrying the initial streaming POST idempotent.
• clientRequestId values may be up to 128 characters. The server stores the value in metadata.clientRequestId on the submitted user message and completed assistant message.
• For persisted streams, backend stores only new non-assistant input history rows to avoid duplicates.
• additionalSystemPrompt, when present directly or loaded from stored chat settings, is prepended to the provider request as a system message and is not inserted into the persisted chat transcript by this endpoint.
• enabledTools limits Sybil-managed tools for this request. When omitted for a saved chat, the stored chat setting is used; otherwise all available tools are enabled by default. An empty array disables Sybil-managed tools.
• maxTokens is optional. For anthropic, when omitted the backend requests the selected model's maximum output token limit from Anthropic's Models API and uses that as max_tokens; if the model limit cannot be loaded, the fallback is 128000. For other providers, omitted maxTokens is not sent as an explicit cap.
• Attachments are optional and are persisted under message.metadata.attachments on stored user messages when persist is true.

Persisted chat streams with a chatId are backend-owned active runs:

• Once started, the backend keeps the stream running even if the HTTP client disconnects or refreshes.
• The backend reserves the active run before persisting submitted messages, preventing a root or fork deletion from interleaving with stream setup.
• While running, GET /v1/active-runs includes the chatId.
• Starting a second persisted stream for the same active chatId returns 409, unless its clientRequestId matches the active submission, in which case the existing stream is replayed.
• Starting a persisted stream while that chat family is being deleted returns 409 { "message": "chat family deletion already in progress" }.
• Clients can reattach with POST /v1/chats/:chatId/stream/attach.

Attach Endpoint

POST /v1/chats/:chatId/stream/attach

• Body: none.
• Response uses the same text/event-stream transport and event names as POST /v1/chat-completions/stream.
• Replays buffered events for the active in-memory stream, then emits new events until done or error.
• Returns 404 { "message": "active chat stream not found" } if no stream is currently active for that chat.
• Authentication is the same as all other API endpoints.

This endpoint is intended for clients that restored an active chatId from GET /v1/active-runs, especially after browser refresh. Replayed delta events may include text that was originally emitted before the client attached.

Quick Question Endpoint

POST /v1/quick-questions/stream

Request body:

{
  "provider": "openai|anthropic|xai|gemini|hermes-agent",
  "model": "string",
  "question": "What is the capital of France?",
  "enabledTools": ["web_search", "fetch_url"],
  "userLocation": "optional city, region, country",
  "temperature": 0.2,
  "maxTokens": 256
}

Behavior notes:

• provider and model are required. Web and native clients default Quick Question to provider: "openai" and model: "chat-latest" when no Quick Question selection has been saved. Saved Quick Question choices take precedence over this default and are separate from regular chat preferences.
• question is required, trimmed by the server, and must not be empty.
• The server prepends a Quick Question system prompt that asks for a succinct, direct, self-contained answer without follow-up questions. Clients do not send or maintain this prompt.
• Quick Questions are always non-persistent. The endpoint does not create a chat or store messages, tool-call logs, assistant output, or LlmCall metadata.
• The response uses the same meta, tool_call, delta, done, and error SSE events as chat completion streams. The meta event has chatId: null and callId: null.
• enabledTools, temperature, and maxTokens are optional and behave as they do for chat completion streams. When enabledTools is omitted, all available Sybil-managed tools are enabled by default.
• User location is inferred from the same request headers as chat completion streams when userLocation is omitted.

Event Stream Contract

Event order:

1. Exactly one meta
2. Zero or more tool_call
3. Zero or more delta
4. Exactly one terminal event: done or error

Each tool invocation can emit multiple tool_call events with the same toolCallId. The backend emits status: "initiated" before the tool starts executing, then emits status: "completed" or status: "failed" when execution finishes. Clients should upsert by toolCallId instead of appending each event.

meta

{
  "type": "meta",
  "chatId": "chat-id-or-null",
  "callId": "llm-call-id-or-null",
  "provider": "openai",
  "model": "gpt-4.1-mini"
}

For persist: false streams, chatId and callId are null.

delta

{ "type": "delta", "text": "next chunk" }

text may contain partial words, punctuation, or whitespace.

tool_call

{
  "toolCallId": "call_123",
  "name": "web_search",
  "status": "initiated",
  "summary": "Searching web for 'latest CPI release'.",
  "args": { "query": "latest CPI release" },
  "startedAt": "2026-03-02T10:00:00.000Z"
}

Terminal tool-call event:

{
  "toolCallId": "call_123",
  "name": "web_search",
  "status": "completed",
  "summary": "Performed web search for 'latest CPI release'.",
  "args": { "query": "latest CPI release" },
  "startedAt": "2026-03-02T10:00:00.000Z",
  "completedAt": "2026-03-02T10:00:00.820Z",
  "durationMs": 820,
  "resultPreview": "{\"ok\":true,...}"
}

status is one of initiated, completed, or failed. completedAt and durationMs are only present on terminal events. error is present on failed terminal events; resultPreview is present on terminal events when available.

done

{
  "type": "done",
  "text": "full assistant response",
  "usage": {
    "inputTokens": 123,
    "outputTokens": 456,
    "totalTokens": 579
  }
}

usage may be omitted when provider does not expose final token accounting for stream mode.

error

{ "type": "error", "message": "provider timeout" }

Provider Streaming Behavior

• openai: backend uses OpenAI's Responses API and may execute internal function tool calls (web_search, fetch_url, optional codex_exec, and optional shell_exec) before producing final text.
• anthropic: backend uses Anthropic's Messages API and may execute the same internal tools with tool_use/tool_result content blocks before producing final text.
• xai: backend uses xAI's OpenAI-compatible Chat Completions API and may execute the same internal tool calls before producing final text.
• gemini: backend uses Google's native Gemini streamGenerateContent API and may execute the same internal tool calls before producing final text.
• fetch_url sends browser-like navigation headers for outbound URL requests to reduce false 403s from sites that reject generic server clients.
• hermes-agent: backend uses the configured Hermes Agent OpenAI-compatible Chat Completions API. Sybil does not add its own tool definitions for this provider; Hermes Agent handles its own tools server-side. Custom Hermes stream events are normalized away unless they produce text deltas in this SSE contract.
• openai: image attachments are sent as Responses input_image items; text attachments are sent as input_text items.
• gemini: image attachments are sent as native Gemini inlineData parts; text attachments are inlined as text parts.
• xai and hermes-agent: image attachments are sent as Chat Completions content parts; text attachments are inlined as text parts.
• openai: Responses calls that can enter the server-managed tool loop use store: true so reasoning and function-call items can be passed between tool rounds.
• anthropic: streamed via event stream; emits delta from content_block_delta with text_delta, and emits normalized tool_call SSE events when Anthropic tool_use blocks are executed. Image attachments are sent as base64 image blocks and text attachments are appended as text blocks.
• web_search uses CHAT_WEB_SEARCH_ENGINE: exa (default), brave (requires BRAVE_SEARCH_API_KEY), or searxng (requires SEARXNG_BASE_URL; the instance must allow format=json). This only affects chat-mode tool calls, not search-mode endpoints.
• Brave searches are queued and evenly paced according to the shortest window in Brave's X-RateLimit-Policy response header. The backend also honors X-RateLimit-Remaining/X-RateLimit-Reset and retries 429 responses up to three times with reset-aware exponential backoff; quota resets beyond the bounded retry window fail immediately.
• codex_exec is available only when CHAT_CODEX_TOOL_ENABLED=true. It SSHes to CHAT_CODEX_REMOTE_HOST, creates/uses CHAT_CODEX_REMOTE_WORKDIR, and runs codex exec --dangerously-bypass-approvals-and-sandbox --skip-git-repo-check <non-interactive wrapped prompt> there with SSH stdin closed. Prefer CHAT_CODEX_SSH_KEY_PATH with a read-only mounted private key; CHAT_CODEX_SSH_PRIVATE_KEY_B64 is also supported.
• shell_exec is available only when CHAT_SHELL_TOOL_ENABLED=true. It uses the same devbox SSH configuration, starts in CHAT_CODEX_REMOTE_WORKDIR, and runs non-interactive shell commands there with SSH stdin closed, not inside the Sybil server container.
• CHAT_MAX_TOOL_ROUNDS controls how many model/tool result cycles may occur before the backend returns a tool-call limit message; default is 100.

Tool-enabled streaming notes (openai/anthropic/xai/gemini):

• Stream still emits standard meta, delta, done|error events.
• Stream may emit tool_call events while tool calls are executed.
• delta events carry assistant text and are emitted incrementally for normal text rounds. The backend may buffer model-native text briefly while determining whether a provider round contains tool calls.
• OpenAI Responses stream events are normalized by the backend into this SSE contract; clients do not consume OpenAI's raw Responses stream event names.

Persistence + Consistency Model

Backend database remains source of truth.

For persisted streams:

• Client may optimistically render accumulated delta text.
• Backend emits initiated tool-call events without persisting them.
• Backend persists each completed or failed tool call as a tool message before emitting its terminal tool_call SSE event, so chat detail refreshes can show completed tool calls while the assistant response is still running.

On successful persisted completion:

• Backend persists assistant Message and updates LlmCall usage/latency in a transaction.
• Backend then emits done.

On persisted failure:

• Backend records call error and emits error.

For persist: false streams:

• Client may render the same meta, tool_call, delta, and terminal events.
• Backend does not write any chat, message, tool-call log, assistant output, or call metadata rows.
• done.text is the canonical assistant text if the client later imports the result into a saved chat.

Client recommendation (for iOS/web):

1. Render deltas in real time for UX.
2. On done, refresh chat detail from REST (GET /v1/chats/:chatId) and use DB-backed data as canonical.
3. On error, preserve user input and show retry affordance.

SSE Parsing Rules

• Concatenate multiple data: lines with newline before JSON parse.
• Event completes on blank line.
• Ignore unknown event names for forward compatibility.

Example Stream

event: meta
data: {"type":"meta","chatId":"c1","callId":"k1","provider":"openai","model":"gpt-4.1-mini"}

event: delta
data: {"type":"delta","text":"Hello"}

event: delta
data: {"type":"delta","text":" world"}

event: done
data: {"type":"done","text":"Hello world"}

Sybil Agent WebSocket

GET /v1/agent/ws upgrades to a continuous WebSocket room subscription. Behind the web proxy use /api/v1/agent/ws, with ws: or wss: matching the page. It remains connected after each assistant turn and receives later background results. Sending messages uses POST /v1/agent/messages.

The browser must send this first frame within five seconds:

{ "type": "auth", "token": "Sybil token", "after": "43" }

In open mode, use token: null. Omit after for recent history; on reconnect supply the last successfully merged cursor. Invalid credentials close with 4401, malformed initial frames with 4400. Malformed upgrade URLs and query parameters are rejected with HTTP 400; unknown upgrade paths return HTTP 404. Never put tokens in URLs. Sybil adds its private Sybil Agent credential only to the upstream Authorization header; browser credentials are not forwarded to the agent.

Native pi-room.v1 frames are forwarded after authentication. This protocol identifier is retained for compatibility with existing clients:

{ "type": "ready", "protocol": "pi-room.v1", "conversationId": "1", "model": "..." }

{
  "type": "messages",
  "conversationId": "1",
  "messages": [{ "id": "outbound:44", "role": "assistant", "content": "The result is ready.", "createdAt": "2026-10-14T18:00:00.000Z", "metadata": { "kind": "message", "entryId": 44, "status": "complete" } }],
  "cursor": "44",
  "beforeCursor": "44",
  "hasMore": false
}

{ "type": "activity", "conversationId": "1", "activity": { "state": "thinking", "isThinking": true, "queued": 0, "backgroundWorkers": 0, "tools": [], "queuedMessages": [] } }

Activity is forwarded from Sybil Agent and also describes tools, queued messages, durable waiting, errors, and workers. Use isThinking for the thinking indicator and backgroundWorkers for ongoing delegated work; no model reasoning is exposed. Tools arrive as role: "tool" messages with metadata.kind: "tool_call", matching Sybil's tool renderer. REST pages and socket messages are upserts keyed by stable id. Save each returned cursor after applying its entire frame, even when it contains no visible messages.

Clients may send { "type": "ping" } (receiving { "type": "pong" }) or { "type": "subscribe", "after": "44" } to change the replay cursor. Transport ping/pong heartbeats detect dead connections. Other writes belong on REST. Errors use { "type": "error", "error": "..." }. Upstream loss closes with 1011 or 1012; slow consumers may receive 1013. Reconnect with backoff and the last merged cursor. Missing agent configuration closes with 4503.

The proxy closes upstream sockets when browsers leave or Sybil shuts down. History remains durable in Sybil Agent. Shared read position is separate from delivery: update GET/PUT /v1/agent/read-state only when messages have actually been viewed.

The web client first accepts source-bound read state and one recent REST history window, then authenticates this socket with that window's cursor. Initial hasMore refers to older history and does not trigger a forward replay from zero. Older pages load on demand using a separate beforeCursor; delayed older pages preserve live edits and never change the socket replay position. Reconnects and foreground catch-up use the last accepted live cursor as before. Incoming output follows the viewport only while it is pinned to the newest end; older prepends and dynamic heights preserve the reader's keyed row. Only a focused, visible, settled latest view acknowledges the rendered cursor, marking the conversation read through that point. See web history windows and unread counts for unloaded-history count semantics and rendering limits.

Agent attachment history, queued activity and replay

Assistant image output uses the same messages frames and durable history rows, with role: "assistant", id: "outbound:<entryId>", metadata.kind: "message", metadata.status: "complete", and metadata.agentAttachments. Each image descriptor contains id, kind: "image", filename, mimeType, sizeBytes and sha256 only; there is no source/workspace path or URL. An omitted caption yields empty content; clients must retain this image-only row. Merge/replay by the unchanged message ID, render authenticated image bytes inline in a fixed frame, and retain loaded history even if the optional capabilities.assistantImages.version: 1 is absent later. That capability does not authorize input uploads. Native tool image blocks are not automatically forwarded; the root agent explicitly calls output_image. Public output tool rows redact arguments/results, and Agent Markdown does not fetch images. See assistant image output for publishing, retry, auth and generic push-preview semantics. No new socket event is introduced.

Uploads and attached sends use REST, as specified in rest.md. WebSocket authentication, event types, stable message IDs, cursor semantics and raw tool rows are unchanged. Capability discovery is optional GET /v1/agent → capabilities.attachments.version: 1; absence disables new attachment sends. Upgrade/authenticate the native runtime and backend before advertising it.

Both messages.messages[] and activity.activity.queuedMessages[] carry attachment descriptors under metadata.agentAttachments, distinct from ordinary chat's inline metadata.attachments:

{
  "id": "inbound:sybil:client-stable-id",
  "role": "user",
  "content": "Read these notes",
  "createdAt": "2026-10-14T18:00:00.000Z",
  "metadata": {
    "kind": "message",
    "channel": "sybil",
    "status": "queued",
    "agentAttachments": [{
      "id": "att_12345678-1234-4234-8234-123456789abc",
      "kind": "text",
      "filename": "notes.txt",
      "mimeType": "text/plain",
      "sizeBytes": 12,
      "sha256": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
      "workspacePath": "/data/workspace/uploads/att_12345678-1234-4234-8234-123456789abc/notes.txt"
    }]
  }
}

workspacePath is optional, authoritative runtime text for display/copy, never a download URL or send parameter. Only attachments:[{id}] goes into POST messages. Committed/replayed messages retain the same ordered descriptors and original caption; file-only content is Attached: notes.txt.. Internal model manifests/previews and native base64 are excluded from public user message content. Root/worker native inputs and raw authorized model endpoints may contain the real image blocks; native token auth must protect all of them.

Merge queued and committed messages by stable message ID. A matching ID alone does not confirm an attachment-bearing optimistic message: compare the ordered metadata.agentAttachments[].id against its local IDs as well. Retain descriptors, error and retry identity on a mismatch or uncertain receipt; show the error inline for touch clients. Matching canonical echoes can acknowledge delivery even if HTTP was lost. Preserve raw tool rows/cursors in client room state and existing transcript filtering. Old clients can ignore the additive metadata and still see a meaningful caption.

Retrieve bytes using authenticated GET /v1/agent/attachments/:id/content; fetch with an Authorization header and use/revoke browser blob URLs for previews. Never add credentials to URLs or use a public file route. Persist completed descriptors and pending upload IDs with the composer/outbox; navigation cannot silently turn an attached draft into a text-only send. Incomplete reloaded uploads require verified file reselection or explicit removal, per the REST contract.

Attached input admission and its request binding are atomic. Replay verifies the durable accepted payload, including the ordered attachment IDs, before acknowledging it. Queued attachment metadata survives restart even if placement did not run before the crash; normal queue recovery places the original input without duplicate charge or delivery. Worker preparation saturation retries through durable Reporter checkpoints; permanent admission failures appear through the existing worker-report path.

Agent attention and shared read events (push v1)

Web auth frames always add clientKind:"web" to the existing type, token, and optional after. They also add presenceSessionId:"per-document-uuid" when secure UUID creation succeeds; omit only that field if unavailable. Report the same UUID to PUT /v1/agent/presence/:sessionId across the entire authenticated workspace. Native iOS clients add clientKind:"native"; no browser-attention lease is required for a native app. These declarations classify a client under the existing shared principal and do not grant new authority. Explicit document visibility/focus/interaction receipts, bound to that credential generation and session, determine attention independently of sockets. Every fresh visible/focused session suppresses, even while idle; hidden/blur transitions and recent visible interaction have 5-second grace, and unrenewed explicit observations expire after 60 seconds.

Old/unclassified authenticated sockets and web sockets with no matching observation share one persisted 5-second introduction grace per credential generation. Reconnects, missing/new UUIDs and transport heartbeat cannot extend or restart it. A spent anonymous window does not weaken any genuine document lease; an expired observed lease does not become unknown merely because its socket remains connected. Ping/pong updates only existing connection liveness, preserving the original kind/session and never recreating a removed row as unknown. The proxy classifies itself upstream using the authenticated X-Sybil-Client: backend-proxy-v1 upgrade header.

The backend adds this optional server-to-client frame after a successful monotonic read advance, including a runtime-native browser acknowledgment merged into the shared row:

{"type":"read_state","cursor":"44","updatedAt":"2026-10-15T03:00:00.000Z","sourceId":"00000000-0000-4000-8000-000000000001"}

sourceId is nullable before source discovery and is never silently rebound after runtime replacement. Source replacement quarantines push delivery; the backend does not reset old read cursors. Apply read cursors monotonically within their source, and only clear notifications whose payload source matches this frame and whose cursor is at/below it. Frames without a source remain usable by legacy unread indicators but cannot safely authorize notification clearing. Clients still poll GET /v1/agent/read-state on foreground/reconnect to recover missed frames; receiving history, activity, a push, or a notification tap does not acknowledge that history was viewed.

Distinguish an unbound room from one already bound to a legacy absent/null source or a source UUID. Bind on the first accepted read state; if history arrives first, pin the legacy state before publishing it. A different source in a later GET, stream frame or PUT acknowledgment, including null-to-UUID or UUID-to-null, must stop stale read writes and require an explicit fresh room load. Preserve the old snapshot and cursors until that reload, then fetch read state before fresh history; never attach a new UUID to an old cursor. Ordinary foreground/network recovery must not reset this stopped room. HTTP 409, an invalid acknowledgment cursor or an acknowledgment below the requested cursor also stops the writer. A transient transport failure can retry only within the same pinned source. None of these client reloads resets server read state.

Older clients may ignore unknown server frame types. read_state and presence are not client subscribe messages and are never forwarded to runtime as generic frames. Push consent, revoke, presence, retry/expiry and notification payload contracts live in rest.md. The runtime built-in browser has its own authenticated presence/read inputs and bounded anonymous grace per runtime source. The separate 60-second restart safety intervals and source/clock uncertainty checks remain in effect. No Web Push is used for invisible read synchronization.

Enabled notifications include a bounded plain-text preview of the eligible completed root public assistant message by default, including on the lock screen when OS settings allow it. The backend resolves only that candidate's exact public content, never activity/reasoning/tool/worker fields or linked resources. APNs uses a native string aps.alert.body; Web Push uses body. This does not change message IDs, stream frames, read acknowledgment or clearing semantics. See the REST contract for Markdown flattening, payload budgets and web worker update ordering.