Sybil-2/docs/api/streaming-chat.md

# Streaming Chat API Contract

This document defines the server-sent events (SSE) contract for chat completions and Quick Questions, and the WebSocket contract for the persistent Agent room.

Endpoint:
- `POST /v1/chat-completions/stream`
- `POST /v1/chats/:chatId/stream/attach`
- `POST /v1/quick-questions/stream`

Transport:
- HTTP response uses `Content-Type: text/event-stream; charset=utf-8`
- Events are emitted in SSE format (`event: ...`, `data: ...`)
- Request body is JSON
- Chat completion request bodies support the same inline attachment schema and limits documented in `docs/api/rest.md`. Quick Questions accept text only.

Authentication:
- Same as REST endpoints (`Authorization: Bearer <token>` when token mode is enabled)
- The web client checks [`GET /v1/auth/session`](rest.md#get-v1authsession) before opening streams. An initial missing-token response prompts sign-in without an error; an incorrect supplied token is an authentication failure. The web sign-in form has no tokenless bypass; open development servers still authenticate automatically. Stream authorization requirements are unchanged.

## Chat Completion Request Body

```json
{
  "chatId": "optional-chat-id",
  "persist": true,
  "clientRequestId": "optional-client-generated-id",
  "provider": "openai|anthropic|xai|gemini|hermes-agent",
  "model": "string",
  "messages": [
    {
      "role": "system|user|assistant|tool",
      "content": "string",
      "name": "optional",
      "attachments": [
        {
          "kind": "image",
          "id": "attachment-id",
          "filename": "photo.jpg",
          "mimeType": "image/jpeg",
          "sizeBytes": 12345,
          "dataUrl": "data:image/jpeg;base64,..."
        },
        {
          "kind": "text",
          "id": "attachment-id",
          "filename": "notes.md",
          "mimeType": "text/markdown",
          "sizeBytes": 4567,
          "text": "# Notes\\n...",
          "truncated": false
        }
      ]
    }
  ],
  "additionalSystemPrompt": "optional one-off system prompt",
  "enabledTools": ["web_search", "fetch_url"],
  "temperature": 0.2,
  "maxTokens": 256
}
```

Notes:
- `persist` defaults to `true`.
- If `persist` is `true` and `chatId` is omitted, backend creates a new chat.
- If `chatId` is provided, backend validates it exists.
- If `persist` is `false`, `chatId` must be omitted. Backend does not create a chat and does not persist input messages, tool-call messages, assistant output, or `LlmCall` metadata.
- `clientRequestId` is optional and is only valid for a persisted stream with a `chatId`. Clients should generate one stable, unique value per user submission and reuse it when retrying a disconnected request.
- A retry with the same `chatId` and `clientRequestId` attaches to and replays the matching active stream. If that submission already completed, the endpoint replays `meta` and `done` without invoking the provider again. This makes retrying the initial streaming `POST` idempotent.
- `clientRequestId` values may be up to 128 characters. The server stores the value in `metadata.clientRequestId` on the submitted user message and completed assistant message.
- For persisted streams, backend stores only new non-assistant input history rows to avoid duplicates.
- `additionalSystemPrompt`, when present directly or loaded from stored chat settings, is prepended to the provider request as a `system` message and is not inserted into the persisted chat transcript by this endpoint.
- `enabledTools` limits Sybil-managed tools for this request. When omitted for a saved chat, the stored chat setting is used; otherwise all available tools are enabled by default. An empty array disables Sybil-managed tools.
- `maxTokens` is optional. For `anthropic`, when omitted the backend requests the selected model's maximum output token limit from Anthropic's Models API and uses that as `max_tokens`; if the model limit cannot be loaded, the fallback is 128000. For other providers, omitted `maxTokens` is not sent as an explicit cap.
- Attachments are optional and are persisted under `message.metadata.attachments` on stored user messages when `persist` is `true`.

Persisted chat streams with a `chatId` are backend-owned active runs:
- Once started, the backend keeps the stream running even if the HTTP client disconnects or refreshes.
- The backend reserves the active run before persisting submitted messages, preventing a root or fork deletion from interleaving with stream setup.
- While running, `GET /v1/active-runs` includes the `chatId`.
- Starting a second persisted stream for the same active `chatId` returns `409`, unless its `clientRequestId` matches the active submission, in which case the existing stream is replayed.
- Starting a persisted stream while that chat family is being deleted returns `409 { "message": "chat family deletion already in progress" }`.
- Clients can reattach with `POST /v1/chats/:chatId/stream/attach`.

## Attach Endpoint

`POST /v1/chats/:chatId/stream/attach`
- Body: none.
- Response uses the same `text/event-stream` transport and event names as `POST /v1/chat-completions/stream`.
- Replays buffered events for the active in-memory stream, then emits new events until `done` or `error`.
- Returns `404 { "message": "active chat stream not found" }` if no stream is currently active for that chat.
- Authentication is the same as all other API endpoints.

This endpoint is intended for clients that restored an active `chatId` from `GET /v1/active-runs`, especially after browser refresh. Replayed `delta` events may include text that was originally emitted before the client attached.

## Quick Question Endpoint

`POST /v1/quick-questions/stream`

Request body:
```json
{
  "provider": "openai|anthropic|xai|gemini|hermes-agent",
  "model": "string",
  "question": "What is the capital of France?",
  "enabledTools": ["web_search", "fetch_url"],
  "userLocation": "optional city, region, country",
  "temperature": 0.2,
  "maxTokens": 256
}
```

Behavior notes:
- `provider` and `model` are required. Web and native clients default Quick Question to `provider: "openai"` and `model: "chat-latest"` when no Quick Question selection has been saved. Saved Quick Question choices take precedence over this default and are separate from regular chat preferences.
- `question` is required, trimmed by the server, and must not be empty.
- The server prepends a Quick Question system prompt that asks for a succinct, direct, self-contained answer without follow-up questions. Clients do not send or maintain this prompt.
- Quick Questions are always non-persistent. The endpoint does not create a chat or store messages, tool-call logs, assistant output, or `LlmCall` metadata.
- The response uses the same `meta`, `tool_call`, `delta`, `done`, and `error` SSE events as chat completion streams. The `meta` event has `chatId: null` and `callId: null`.
- `enabledTools`, `temperature`, and `maxTokens` are optional and behave as they do for chat completion streams. When `enabledTools` is omitted, all available Sybil-managed tools are enabled by default.
- User location is inferred from the same request headers as chat completion streams when `userLocation` is omitted.

## Event Stream Contract

Event order:
1. Exactly one `meta`
2. Zero or more `tool_call`
3. Zero or more `delta`
4. Exactly one terminal event: `done` or `error`

Each tool invocation can emit multiple `tool_call` events with the same `toolCallId`. The backend emits `status: "initiated"` before the tool starts executing, then emits `status: "completed"` or `status: "failed"` when execution finishes. Clients should upsert by `toolCallId` instead of appending each event.

### `meta`

```json
{
  "type": "meta",
  "chatId": "chat-id-or-null",
  "callId": "llm-call-id-or-null",
  "provider": "openai",
  "model": "gpt-4.1-mini"
}
```

For `persist: false` streams, `chatId` and `callId` are `null`.

### `delta`

```json
{ "type": "delta", "text": "next chunk" }
```

`text` may contain partial words, punctuation, or whitespace.

### `tool_call`

```json
{
  "toolCallId": "call_123",
  "name": "web_search",
  "status": "initiated",
  "summary": "Searching web for 'latest CPI release'.",
  "args": { "query": "latest CPI release" },
  "startedAt": "2026-03-02T10:00:00.000Z"
}
```

Terminal tool-call event:

```json
{
  "toolCallId": "call_123",
  "name": "web_search",
  "status": "completed",
  "summary": "Performed web search for 'latest CPI release'.",
  "args": { "query": "latest CPI release" },
  "startedAt": "2026-03-02T10:00:00.000Z",
  "completedAt": "2026-03-02T10:00:00.820Z",
  "durationMs": 820,
  "resultPreview": "{\"ok\":true,...}"
}
```

`status` is one of `initiated`, `completed`, or `failed`. `completedAt` and `durationMs` are only present on terminal events. `error` is present on failed terminal events; `resultPreview` is present on terminal events when available.

### `done`

```json
{
  "type": "done",
  "text": "full assistant response",
  "usage": {
    "inputTokens": 123,
    "outputTokens": 456,
    "totalTokens": 579
  }
}
```

`usage` may be omitted when provider does not expose final token accounting for stream mode.

### `error`

```json
{ "type": "error", "message": "provider timeout" }
```

## Provider Streaming Behavior

- `openai`: backend uses OpenAI's Responses API and may execute internal function tool calls (`web_search`, `fetch_url`, optional `codex_exec`, and optional `shell_exec`) before producing final text.
- `anthropic`: backend uses Anthropic's Messages API and may execute the same internal tools with `tool_use`/`tool_result` content blocks before producing final text.
- `xai`: backend uses xAI's OpenAI-compatible Chat Completions API and may execute the same internal tool calls before producing final text.
- `gemini`: backend uses Google's native Gemini `streamGenerateContent` API and may execute the same internal tool calls before producing final text.
- `fetch_url` sends browser-like navigation headers for outbound URL requests to reduce false 403s from sites that reject generic server clients.
- `hermes-agent`: backend uses the configured Hermes Agent OpenAI-compatible Chat Completions API. Sybil does not add its own tool definitions for this provider; Hermes Agent handles its own tools server-side. Custom Hermes stream events are normalized away unless they produce text deltas in this SSE contract.
- `openai`: image attachments are sent as Responses `input_image` items; text attachments are sent as `input_text` items.
- `gemini`: image attachments are sent as native Gemini `inlineData` parts; text attachments are inlined as text parts.
- `xai` and `hermes-agent`: image attachments are sent as Chat Completions content parts; text attachments are inlined as text parts.
- `openai`: Responses calls that can enter the server-managed tool loop use `store: true` so reasoning and function-call items can be passed between tool rounds.
- `anthropic`: streamed via event stream; emits `delta` from `content_block_delta` with `text_delta`, and emits normalized `tool_call` SSE events when Anthropic `tool_use` blocks are executed. Image attachments are sent as base64 `image` blocks and text attachments are appended as `text` blocks.
- `web_search` uses `CHAT_WEB_SEARCH_ENGINE`: `exa` (default), `brave` (requires `BRAVE_SEARCH_API_KEY`), or `searxng` (requires `SEARXNG_BASE_URL`; the instance must allow `format=json`). This only affects chat-mode tool calls, not search-mode endpoints.
- Brave searches are queued and evenly paced according to the shortest window in Brave's `X-RateLimit-Policy` response header. The backend also honors `X-RateLimit-Remaining`/`X-RateLimit-Reset` and retries `429` responses up to three times with reset-aware exponential backoff; quota resets beyond the bounded retry window fail immediately.
- `codex_exec` is available only when `CHAT_CODEX_TOOL_ENABLED=true`. It SSHes to `CHAT_CODEX_REMOTE_HOST`, creates/uses `CHAT_CODEX_REMOTE_WORKDIR`, and runs `codex exec --dangerously-bypass-approvals-and-sandbox --skip-git-repo-check <non-interactive wrapped prompt>` there with SSH stdin closed. Prefer `CHAT_CODEX_SSH_KEY_PATH` with a read-only mounted private key; `CHAT_CODEX_SSH_PRIVATE_KEY_B64` is also supported.
- `shell_exec` is available only when `CHAT_SHELL_TOOL_ENABLED=true`. It uses the same devbox SSH configuration, starts in `CHAT_CODEX_REMOTE_WORKDIR`, and runs non-interactive shell commands there with SSH stdin closed, not inside the Sybil server container.
- `CHAT_MAX_TOOL_ROUNDS` controls how many model/tool result cycles may occur before the backend returns a tool-call limit message; default is 100.

Tool-enabled streaming notes (`openai`/`anthropic`/`xai`/`gemini`):
- Stream still emits standard `meta`, `delta`, `done|error` events.
- Stream may emit `tool_call` events while tool calls are executed.
- `delta` events carry assistant text and are emitted incrementally for normal text rounds. The backend may buffer model-native text briefly while determining whether a provider round contains tool calls.
- OpenAI Responses stream events are normalized by the backend into this SSE contract; clients do not consume OpenAI's raw Responses stream event names.

## Persistence + Consistency Model

Backend database remains source of truth.

For persisted streams:
- Client may optimistically render accumulated `delta` text.
- Backend emits initiated tool-call events without persisting them.
- Backend persists each completed or failed tool call as a `tool` message before emitting its terminal `tool_call` SSE event, so chat detail refreshes can show completed tool calls while the assistant response is still running.

On successful persisted completion:
- Backend persists assistant `Message` and updates `LlmCall` usage/latency in a transaction.
- Backend then emits `done`.

On persisted failure:
- Backend records call error and emits `error`.

For `persist: false` streams:
- Client may render the same `meta`, `tool_call`, `delta`, and terminal events.
- Backend does not write any chat, message, tool-call log, assistant output, or call metadata rows.
- `done.text` is the canonical assistant text if the client later imports the result into a saved chat.

Client recommendation (for iOS/web):
1. Render deltas in real time for UX.
2. On `done`, refresh chat detail from REST (`GET /v1/chats/:chatId`) and use DB-backed data as canonical.
3. On `error`, preserve user input and show retry affordance.

## SSE Parsing Rules

- Concatenate multiple `data:` lines with newline before JSON parse.
- Event completes on blank line.
- Ignore unknown event names for forward compatibility.

## Example Stream

```text
event: meta
data: {"type":"meta","chatId":"c1","callId":"k1","provider":"openai","model":"gpt-4.1-mini"}

event: delta
data: {"type":"delta","text":"Hello"}

event: delta
data: {"type":"delta","text":" world"}

event: done
data: {"type":"done","text":"Hello world"}

```

## Sybil Agent WebSocket

`GET /v1/agent/ws` upgrades to a continuous WebSocket room subscription. Behind the
web proxy use `/api/v1/agent/ws`, with `ws:` or `wss:` matching the page. It remains
connected after each assistant turn and receives later background results. Sending
messages uses `POST /v1/agent/messages`.

The browser must send this first frame within five seconds:

```json
{ "type": "auth", "token": "Sybil token", "after": "43" }
```

In open mode, use `token: null`. Omit `after` for recent history; on reconnect supply
the last successfully merged cursor. Invalid credentials close with 4401, malformed
initial frames with 4400. Malformed upgrade URLs and query parameters are rejected
with HTTP 400; unknown upgrade paths return HTTP 404. Never put tokens in URLs.
Sybil adds its private Sybil Agent credential only to the upstream Authorization header;
browser credentials are not forwarded to the agent.

Native `pi-room.v1` frames are forwarded after authentication. This protocol
identifier is retained for compatibility with existing clients:

```json
{ "type": "ready", "protocol": "pi-room.v1", "conversationId": "1", "model": "..." }
```

```json
{
  "type": "messages",
  "conversationId": "1",
  "messages": [{ "id": "outbound:44", "role": "assistant", "content": "The result is ready.", "createdAt": "2026-10-14T18:00:00.000Z", "metadata": { "kind": "message", "entryId": 44, "status": "complete" } }],
  "cursor": "44",
  "beforeCursor": "44",
  "hasMore": false
}
```

```json
{ "type": "activity", "conversationId": "1", "activity": { "state": "thinking", "isThinking": true, "queued": 0, "backgroundWorkers": 0, "tools": [], "queuedMessages": [] } }
```

Activity is forwarded from Sybil Agent and also describes tools, queued messages, durable
waiting, errors, and workers. Use `isThinking` for the thinking indicator and
`backgroundWorkers` for ongoing delegated work; no model reasoning is exposed. Tools arrive as `role: "tool"` messages
with `metadata.kind: "tool_call"`, matching Sybil's tool renderer. REST pages and
socket messages are upserts keyed by stable `id`. Save each returned cursor after
applying its entire frame, even when it contains no visible messages.

Clients may send `{ "type": "ping" }` (receiving `{ "type": "pong" }`) or
`{ "type": "subscribe", "after": "44" }` to change the replay cursor. Transport
ping/pong heartbeats detect dead connections. Other writes belong on REST.
Errors use `{ "type": "error", "error": "..." }`. Upstream loss closes with 1011
or 1012; slow consumers may receive 1013. Reconnect with backoff and the last merged
cursor. Missing agent configuration closes with 4503.

The proxy closes upstream sockets when browsers leave or Sybil shuts down.
History remains durable in Sybil Agent. Shared read position is separate from delivery:
update `GET`/`PUT /v1/agent/read-state` only when messages have actually been viewed.

### Agent attachment history, queued activity and replay

Uploads and attached sends use REST, as specified in [rest.md](./rest.md#agent-attachments-v1). WebSocket authentication, event types, stable message IDs, cursor semantics and raw tool rows are unchanged. Capability discovery is optional `GET /v1/agent` → `capabilities.attachments.version: 1`; absence disables new attachment sends. Upgrade/authenticate the native runtime and backend before advertising it.

Both `messages.messages[]` and `activity.activity.queuedMessages[]` carry attachment descriptors under **`metadata.agentAttachments`**, distinct from ordinary chat's inline `metadata.attachments`:

```json
{
  "id": "inbound:sybil:client-stable-id",
  "role": "user",
  "content": "Read these notes",
  "createdAt": "2026-10-14T18:00:00.000Z",
  "metadata": {
    "kind": "message",
    "channel": "sybil",
    "status": "queued",
    "agentAttachments": [{
      "id": "att_12345678-1234-4234-8234-123456789abc",
      "kind": "text",
      "filename": "notes.txt",
      "mimeType": "text/plain",
      "sizeBytes": 12,
      "sha256": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
      "workspacePath": "/data/workspace/uploads/att_12345678-1234-4234-8234-123456789abc/notes.txt"
    }]
  }
}
```

`workspacePath` is optional, authoritative runtime text for display/copy, never a download URL or send parameter. Only `attachments:[{id}]` goes into POST messages. Committed/replayed messages retain the same ordered descriptors and original caption; file-only content is `Attached: notes.txt.`. Internal model manifests/previews and native base64 are excluded from public user message content. Root/worker native inputs and raw authorized model endpoints may contain the real image blocks; native token auth must protect all of them.

Merge queued and committed messages by stable message ID. A matching ID alone does not confirm an attachment-bearing optimistic message: compare the ordered `metadata.agentAttachments[].id` against its local IDs as well. Retain descriptors, error and retry identity on a mismatch or uncertain receipt; show the error inline for touch clients. Matching canonical echoes can acknowledge delivery even if HTTP was lost. Preserve raw tool rows/cursors in client room state and existing transcript filtering. Old clients can ignore the additive metadata and still see a meaningful caption.

Retrieve bytes using authenticated `GET /v1/agent/attachments/:id/content`; fetch with an Authorization header and use/revoke browser blob URLs for previews. Never add credentials to URLs or use a public file route. Persist completed descriptors and pending upload IDs with the composer/outbox; navigation cannot silently turn an attached draft into a text-only send. Incomplete reloaded uploads require verified file reselection or explicit removal, per the REST contract.

Attached input admission and its request binding are atomic. Replay verifies the durable accepted payload, including the ordered attachment IDs, before acknowledging it. Queued attachment metadata survives restart even if placement did not run before the crash; normal queue recovery places the original input without duplicate charge or delivery. Worker preparation saturation retries through durable Reporter checkpoints; permanent admission failures appear through the existing worker-report path.


### Agent attention and shared read events (push v1)

Web auth frames always add `clientKind:"web"` to the existing `type`, `token`, and optional `after`. They also add `presenceSessionId:"per-document-uuid"` when secure UUID creation succeeds; omit only that field if unavailable. Report the same UUID to `PUT /v1/agent/presence/:sessionId` across the entire authenticated workspace. Native iOS clients add `clientKind:"native"`; no browser-attention lease is required for a native app. These declarations classify a client under the existing shared principal and do not grant new authority. Explicit document visibility/focus/interaction receipts, bound to that credential generation and session, determine attention independently of sockets. Every fresh visible/focused session suppresses, even while idle; hidden/blur transitions and recent visible interaction have 5-second grace, and unrenewed explicit observations expire after 60 seconds.

Old/unclassified authenticated sockets and web sockets with no matching observation share one persisted 5-second introduction grace per credential generation. Reconnects, missing/new UUIDs and transport heartbeat cannot extend or restart it. A spent anonymous window does not weaken any genuine document lease; an expired observed lease does not become unknown merely because its socket remains connected. Ping/pong updates only existing connection liveness, preserving the original kind/session and never recreating a removed row as unknown. The proxy classifies itself upstream using the authenticated `X-Sybil-Client: backend-proxy-v1` upgrade header.

The backend adds this optional server-to-client frame after a successful monotonic read advance, including a runtime-native browser acknowledgment merged into the shared row:

```json
{"type":"read_state","cursor":"44","updatedAt":"2026-10-15T03:00:00.000Z","sourceId":"00000000-0000-4000-8000-000000000001"}
```

`sourceId` is nullable before source discovery and is never silently rebound after runtime replacement. Source replacement quarantines push delivery; the backend does not reset old read cursors. Apply read cursors monotonically within their source, and only clear notifications whose payload source matches this frame and whose cursor is at/below it. Frames without a source remain usable by legacy unread indicators but cannot safely authorize notification clearing. Clients still poll `GET /v1/agent/read-state` on foreground/reconnect to recover missed frames; receiving history, activity, a push, or a notification tap does not acknowledge that history was viewed.

Distinguish an unbound room from one already bound to a legacy absent/null source or a source UUID. Bind on the first accepted read state; if history arrives first, pin the legacy state before publishing it. A different source in a later GET, stream frame or PUT acknowledgment, including null-to-UUID or UUID-to-null, must stop stale read writes and require an explicit fresh room load. Preserve the old snapshot and cursors until that reload, then fetch read state before fresh history; never attach a new UUID to an old cursor. Ordinary foreground/network recovery must not reset this stopped room. HTTP 409, an invalid acknowledgment cursor or an acknowledgment below the requested cursor also stops the writer. A transient transport failure can retry only within the same pinned source. None of these client reloads resets server read state.

Older clients may ignore unknown server frame types. `read_state` and presence are not client subscribe messages and are never forwarded to runtime as generic frames. Push consent, revoke, presence, retry/expiry and notification payload contracts live in [rest.md](./rest.md#agent-push-notifications-v1). The runtime built-in browser has its own authenticated presence/read inputs and bounded anonymous grace per runtime source. The separate 60-second restart safety intervals and source/clock uncertainty checks remain in effect. No Web Push is used for invisible read synchronization.

Enabled notifications include a bounded plain-text preview of the eligible completed root public assistant message by default, including on the lock screen when OS settings allow it. The backend resolves only that candidate's exact public `content`, never activity/reasoning/tool/worker fields or linked resources. APNs uses a native string `aps.alert.body`; Web Push uses `body`. This does not change message IDs, stream frames, read acknowledgment or clearing semantics. See the REST contract for Markdown flattening, payload budgets and web worker update ordering.