# Streaming Chat API Contract

This document defines the server-sent events (SSE) contract for chat completions.

Endpoint:
- `POST /v1/chat-completions/stream`
- `POST /v1/chats/:chatId/stream/attach`

Transport:
- HTTP response uses `Content-Type: text/event-stream; charset=utf-8`
- Events are emitted in SSE format (`event: ...`, `data: ...`)
- Request body is JSON
- Request body supports the same inline attachment schema and limits documented in `docs/api/rest.md`.

Authentication:
- Same as REST endpoints (`Authorization: Bearer <token>` when token mode is enabled)

## Request Body

```json
{
  "chatId": "optional-chat-id",
  "persist": true,
  "clientRequestId": "optional-client-generated-id",
  "provider": "openai|anthropic|xai|gemini|hermes-agent",
  "model": "string",
  "messages": [
    {
      "role": "system|user|assistant|tool",
      "content": "string",
      "name": "optional",
      "attachments": [
        {
          "kind": "image",
          "id": "attachment-id",
          "filename": "photo.jpg",
          "mimeType": "image/jpeg",
          "sizeBytes": 12345,
          "dataUrl": "data:image/jpeg;base64,..."
        },
        {
          "kind": "text",
          "id": "attachment-id",
          "filename": "notes.md",
          "mimeType": "text/markdown",
          "sizeBytes": 4567,
          "text": "# Notes\\n...",
          "truncated": false
        }
      ]
    }
  ],
  "additionalSystemPrompt": "optional one-off system prompt",
  "enabledTools": ["web_search", "fetch_url"],
  "temperature": 0.2,
  "maxTokens": 256
}
```

Notes:
- `persist` defaults to `true`.
- If `persist` is `true` and `chatId` is omitted, backend creates a new chat.
- If `chatId` is provided, backend validates it exists.
- If `persist` is `false`, `chatId` must be omitted. Backend does not create a chat and does not persist input messages, tool-call messages, assistant output, or `LlmCall` metadata.
- `clientRequestId` is optional and is only valid for a persisted stream with a `chatId`. Clients should generate one stable, unique value per user submission and reuse it when retrying a disconnected request.
- A retry with the same `chatId` and `clientRequestId` attaches to and replays the matching active stream. If that submission already completed, the endpoint replays `meta` and `done` without invoking the provider again. This makes retrying the initial streaming `POST` idempotent.
- `clientRequestId` values may be up to 128 characters. The server stores the value in `metadata.clientRequestId` on the submitted user message and completed assistant message.
- For persisted streams, backend stores only new non-assistant input history rows to avoid duplicates.
- `additionalSystemPrompt`, when present directly or loaded from stored chat settings, is prepended to the provider request as a `system` message and is not inserted into the persisted chat transcript by this endpoint.
- `enabledTools` limits Sybil-managed tools for this request. When omitted for a saved chat, the stored chat setting is used; otherwise all available tools are enabled by default. An empty array disables Sybil-managed tools.
- `maxTokens` is optional. For `anthropic`, when omitted the backend requests the selected model's maximum output token limit from Anthropic's Models API and uses that as `max_tokens`; if the model limit cannot be loaded, the fallback is 128000. For other providers, omitted `maxTokens` is not sent as an explicit cap.
- Attachments are optional and are persisted under `message.metadata.attachments` on stored user messages when `persist` is `true`.

Persisted chat streams with a `chatId` are backend-owned active runs:
- Once started, the backend keeps the stream running even if the HTTP client disconnects or refreshes.
- The backend reserves the active run before persisting submitted messages, preventing a root or fork deletion from interleaving with stream setup.
- While running, `GET /v1/active-runs` includes the `chatId`.
- Starting a second persisted stream for the same active `chatId` returns `409`, unless its `clientRequestId` matches the active submission, in which case the existing stream is replayed.
- Starting a persisted stream while that chat family is being deleted returns `409 { "message": "chat family deletion already in progress" }`.
- Clients can reattach with `POST /v1/chats/:chatId/stream/attach`.

## Attach Endpoint

`POST /v1/chats/:chatId/stream/attach`
- Body: none.
- Response uses the same `text/event-stream` transport and event names as `POST /v1/chat-completions/stream`.
- Replays buffered events for the active in-memory stream, then emits new events until `done` or `error`.
- Returns `404 { "message": "active chat stream not found" }` if no stream is currently active for that chat.
- Authentication is the same as all other API endpoints.

This endpoint is intended for clients that restored an active `chatId` from `GET /v1/active-runs`, especially after browser refresh. Replayed `delta` events may include text that was originally emitted before the client attached.

## Event Stream Contract

Event order:
1. Exactly one `meta`
2. Zero or more `tool_call`
3. Zero or more `delta`
4. Exactly one terminal event: `done` or `error`

Each tool invocation can emit multiple `tool_call` events with the same `toolCallId`. The backend emits `status: "initiated"` before the tool starts executing, then emits `status: "completed"` or `status: "failed"` when execution finishes. Clients should upsert by `toolCallId` instead of appending each event.

### `meta`

```json
{
  "type": "meta",
  "chatId": "chat-id-or-null",
  "callId": "llm-call-id-or-null",
  "provider": "openai",
  "model": "gpt-4.1-mini"
}
```

For `persist: false` streams, `chatId` and `callId` are `null`.

### `delta`

```json
{ "type": "delta", "text": "next chunk" }
```

`text` may contain partial words, punctuation, or whitespace.

### `tool_call`

```json
{
  "toolCallId": "call_123",
  "name": "web_search",
  "status": "initiated",
  "summary": "Searching web for 'latest CPI release'.",
  "args": { "query": "latest CPI release" },
  "startedAt": "2026-03-02T10:00:00.000Z"
}
```

Terminal tool-call event:

```json
{
  "toolCallId": "call_123",
  "name": "web_search",
  "status": "completed",
  "summary": "Performed web search for 'latest CPI release'.",
  "args": { "query": "latest CPI release" },
  "startedAt": "2026-03-02T10:00:00.000Z",
  "completedAt": "2026-03-02T10:00:00.820Z",
  "durationMs": 820,
  "resultPreview": "{\"ok\":true,...}"
}
```

`status` is one of `initiated`, `completed`, or `failed`. `completedAt` and `durationMs` are only present on terminal events. `error` is present on failed terminal events; `resultPreview` is present on terminal events when available.

### `done`

```json
{
  "type": "done",
  "text": "full assistant response",
  "usage": {
    "inputTokens": 123,
    "outputTokens": 456,
    "totalTokens": 579
  }
}
```

`usage` may be omitted when provider does not expose final token accounting for stream mode.

### `error`

```json
{ "type": "error", "message": "provider timeout" }
```

## Provider Streaming Behavior

- `openai`: backend uses OpenAI's Responses API and may execute internal function tool calls (`web_search`, `fetch_url`, optional `codex_exec`, and optional `shell_exec`) before producing final text.
- `anthropic`: backend uses Anthropic's Messages API and may execute the same internal tools with `tool_use`/`tool_result` content blocks before producing final text.
- `xai`: backend uses xAI's OpenAI-compatible Chat Completions API and may execute the same internal tool calls before producing final text.
- `gemini`: backend uses Google's native Gemini `streamGenerateContent` API and may execute the same internal tool calls before producing final text.
- `fetch_url` sends browser-like navigation headers for outbound URL requests to reduce false 403s from sites that reject generic server clients.
- `hermes-agent`: backend uses the configured Hermes Agent OpenAI-compatible Chat Completions API. Sybil does not add its own tool definitions for this provider; Hermes Agent handles its own tools server-side. Custom Hermes stream events are normalized away unless they produce text deltas in this SSE contract.
- `openai`: image attachments are sent as Responses `input_image` items; text attachments are sent as `input_text` items.
- `gemini`: image attachments are sent as native Gemini `inlineData` parts; text attachments are inlined as text parts.
- `xai` and `hermes-agent`: image attachments are sent as Chat Completions content parts; text attachments are inlined as text parts.
- `openai`: Responses calls that can enter the server-managed tool loop use `store: true` so reasoning and function-call items can be passed between tool rounds.
- `anthropic`: streamed via event stream; emits `delta` from `content_block_delta` with `text_delta`, and emits normalized `tool_call` SSE events when Anthropic `tool_use` blocks are executed. Image attachments are sent as base64 `image` blocks and text attachments are appended as `text` blocks.
- `web_search` uses `CHAT_WEB_SEARCH_ENGINE`: `exa` (default), `brave` (requires `BRAVE_SEARCH_API_KEY`), or `searxng` (requires `SEARXNG_BASE_URL`; the instance must allow `format=json`). This only affects chat-mode tool calls, not search-mode endpoints.
- Brave searches are queued and evenly paced according to the shortest window in Brave's `X-RateLimit-Policy` response header. The backend also honors `X-RateLimit-Remaining`/`X-RateLimit-Reset` and retries `429` responses up to three times with reset-aware exponential backoff; quota resets beyond the bounded retry window fail immediately.
- `codex_exec` is available only when `CHAT_CODEX_TOOL_ENABLED=true`. It SSHes to `CHAT_CODEX_REMOTE_HOST`, creates/uses `CHAT_CODEX_REMOTE_WORKDIR`, and runs `codex exec --dangerously-bypass-approvals-and-sandbox --skip-git-repo-check <non-interactive wrapped prompt>` there with SSH stdin closed. Prefer `CHAT_CODEX_SSH_KEY_PATH` with a read-only mounted private key; `CHAT_CODEX_SSH_PRIVATE_KEY_B64` is also supported.
- `shell_exec` is available only when `CHAT_SHELL_TOOL_ENABLED=true`. It uses the same devbox SSH configuration, starts in `CHAT_CODEX_REMOTE_WORKDIR`, and runs non-interactive shell commands there with SSH stdin closed, not inside the Sybil server container.
- `CHAT_MAX_TOOL_ROUNDS` controls how many model/tool result cycles may occur before the backend returns a tool-call limit message; default is 100.

Tool-enabled streaming notes (`openai`/`anthropic`/`xai`/`gemini`):
- Stream still emits standard `meta`, `delta`, `done|error` events.
- Stream may emit `tool_call` events while tool calls are executed.
- `delta` events carry assistant text and are emitted incrementally for normal text rounds. The backend may buffer model-native text briefly while determining whether a provider round contains tool calls.
- OpenAI Responses stream events are normalized by the backend into this SSE contract; clients do not consume OpenAI's raw Responses stream event names.

## Persistence + Consistency Model

Backend database remains source of truth.

For persisted streams:
- Client may optimistically render accumulated `delta` text.
- Backend emits initiated tool-call events without persisting them.
- Backend persists each completed or failed tool call as a `tool` message before emitting its terminal `tool_call` SSE event, so chat detail refreshes can show completed tool calls while the assistant response is still running.

On successful persisted completion:
- Backend persists assistant `Message` and updates `LlmCall` usage/latency in a transaction.
- Backend then emits `done`.

On persisted failure:
- Backend records call error and emits `error`.

For `persist: false` streams:
- Client may render the same `meta`, `tool_call`, `delta`, and terminal events.
- Backend does not write any chat, message, tool-call log, assistant output, or call metadata rows.
- `done.text` is the canonical assistant text if the client later imports the result into a saved chat.

Client recommendation (for iOS/web):
1. Render deltas in real time for UX.
2. On `done`, refresh chat detail from REST (`GET /v1/chats/:chatId`) and use DB-backed data as canonical.
3. On `error`, preserve user input and show retry affordance.

## SSE Parsing Rules

- Concatenate multiple `data:` lines with newline before JSON parse.
- Event completes on blank line.
- Ignore unknown event names for forward compatibility.

## Example Stream

```text
event: meta
data: {"type":"meta","chatId":"c1","callId":"k1","provider":"openai","model":"gpt-4.1-mini"}

event: delta
data: {"type":"delta","text":"Hello"}

event: delta
data: {"type":"delta","text":" world"}

event: done
data: {"type":"done","text":"Hello world"}

```
