Support durable assistant image attachments in Agent clients
This commit is contained in:
+44
-1
@@ -708,11 +708,54 @@ Global `SYBIL_AGENT_ATTACHMENT_QUOTA_BYTES` defaults to 1 GiB. Concurrent reserv
|
||||
|
||||
Release requires native `SYBIL_AGENT_AUTH_MODE=token` with nonempty `SYBIL_AGENT_API_KEY`, plus backend `ADMIN_TOKEN` and matching server-side `SYBIL_AGENT_API_KEY` / `SYBIL_AGENT_BASE_URL`. Native open mode cannot advertise/store/download attachments. A runtime containing attachment records/files refuses open startup; raw/model APIs and sockets are protected in token mode. An open Sybil proxy also refuses access to an attachment-capable native runtime, rather than exposing its history through a private server credential. Explicit text-only open development remains supported with an open/legacy runtime lacking attachment capability. Native keys never enter browser/iOS payloads. No insecure development opt-in is enabled.
|
||||
|
||||
The runtime source is built into its container: rebuild it with pinned busboy/sharp dependencies for the target Node/Linux platform before release. The deployed open auth mode must be changed separately; no deployed config or mount is modified by this implementation. Backend and web must be upgraded with runtime capability/auth configured before clients advertise uploads.
|
||||
The runtime source is built into its container: rebuild it with pinned busboy/sharp dependencies for the target Node/Linux platform before release. Deployment must use native token auth and matching backend credentials before clients advertise attachments; no additional public mount is needed.
|
||||
|
||||
Browser drafts retain upload IDs and completed descriptors across pane changes and reload. Incomplete uploads require reselection of the original filename/type/size and, once network delivery has begun, matching SHA-256 bytes; retries reuse uploadId. Sending persists the attachment-bearing outbox before clearing the draft. Message confirmation requires the same ordered attachment IDs in receipts and canonical echoes. Mismatched echoes must retain local attachments and delivery errors; they are not acknowledgement of a file submission.
|
||||
|
||||
|
||||
### Assistant image output (v1)
|
||||
|
||||
The independent optional capability `capabilities.assistantImages: {"version":1}`
|
||||
advertises the runtime's root-only `output_image` tool. It does **not** enable input
|
||||
upload controls; those still require `capabilities.attachments.version == 1`.
|
||||
Authenticated downloads accept either supported capability. Open proxies refuse
|
||||
history and socket access if either private attachment capability is advertised.
|
||||
Capability absence does not hide previously stored assistant image messages.
|
||||
|
||||
The agent invokes `output_image({path, uploadId, filename?, caption?})` with a stable
|
||||
UUID retry identity and a local persistent raster exported beneath the workspace's
|
||||
`artifacts/` or `downloads/` directory. It copies validated bytes into the existing
|
||||
attachment store and publishes one durable assistant message. This is an internal
|
||||
agent tool, with no browser/iOS publishing endpoint or client-supplied role.
|
||||
Workers cannot publish through it. No URL fetch, remote image proxy, file execution
|
||||
or implicit forwarding of native tool image blocks is performed. Generation is
|
||||
available only when a separate generation provider/tool is actually installed.
|
||||
|
||||
Output uses the same original-byte/hash/quota limits (PNG/JPEG/GIF/WebP, 6 MiB,
|
||||
50 million pixels, 16,384 pixels per axis), authenticated content route and backup
|
||||
policy as uploads. Sources must be confined regular files without symlink/hardlink
|
||||
substitution or private path segments. Upload IDs are bound to output provenance;
|
||||
conflicting arguments fail and identical retries retain their message/attachment IDs.
|
||||
Retries reuse the stored snapshot even if the original source moves or changes;
|
||||
publishing changed bytes requires a new upload UUID.
|
||||
|
||||
Public output rows have `id: "outbound:<entryId>"`, `role: "assistant"`,
|
||||
`metadata.kind: "message"`, `metadata.status: "complete"`, and
|
||||
`metadata.agentAttachments` containing image descriptors. `content` is the optional
|
||||
caption, otherwise an empty string; attachment-only rows remain visible. Descriptors
|
||||
contain only `id`, `kind`, safe `filename`, `mimeType`, `sizeBytes` and `sha256`.
|
||||
They contain no workspace/source path, URL, credential, or image bytes. The public
|
||||
tool row redacts output tool arguments/results; internal task/submission provenance
|
||||
stays in the existing durable outbox.
|
||||
|
||||
Web/iOS automatically render assistant rasters in fixed preview frames, retrieving
|
||||
only authenticated ID-based originals, checking size and SHA-256, rejecting
|
||||
redirects, and offering retry/download. Web limits downloads to two at a time and
|
||||
uses revocable blob URLs. Clients discard assistant descriptor paths and unsupported
|
||||
types. Agent Markdown cannot load images; external links remain explicit navigation.
|
||||
Image-bearing push alerts use **You have a new message.**, including when a caption
|
||||
exists, and never include attachment names, paths, bytes or URLs.
|
||||
|
||||
## Agent push notifications (v1)
|
||||
|
||||
All push/config/presence routes require a configured `ADMIN_TOKEN` and bearer authentication; open development mode returns 503. Missing/wrong bearer returns 401/403. Bodies are limited to 16 KiB, responses use `Cache-Control: no-store`, invalid bodies return 400, and unavailable configuration returns 503. Do not put credentials, device tokens or subscription endpoints into URLs. Ownership is one **shared authenticated personal principal**, derived on the server from the configured credential generation. Anyone possessing that shared token has authority over all its installations; UUIDs identify installations and do not create per-human isolation. No `ownerId` is accepted. Changing `ADMIN_TOKEN` invalidates all prior push opt-ins. Protect the SQLite database/backups because endpoint credentials are stored there.
|
||||
|
||||
@@ -351,6 +351,20 @@ for unloaded-history count semantics and rendering limits.
|
||||
|
||||
### Agent attachment history, queued activity and replay
|
||||
|
||||
Assistant image output uses the same `messages` frames and durable history rows,
|
||||
with `role: "assistant"`, `id: "outbound:<entryId>"`, `metadata.kind: "message"`,
|
||||
`metadata.status: "complete"`, and `metadata.agentAttachments`. Each image descriptor
|
||||
contains `id`, `kind: "image"`, `filename`, `mimeType`, `sizeBytes` and `sha256` only;
|
||||
there is no source/workspace path or URL. An omitted caption yields empty `content`;
|
||||
clients must retain this image-only row. Merge/replay by the unchanged message ID,
|
||||
render authenticated image bytes inline in a fixed frame, and retain loaded history
|
||||
even if the optional `capabilities.assistantImages.version: 1` is absent later.
|
||||
That capability does not authorize input uploads. Native tool image blocks are not
|
||||
automatically forwarded; the root agent explicitly calls `output_image`. Public
|
||||
output tool rows redact arguments/results, and Agent Markdown does not fetch images.
|
||||
See [assistant image output](./rest.md#assistant-image-output-v1) for publishing,
|
||||
retry, auth and generic push-preview semantics. No new socket event is introduced.
|
||||
|
||||
Uploads and attached sends use REST, as specified in [rest.md](./rest.md#agent-attachments-v1). WebSocket authentication, event types, stable message IDs, cursor semantics and raw tool rows are unchanged. Capability discovery is optional `GET /v1/agent` → `capabilities.attachments.version: 1`; absence disables new attachment sends. Upgrade/authenticate the native runtime and backend before advertising it.
|
||||
|
||||
Both `messages.messages[]` and `activity.activity.queuedMessages[]` carry attachment descriptors under **`metadata.agentAttachments`**, distinct from ordinary chat's inline `metadata.attachments`:
|
||||
|
||||
Reference in New Issue
Block a user