Compare commits

..
Author SHA1 Message Date
buzzert b743523fd5 ios: bound assistant image sets and keep output tools quiet 2026-10-20 14:31:15 -07:00
buzzert ee845c9fa3 ios: support safe assistant inline images 2026-10-20 14:31:15 -07:00
buzzert 9098fee32a Keep assistant image previews stable through live delivery and reload 2026-10-20 14:19:23 -07:00
buzzert 489c347976 Support durable assistant image attachments in Agent clients 2026-10-20 14:14:09 -07:00
buzzert 7b48201e28 web: open Agent at recent history and load older pages on demand 2026-10-16 21:18:15 -07:00
buzzert 671ee98ab1 ios: smooth Agent entrances and refresh app icon
TestFlight / Build and upload (push) Successful in 1m49s
Grow row height and spacing with the existing entrance animation, remove the Latest overlay while retaining scroll/read/send behavior, and reuse the current PWA icon artwork. Cover occupied-height interpolation and scroll-to-latest navigation in focused checks.
2026-10-16 16:54:46 -07:00
buzzert f59c7db7bb web: remove Agent jump-to-latest button 2026-10-16 16:42:58 -07:00
39 changed files with 1290 additions and 156 deletions
+66 -1
View File
@@ -605,6 +605,28 @@ durable entries and can advance without visible messages. Use returned cursors;
do not derive them from message IDs. `limit` bounds source entries, not displayed do not derive them from message IDs. `limit` bounds source entries, not displayed
messages. messages.
The web Agent starts with one cursorless `limit=100` window and renders it in
chronological order, initially positioned at its newest end. `hasMore` on this
window or a `before` page means **older** source entries exist; on an `after`
page it means **newer** entries remain. Each **Load older messages** action fetches
one `before=<beforeCursor>&limit=100` page, even if that page has no visible rows.
Older pages do not advance the live replay/read cursor or overwrite newer copies
of the same message. Initial loading never walks forward from zero or drains
older history. Reconnect recovery still follows `after` from the accepted live
cursor. Aborted/replaced room sessions discard late responses.
Web unread counts cover loaded public replies. A `+` suffix (or a dot when no
loaded reply is known unread) indicates that earlier messages may also be unread;
the API does not supply an exact total for unloaded history. Observing the latest
end in a focused visible conversation marks it read through that rendered cursor,
including earlier unloaded history. Fetching or reading older pages does not
acknowledge newer replies. An internal-only window with no rendered public
messages does not acknowledge history; older loading remains an explicit action.
Initial source-entry/network work is bounded; retained
messages/DOM grow with explicit older loads and live updates. Offscreen web rows
use CSS `content-visibility: auto`, which skips rendering work while retaining
chronological DOM, selection, find and accessibility; it is not DOM virtualization.
### `POST /v1/agent/messages` ### `POST /v1/agent/messages`
Body: `{ "content": "Check the oven", "requestId": "client-generated-uuid" }`. Body: `{ "content": "Check the oven", "requestId": "client-generated-uuid" }`.
@@ -686,11 +708,54 @@ Global `SYBIL_AGENT_ATTACHMENT_QUOTA_BYTES` defaults to 1 GiB. Concurrent reserv
Release requires native `SYBIL_AGENT_AUTH_MODE=token` with nonempty `SYBIL_AGENT_API_KEY`, plus backend `ADMIN_TOKEN` and matching server-side `SYBIL_AGENT_API_KEY` / `SYBIL_AGENT_BASE_URL`. Native open mode cannot advertise/store/download attachments. A runtime containing attachment records/files refuses open startup; raw/model APIs and sockets are protected in token mode. An open Sybil proxy also refuses access to an attachment-capable native runtime, rather than exposing its history through a private server credential. Explicit text-only open development remains supported with an open/legacy runtime lacking attachment capability. Native keys never enter browser/iOS payloads. No insecure development opt-in is enabled. Release requires native `SYBIL_AGENT_AUTH_MODE=token` with nonempty `SYBIL_AGENT_API_KEY`, plus backend `ADMIN_TOKEN` and matching server-side `SYBIL_AGENT_API_KEY` / `SYBIL_AGENT_BASE_URL`. Native open mode cannot advertise/store/download attachments. A runtime containing attachment records/files refuses open startup; raw/model APIs and sockets are protected in token mode. An open Sybil proxy also refuses access to an attachment-capable native runtime, rather than exposing its history through a private server credential. Explicit text-only open development remains supported with an open/legacy runtime lacking attachment capability. Native keys never enter browser/iOS payloads. No insecure development opt-in is enabled.
The runtime source is built into its container: rebuild it with pinned busboy/sharp dependencies for the target Node/Linux platform before release. The deployed open auth mode must be changed separately; no deployed config or mount is modified by this implementation. Backend and web must be upgraded with runtime capability/auth configured before clients advertise uploads. The runtime source is built into its container: rebuild it with pinned busboy/sharp dependencies for the target Node/Linux platform before release. Deployment must use native token auth and matching backend credentials before clients advertise attachments; no additional public mount is needed.
Browser drafts retain upload IDs and completed descriptors across pane changes and reload. Incomplete uploads require reselection of the original filename/type/size and, once network delivery has begun, matching SHA-256 bytes; retries reuse uploadId. Sending persists the attachment-bearing outbox before clearing the draft. Message confirmation requires the same ordered attachment IDs in receipts and canonical echoes. Mismatched echoes must retain local attachments and delivery errors; they are not acknowledgement of a file submission. Browser drafts retain upload IDs and completed descriptors across pane changes and reload. Incomplete uploads require reselection of the original filename/type/size and, once network delivery has begun, matching SHA-256 bytes; retries reuse uploadId. Sending persists the attachment-bearing outbox before clearing the draft. Message confirmation requires the same ordered attachment IDs in receipts and canonical echoes. Mismatched echoes must retain local attachments and delivery errors; they are not acknowledgement of a file submission.
### Assistant image output (v1)
The independent optional capability `capabilities.assistantImages: {"version":1}`
advertises the runtime's root-only `output_image` tool. It does **not** enable input
upload controls; those still require `capabilities.attachments.version == 1`.
Authenticated downloads accept either supported capability. Open proxies refuse
history and socket access if either private attachment capability is advertised.
Capability absence does not hide previously stored assistant image messages.
The agent invokes `output_image({path, uploadId, filename?, caption?})` with a stable
UUID retry identity and a local persistent raster exported beneath the workspace's
`artifacts/` or `downloads/` directory. It copies validated bytes into the existing
attachment store and publishes one durable assistant message. This is an internal
agent tool, with no browser/iOS publishing endpoint or client-supplied role.
Workers cannot publish through it. No URL fetch, remote image proxy, file execution
or implicit forwarding of native tool image blocks is performed. Generation is
available only when a separate generation provider/tool is actually installed.
Output uses the same original-byte/hash/quota limits (PNG/JPEG/GIF/WebP, 6 MiB,
50 million pixels, 16,384 pixels per axis), authenticated content route and backup
policy as uploads. Sources must be confined regular files without symlink/hardlink
substitution or private path segments. Upload IDs are bound to output provenance;
conflicting arguments fail and identical retries retain their message/attachment IDs.
Retries reuse the stored snapshot even if the original source moves or changes;
publishing changed bytes requires a new upload UUID.
Public output rows have `id: "outbound:<entryId>"`, `role: "assistant"`,
`metadata.kind: "message"`, `metadata.status: "complete"`, and
`metadata.agentAttachments` containing image descriptors. `content` is the optional
caption, otherwise an empty string; attachment-only rows remain visible. Descriptors
contain only `id`, `kind`, safe `filename`, `mimeType`, `sizeBytes` and `sha256`.
They contain no workspace/source path, URL, credential, or image bytes. The public
tool row redacts output tool arguments/results; internal task/submission provenance
stays in the existing durable outbox.
Web/iOS automatically render assistant rasters in fixed preview frames, retrieving
only authenticated ID-based originals, checking size and SHA-256, rejecting
redirects, and offering retry/download. Web limits downloads to two at a time and
uses revocable blob URLs. Clients discard assistant descriptor paths and unsupported
types. Agent Markdown cannot load images; external links remain explicit navigation.
Image-bearing push alerts use **You have a new message.**, including when a caption
exists, and never include attachment names, paths, bytes or URLs.
## Agent push notifications (v1) ## Agent push notifications (v1)
All push/config/presence routes require a configured `ADMIN_TOKEN` and bearer authentication; open development mode returns 503. Missing/wrong bearer returns 401/403. Bodies are limited to 16 KiB, responses use `Cache-Control: no-store`, invalid bodies return 400, and unavailable configuration returns 503. Do not put credentials, device tokens or subscription endpoints into URLs. Ownership is one **shared authenticated personal principal**, derived on the server from the configured credential generation. Anyone possessing that shared token has authority over all its installations; UUIDs identify installations and do not create per-human isolation. No `ownerId` is accepted. Changing `ADMIN_TOKEN` invalidates all prior push opt-ins. Protect the SQLite database/backups because endpoint credentials are stored there. All push/config/presence routes require a configured `ADMIN_TOKEN` and bearer authentication; open development mode returns 503. Missing/wrong bearer returns 401/403. Bodies are limited to 16 KiB, responses use `Cache-Control: no-store`, invalid bodies return 400, and unavailable configuration returns 503. Do not put credentials, device tokens or subscription endpoints into URLs. Ownership is one **shared authenticated personal principal**, derived on the server from the configured credential generation. Anyone possessing that shared token has authority over all its installations; UUIDs identify installations and do not create per-human isolation. No `ownerId` is accepted. Changing `ADMIN_TOKEN` invalidates all prior push opt-ins. Protect the SQLite database/backups because endpoint credentials are stored there.
+26
View File
@@ -337,8 +337,34 @@ The proxy closes upstream sockets when browsers leave or Sybil shuts down.
History remains durable in Sybil Agent. Shared read position is separate from delivery: History remains durable in Sybil Agent. Shared read position is separate from delivery:
update `GET`/`PUT /v1/agent/read-state` only when messages have actually been viewed. update `GET`/`PUT /v1/agent/read-state` only when messages have actually been viewed.
The web client first accepts source-bound read state and one recent REST history
window, then authenticates this socket with that window's `cursor`. Initial
`hasMore` refers to older history and does not trigger a forward replay from zero.
Older pages load on demand using a separate `beforeCursor`; delayed older pages
preserve live edits and never change the socket replay position. Reconnects and
foreground catch-up use the last accepted live cursor as before. Incoming output
follows the viewport only while it is pinned to the newest end; older prepends and
dynamic heights preserve the reader's keyed row. Only a focused, visible, settled
latest view acknowledges the rendered cursor, marking the conversation read
through that point. See [web history windows and unread counts](rest.md#get-v1agentmessages)
for unloaded-history count semantics and rendering limits.
### Agent attachment history, queued activity and replay ### Agent attachment history, queued activity and replay
Assistant image output uses the same `messages` frames and durable history rows,
with `role: "assistant"`, `id: "outbound:<entryId>"`, `metadata.kind: "message"`,
`metadata.status: "complete"`, and `metadata.agentAttachments`. Each image descriptor
contains `id`, `kind: "image"`, `filename`, `mimeType`, `sizeBytes` and `sha256` only;
there is no source/workspace path or URL. An omitted caption yields empty `content`;
clients must retain this image-only row. Merge/replay by the unchanged message ID,
render authenticated image bytes inline in a fixed frame, and retain loaded history
even if the optional `capabilities.assistantImages.version: 1` is absent later.
That capability does not authorize input uploads. Native tool image blocks are not
automatically forwarded; the root agent explicitly calls `output_image`. Public
output tool rows redact arguments/results, and Agent Markdown does not fetch images.
See [assistant image output](./rest.md#assistant-image-output-v1) for publishing,
retry, auth and generic push-preview semantics. No new socket event is introduced.
Uploads and attached sends use REST, as specified in [rest.md](./rest.md#agent-attachments-v1). WebSocket authentication, event types, stable message IDs, cursor semantics and raw tool rows are unchanged. Capability discovery is optional `GET /v1/agent` → `capabilities.attachments.version: 1`; absence disables new attachment sends. Upgrade/authenticate the native runtime and backend before advertising it. Uploads and attached sends use REST, as specified in [rest.md](./rest.md#agent-attachments-v1). WebSocket authentication, event types, stable message IDs, cursor semantics and raw tool rows are unchanged. Capability discovery is optional `GET /v1/agent` → `capabilities.attachments.version: 1`; absence disables new attachment sends. Upgrade/authenticate the native runtime and backend before advertising it.
Both `messages.messages[]` and `activity.activity.queuedMessages[]` carry attachment descriptors under **`metadata.agentAttachments`**, distinct from ordinary chat's inline `metadata.attachments`: Both `messages.messages[]` and `activity.activity.queuedMessages[]` carry attachment descriptors under **`metadata.agentAttachments`**, distinct from ordinary chat's inline `metadata.attachments`:
Binary file not shown.

After

Width:  |  Height:  |  Size: 791 KiB

@@ -1,7 +1,7 @@
{ {
"images" : [ "images" : [
{ {
"filename" : "buzzert_iOS_app_icon_prophet_Sibyl_simple_minimal_mystic_blue_b_08983f8f-941b-4ff3-abb4-9df3f3f7612d.png", "filename" : "AppIcon.png",
"idiom" : "universal", "idiom" : "universal",
"platform" : "ios", "platform" : "ios",
"size" : "1024x1024" "size" : "1024x1024"
+6 -2
View File
@@ -15,7 +15,9 @@ let package = Package(
targets: ["Sybil"]), targets: ["Sybil"]),
], ],
dependencies: [ dependencies: [
.package(url: "https://github.com/gonzalezreal/swift-markdown-ui.git", from: "2.4.1") .package(url: "https://github.com/gonzalezreal/swift-markdown-ui.git", from: "2.4.1"),
// Use MarkdownUI's existing parser to remove Agent image/HTML nodes structurally.
.package(url: "https://github.com/swiftlang/swift-cmark", from: "0.7.1")
], ],
targets: [ targets: [
// Targets are the basic building blocks of a package, defining a module or a test suite. // Targets are the basic building blocks of a package, defining a module or a test suite.
@@ -23,7 +25,9 @@ let package = Package(
.target( .target(
name: "Sybil", name: "Sybil",
dependencies: [ dependencies: [
.product(name: "MarkdownUI", package: "swift-markdown-ui") .product(name: "MarkdownUI", package: "swift-markdown-ui"),
.product(name: "cmark-gfm", package: "swift-cmark"),
.product(name: "cmark-gfm-extensions", package: "swift-cmark")
], ],
resources: [.process("Resources")]), resources: [.process("Resources")]),
.testTarget( .testTarget(
@@ -0,0 +1,87 @@
import Foundation
import MarkdownUI
import cmark_gfm
import cmark_gfm_extensions
/// Agent images come only from validated attachment descriptors. Rewrite parsed nodes,
/// including reference/nested images, so neither MarkdownUI image loader receives a URL.
enum AgentMessageMarkdown {
static func content(_ source: String) -> MarkdownContent {
cmark_gfm_core_extensions_ensure_registered()
guard let parser = cmark_parser_new(CMARK_OPT_DEFAULT) else { return literal(source) }
defer { cmark_parser_free(parser) }
for name in ["table", "strikethrough", "autolink", "tasklist"] {
if let ext = cmark_find_syntax_extension(name) { cmark_parser_attach_syntax_extension(parser, ext) }
}
source.withCString { cmark_parser_feed(parser, $0, source.utf8.count) }
guard let document = cmark_parser_finish(parser) else { return literal(source) }
defer { cmark_node_free(document) }
stripImagesAndHTML(document)
guard let markdown = cmark_render_commonmark(document, CMARK_OPT_DEFAULT, 0) else { return literal(source) }
defer { free(markdown) }
return MarkdownContent(String(cString: markdown))
}
private static func literal(_ source: String) -> MarkdownContent {
MarkdownContent { Paragraph { source } }
}
private static func stripImagesAndHTML(_ parent: UnsafeMutablePointer<cmark_node>) {
var child = cmark_node_first_child(parent)
while let node = child {
child = cmark_node_next(node)
let type = cmark_node_get_type(node)
if type == CMARK_NODE_IMAGE || type == CMARK_NODE_HTML_INLINE || type == CMARK_NODE_HTML_BLOCK {
let text = type == CMARK_NODE_IMAGE ? altText(node) : htmlText(cmark_node_get_literal(node).map { String(cString: $0) } ?? "")
let replacement = cmark_node_new(type == CMARK_NODE_HTML_BLOCK ? CMARK_NODE_PARAGRAPH : CMARK_NODE_TEXT)!
if type == CMARK_NODE_HTML_BLOCK {
let inline = cmark_node_new(CMARK_NODE_TEXT)!
// cmark includes the HTML block's terminating newline in its literal.
// It is a block delimiter, not an extra line in the visible alt text.
cmark_node_set_literal(inline, text.trimmingCharacters(in: .newlines))
cmark_node_append_child(replacement, inline)
} else {
cmark_node_set_literal(replacement, text)
}
cmark_node_replace(node, replacement)
cmark_node_free(node)
} else {
stripImagesAndHTML(node)
}
}
}
private static func altText(_ parent: UnsafeMutablePointer<cmark_node>) -> String {
var result = "", child = cmark_node_first_child(parent)
while let node = child {
let type = cmark_node_get_type(node)
if type == CMARK_NODE_TEXT || type == CMARK_NODE_CODE {
result += cmark_node_get_literal(node).map { String(cString: $0) } ?? ""
} else if type == CMARK_NODE_SOFTBREAK || type == CMARK_NODE_LINEBREAK {
result += " "
} else if type != CMARK_NODE_HTML_INLINE {
result += altText(node)
}
child = cmark_node_next(node)
}
return result
}
private static func htmlText(_ source: String) -> String {
// HTML is never rendered. Retain ordinary text and img alt attributes as text
// nodes; quoted '>' characters cannot expose the rest of an image attribute.
let tags = try! NSRegularExpression(pattern: #"(?s)<(?:[^>"']|"[^"]*"|'[^']*')*>"#)
let alt = try! NSRegularExpression(pattern: #"(?is)^<img\b.*?\s+alt\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s>]+))"#)
var result = source
for tag in tags.matches(in: source, range: NSRange(source.startIndex..., in: source)).reversed() {
let range = Range(tag.range, in: source)!
let value = String(source[range])
let match = alt.firstMatch(in: value, range: NSRange(value.startIndex..., in: value))
let label = match.flatMap { match in
(1...3).compactMap { Range(match.range(at: $0), in: value).map { String(value[$0]) } }.first
} ?? ""
result.replaceSubrange(Range(tag.range, in: result)!, with: label)
}
return result
}
}
@@ -46,7 +46,10 @@ actor SybilAPIClient: SybilAPIClienting, AgentAPIClienting, AgentPushAPIClientin
guard value.type == "persistent-agent", value.protocol == "pi-room.v1" else { guard value.type == "persistent-agent", value.protocol == "pi-room.v1" else {
throw AgentFailure(message: "Unsupported Agent room protocol.") throw AgentFailure(message: "Unsupported Agent room protocol.")
} }
if configuration.authToken?.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty != false { value.capabilities?.attachments = nil } if configuration.authToken?.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty != false {
value.capabilities?.attachments = nil
value.capabilities?.assistantImages = nil
}
return value return value
} }
@@ -174,9 +174,15 @@ struct AgentAttachmentCard: View {
if attachment.kind == .image { if attachment.kind == .image {
ZStack { ZStack {
SybilTheme.surface SybilTheme.surface
if let image { Image(uiImage: image).resizable().scaledToFit() } if let image {
Image(uiImage: image).resizable().scaledToFit()
.accessibilityLabel("Image: \(attachment.filename)")
.accessibilityIdentifier("agent-image-preview")
}
else { Image(systemName: "photo").foregroundStyle(SybilTheme.textMuted) } else { Image(systemName: "photo").foregroundStyle(SybilTheme.textMuted) }
}.frame(height: 180).clipped() }.frame(height: 180).clipped()
.accessibilityElement(children: .contain)
.accessibilityIdentifier("agent-image-frame")
} }
if let text { if let text {
Text(text).font(.system(.caption, design: .monospaced)).lineLimit(6) Text(text).font(.system(.caption, design: .monospaced)).lineLimit(6)
@@ -44,14 +44,24 @@ struct AgentAttachmentCapabilities: Codable, Equatable, Sendable {
} }
} }
struct AgentAssistantImageCapabilities: Decodable, Equatable, Sendable {
var version: Int
var supported: Bool { version == 1 }
}
struct AgentCapabilities: Decodable, Sendable { struct AgentCapabilities: Decodable, Sendable {
var attachments: AgentAttachmentCapabilities? var attachments: AgentAttachmentCapabilities?
init(attachments: AgentAttachmentCapabilities? = nil) { self.attachments = attachments } var assistantImages: AgentAssistantImageCapabilities?
private enum CodingKeys: String, CodingKey { case attachments } init(attachments: AgentAttachmentCapabilities? = nil, assistantImages: AgentAssistantImageCapabilities? = nil) {
self.attachments = attachments
self.assistantImages = assistantImages
}
private enum CodingKeys: String, CodingKey { case attachments, assistantImages }
init(from decoder: Decoder) throws { init(from decoder: Decoder) throws {
let values = try decoder.container(keyedBy: CodingKeys.self) let values = try decoder.container(keyedBy: CodingKeys.self)
// Unknown versions or malformed future capabilities must not break text chat. // Unknown versions or malformed future capabilities must not break text chat.
attachments = try? values.decode(AgentAttachmentCapabilities.self, forKey: .attachments) attachments = try? values.decode(AgentAttachmentCapabilities.self, forKey: .attachments)
assistantImages = try? values.decode(AgentAssistantImageCapabilities.self, forKey: .assistantImages)
} }
} }
@@ -91,13 +101,34 @@ struct AgentAttachment: Codable, Hashable, Identifiable, Sendable {
extension Message { extension Message {
var agentAttachments: [AgentAttachment] { var agentAttachments: [AgentAttachment] {
guard let entries = metadata?.objectValue?["agentAttachments"]?.arrayValue else { return [] } guard role == .user || role == .assistant,
return entries.compactMap { let info = metadata?.objectValue,
guard let data = try? JSONEncoder().encode($0), let entries = info["agentAttachments"]?.arrayValue else { return [] }
if role == .assistant {
guard info["kind"]?.stringValue == "message", info["status"]?.stringValue == "complete",
entries.count <= 8 else { return [] }
}
let attachments: [AgentAttachment] = entries.compactMap { entry in
// Inspect the original JSON before Codable can discard unknown fields. Output
// descriptors are deliberately path/URL-free; legacy user uploads stay compatible.
if role == .assistant {
let safeFields: Set<String> = ["id", "kind", "filename", "mimeType", "sizeBytes", "sha256"]
guard let fields = entry.objectValue, Set(fields.keys) == safeFields,
fields["kind"]?.stringValue == "image" else { return nil }
}
guard let data = try? JSONEncoder().encode(entry),
let attachment = try? JSONDecoder().decode(AgentAttachment.self, from: data), let attachment = try? JSONDecoder().decode(AgentAttachment.self, from: data),
(try? attachment.validate()) != nil else { return nil } (try? attachment.validate()) != nil else { return nil }
return attachment return attachment
} }
if role == .assistant {
// Reject the entire output set, including partially invalid arrays. The
// count and per-image bounds above also bound this aggregate calculation.
guard attachments.count == entries.count,
Set(attachments.map(\.id)).count == attachments.count,
attachments.reduce(0, { $0 + $1.sizeBytes }) <= 32 * 1024 * 1024 else { return [] }
}
return attachments
} }
} }
@@ -239,7 +239,7 @@ struct AgentPresentation {
private(set) var suppressed = false private(set) var suppressed = false
private(set) var busySince: Date? private(set) var busySince: Date?
private var seenTools: Set<String> = [] private var seenTools: Set<String> = []
static let quietTools: Set<String> = ["user_message", "notify_parent", "clock_wait", "event_decision"] static let quietTools: Set<String> = ["user_message", "output_image", "notify_parent", "clock_wait", "event_decision"]
mutating func receive(_ event: AgentEvent, before: AgentRoomState, after: AgentRoomState, replay: Bool, now: Date) { mutating func receive(_ event: AgentEvent, before: AgentRoomState, after: AgentRoomState, replay: Bool, now: Date) {
switch event { switch event {
@@ -249,7 +249,7 @@ struct AgentPresentation {
.sorted { AgentCursor.less($0.agentEntryCursor ?? "0", $1.agentEntryCursor ?? "0") } .sorted { AgentCursor.less($0.agentEntryCursor ?? "0", $1.agentEntryCursor ?? "0") }
if !replay { if !replay {
for message in fresh { for message in fresh {
if message.isAgentPublicReply && !message.content.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { suppressed = true } if message.isAgentPublicReply && (!message.content.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty || !message.agentAttachments.isEmpty) { suppressed = true }
else if message.role == .user { suppressed = false } else if message.role == .user { suppressed = false }
else if message.toolCallMetadata?.status == "initiated", else if message.toolCallMetadata?.status == "initiated",
!Self.quietTools.contains(message.toolCallMetadata?.toolName ?? ""), !seenTools.contains(message.agentToolID) { suppressed = false } !Self.quietTools.contains(message.toolCallMetadata?.toolName ?? ""), !seenTools.contains(message.agentToolID) { suppressed = false }
@@ -44,7 +44,7 @@ private struct AgentTranscriptView: View {
let displayed = transcript.displayed let displayed = transcript.displayed
ScrollViewReader { proxy in ScrollViewReader { proxy in
ScrollView { ScrollView {
LazyVStack(alignment: .leading, spacing: 15) { LazyVStack(alignment: .leading, spacing: 0) {
// The native scroll origin is the visual bottom. Keep model // The native scroll origin is the visual bottom. Keep model
// and cursor order ascending; reverse only presentation. // and cursor order ascending; reverse only presentation.
Color.clear.frame(height: 17).id(Self.bottomID) Color.clear.frame(height: 17).id(Self.bottomID)
@@ -56,18 +56,22 @@ private struct AgentTranscriptView: View {
} }
.foregroundStyle(SybilTheme.textMuted) .foregroundStyle(SybilTheme.textMuted)
.accessibilityIdentifier("agent-typing") .accessibilityIdentifier("agent-typing")
.padding(.bottom, 15)
.scaleEffect(x: 1, y: -1) .scaleEffect(x: 1, y: -1)
} }
ForEach(Array(displayed.enumerated().reversed()), id: \.element.id) { index, message in ForEach(Array(displayed.enumerated().reversed()), id: \.element.id) { index, message in
AgentTranscriptRow(message: message, room: room, AgentTranscriptRow(message: message, room: room,
startsDay: index == 0 || !Calendar.current.isDate(message.createdAt, inSameDayAs: displayed[index - 1].createdAt)) startsDay: index == 0 || !Calendar.current.isDate(message.createdAt, inSameDayAs: displayed[index - 1].createdAt))
.equatable() .equatable()
// Include the gap in the occupied-space entrance.
.padding(.bottom, 15)
.modifier(AgentEntrance(animate: appeared && rendered.entranceIDs.contains(message.id), reduceMotion: reduceMotion)) .modifier(AgentEntrance(animate: appeared && rendered.entranceIDs.contains(message.id), reduceMotion: reduceMotion))
.scaleEffect(x: 1, y: -1) .scaleEffect(x: 1, y: -1)
.id(message.id) .id(message.id)
} }
if room.isLoading { if room.isLoading {
ProgressView("Loading messages…").frame(maxWidth: .infinity) ProgressView("Loading messages…").frame(maxWidth: .infinity)
.padding(.bottom, 15)
.scaleEffect(x: 1, y: -1) .scaleEffect(x: 1, y: -1)
} }
if allMessages.count > displayed.count { if allMessages.count > displayed.count {
@@ -78,6 +82,7 @@ private struct AgentTranscriptView: View {
} }
.frame(maxWidth: .infinity) .frame(maxWidth: .infinity)
.padding(.vertical, 10) .padding(.vertical, 10)
.padding(.bottom, 15)
.scaleEffect(x: 1, y: -1) .scaleEffect(x: 1, y: -1)
} }
} }
@@ -104,28 +109,12 @@ private struct AgentTranscriptView: View {
AgentRenderDiagnostics.record("scrollGeometry") AgentRenderDiagnostics.record("scrollGeometry")
#endif #endif
scrollTracking.atBottom = atBottom scrollTracking.atBottom = atBottom
// Only user scrolling changes follow intent. Animated Latest // Only user scrolling changes follow intent. Send navigation
// navigation and native keyboard resizing must not cancel it. // and native keyboard resizing must not cancel it.
if isScrolling && scrollPhase != .animating { if isScrolling && scrollPhase != .animating {
atLatest = atBottom atLatest = atBottom
} }
} }
.overlay(alignment: .bottomTrailing) {
if !atLatest {
Button {
atLatest = true
withAnimation(reduceMotion ? nil : .easeOut(duration: 0.24)) { proxy.scrollTo(Self.bottomID, anchor: .top) }
} label: {
Label(room.unreadCount > 0 ? "\(room.unreadCount) unread" : "Latest", systemImage: "arrow.down")
.font(.sybil(.caption, weight: .semibold))
.padding(12)
.background(SybilTheme.surfaceStrong, in: Capsule())
.overlay(Capsule().stroke(SybilTheme.border))
}
.padding(14)
.accessibilityIdentifier("agent-jump-latest")
}
}
.onAppear { .onAppear {
appeared = true appeared = true
if !room.isLoading { if !room.isLoading {
@@ -286,7 +275,9 @@ private struct AgentMessageRow: View {
ForEach(message.agentAttachments) { attachment in ForEach(message.agentAttachments) { attachment in
AgentAttachmentCard(attachment: attachment, room: room) AgentAttachmentCard(attachment: attachment, room: room)
} }
MessageBubble(message: message, isSending: false, streamingAssistantPresentation: nil, isForking: false, onFork: nil, agentStyle: true) if !message.content.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty || message.agentAttachments.isEmpty {
MessageBubble(message: message, isSending: false, streamingAssistantPresentation: nil, isForking: false, onFork: nil, agentStyle: true)
}
HStack(spacing: 6) { HStack(spacing: 6) {
Text(own ? "You" : "Sybil") Text(own ? "You" : "Sybil")
Text(message.createdAt, style: .time) Text(message.createdAt, style: .time)
@@ -306,17 +297,37 @@ private struct AgentMessageRow: View {
} }
} }
// Keep the row's measured height stable for lazy layout. Animate its appearance // Grow the occupied space with the bubble's entrance so neighboring rows move
// without collapsing the scroll target while the composer or keyboard resizes. // smoothly. Measure at natural height; no geometry state or scroll correction.
private struct AgentEntranceLayout: Layout {
var progress: CGFloat
var animatableData: CGFloat {
get { progress }
set { progress = newValue }
}
func sizeThatFits(proposal: ProposedViewSize, subviews: Subviews, cache: inout ()) -> CGSize {
let size = subviews.first?.sizeThatFits(.init(width: proposal.width, height: nil)) ?? .zero
return CGSize(width: size.width, height: size.height * progress)
}
func placeSubviews(in bounds: CGRect, proposal: ProposedViewSize, subviews: Subviews, cache: inout ()) {
subviews.first?.place(at: bounds.origin, proposal: .init(width: bounds.width, height: nil))
}
}
private struct AgentEntrance: ViewModifier { private struct AgentEntrance: ViewModifier {
var animate: Bool var animate: Bool
var reduceMotion: Bool var reduceMotion: Bool
@State private var entered = false @State private var entered = false
func body(content: Content) -> some View { func body(content: Content) -> some View {
let preparing = animate && !entered && !reduceMotion let preparing = animate && !entered && !reduceMotion
content AgentEntranceLayout(progress: preparing ? 0 : 1) {
.opacity(preparing ? 0 : 1) content
.offset(y: preparing ? 16 : 0) .opacity(preparing ? 0 : 1)
.offset(y: preparing ? 16 : 0)
}
.clipped()
.onAppear { .onAppear {
withAnimation(animate && !reduceMotion ? .easeOut(duration: 0.24) : nil) { entered = true } withAnimation(animate && !reduceMotion ? .easeOut(duration: 0.24) : nil) { entered = true }
} }
@@ -223,14 +223,14 @@ struct MessageBubble: View {
) )
} }
if isStreamingAssistant { if isStreamingAssistant && !agentStyle {
StreamingAssistantContentView( StreamingAssistantContentView(
content: message.content, content: message.content,
isSending: isSending, isSending: isSending,
presentation: matchingStreamingAssistantPresentation presentation: matchingStreamingAssistantPresentation
) )
} else if !message.content.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { } else if !message.content.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty {
SybilMessageText(content: message.content, format: isUser ? .literal : .markdown) SybilMessageText(content: message.content, format: isUser ? .literal : .markdown, agentStyle: agentStyle)
} }
} }
.padding(.horizontal, isUser || agentStyle ? 14 : 2) .padding(.horizontal, isUser || agentStyle ? 14 : 2)
@@ -7,6 +7,7 @@ struct SybilMessageText: View {
enum Format { case literal, markdown } enum Format { case literal, markdown }
var content: String var content: String
var format: Format var format: Format
var agentStyle = false
static func literalContent(_ text: String) -> MarkdownContent { static func literalContent(_ text: String) -> MarkdownContent {
// Pasted CRLF/CR and blank lines containing spaces or tabs have the same // Pasted CRLF/CR and blank lines containing spaces or tabs have the same
@@ -31,7 +32,7 @@ struct SybilMessageText: View {
#if DEBUG #if DEBUG
let _ = AgentRenderDiagnostics.record("messageTextBuild") let _ = AgentRenderDiagnostics.record("messageTextBuild")
#endif #endif
Markdown(format == .literal ? Self.literalContent(content) : MarkdownContent(content)) Markdown(format == .literal ? Self.literalContent(content) : agentStyle ? AgentMessageMarkdown.content(content) : MarkdownContent(content))
.tint(SybilTheme.primary) .tint(SybilTheme.primary)
.foregroundStyle(format == .literal ? SybilTheme.text : SybilTheme.text.opacity(0.95)) .foregroundStyle(format == .literal ? SybilTheme.text : SybilTheme.text.opacity(0.95))
.markdownTheme(.sybilReadable) .markdownTheme(.sybilReadable)
@@ -14,10 +14,15 @@ private func sampleAttachment(_ id: String = "opaque-server-id", data: Data = or
AgentAttachment(id: id, kind: .file, filename: name, mimeType: "application/octet-stream", sizeBytes: data.count, AgentAttachment(id: id, kind: .file, filename: name, mimeType: "application/octet-stream", sizeBytes: data.count,
sha256: SHA256.hash(data: data).map { String(format: "%02x", $0) }.joined(), workspacePath: "/runtime/provided/\(name)") sha256: SHA256.hash(data: data).map { String(format: "%02x", $0) }.joined(), workspacePath: "/runtime/provided/\(name)")
} }
private func descriptorData(capability: Bool = true) throws -> Data { private func descriptorData(capability: Bool = true, assistantImages: Bool = false) throws -> Data {
var object: [String: Any] = ["id": "root", "title": "Sybil", "type": "persistent-agent", "model": "fixture", "protocol": "pi-room.v1", "websocket": "/v1/agent/ws", var object: [String: Any] = ["id": "root", "title": "Sybil", "type": "persistent-agent", "model": "fixture", "protocol": "pi-room.v1", "websocket": "/v1/agent/ws",
"activity": ["state": "idle", "isThinking": false, "queued": 0, "backgroundWorkers": 0, "tools": [], "queuedMessages": []]] "activity": ["state": "idle", "isThinking": false, "queued": 0, "backgroundWorkers": 0, "tools": [], "queuedMessages": []]]
if capability { object["capabilities"] = ["attachments": try JSONSerialization.jsonObject(with: JSONEncoder().encode(attachmentLimits)), "future": true] } if capability { object["capabilities"] = ["attachments": try JSONSerialization.jsonObject(with: JSONEncoder().encode(attachmentLimits)), "future": true] }
if assistantImages {
var capabilities = object["capabilities"] as? [String: Any] ?? [:]
capabilities["assistantImages"] = ["version": 1]
object["capabilities"] = capabilities
}
object["unknown"] = ["additive": true] object["unknown"] = ["additive": true]
return try JSONSerialization.data(withJSONObject: object) return try JSONSerialization.data(withJSONObject: object)
} }
@@ -124,6 +129,126 @@ private func requestBytes(_ request: URLRequest) -> Data {
} }
@Suite(.serialized) struct AgentAttachmentTests { @Suite(.serialized) struct AgentAttachmentTests {
@Test @MainActor func assistantCapabilityDoesNotAdmitInputAndRequiresToken() async throws {
AttachmentURLProtocol.install { request in
#expect(request.url!.path == "/api/v1/agent") // Never upload or submit references.
return (200, [:], try descriptorData(capability: false, assistantImages: true))
}
let session = attachmentSession(); defer { session.invalidateAndCancel() }
for token in [nil, " ", "synthetic-test"] {
let client = SybilAPIClient(configuration: .init(baseURL: URL(string: "https://fixture.invalid/api")!, authToken: token), session: session)
let descriptor = try await client.agentDescriptor()
#expect(descriptor.capabilities?.attachments == nil)
#expect((descriptor.capabilities?.assistantImages?.supported == true) == (token == "synthetic-test"))
let draft = AgentAttachmentDraft()
draft.configure(client: client, storage: nil); draft.updateCapability(descriptor.capabilities?.attachments)
#expect(!draft.supportsAttachments)
#expect(draft.add(filename: "original.bin") { .init(filename: "original.bin", mimeType: "application/octet-stream", data: originalBytes) } == nil)
await #expect(throws: AgentAttachmentIssue.self) { try await client.agentUpload(file: .init(filename: "original.bin", mimeType: "application/octet-stream", data: originalBytes), uploadID: UUID().uuidString) }
await #expect(throws: AgentAttachmentIssue.self) { try await client.agentSubmit(AgentSubmission(content: "", requestID: "output-only", attachments: [sampleAttachment()])) }
}
for json in [#"{"assistantImages":{"version":2}}"#, #"{"assistantImages":{"version":"future"}}"#] {
#expect(try JSONDecoder().decode(AgentCapabilities.self, from: Data(json.utf8)).assistantImages?.supported != true)
}
}
@Test func assistantDescriptorsRequireCompletedAssistantImageAndSafeFields() throws {
let image = AgentAttachment(id: "output-image", kind: .image, filename: "landscape.png", mimeType: "image/png", sizeBytes: 12, sha256: String(repeating: "a", count: 64))
var row = Message(id: "outbound:1", createdAt: Date(), role: .assistant, content: "", metadata: .object([
"kind": .string("message"), "status": .string("complete"), "agentAttachments": .array([image.metadataValue])]))
#expect(row.agentAttachments == [image])
for role in [MessageRole.system, .tool] {
var wrong = row; wrong.role = role
#expect(wrong.agentAttachments.isEmpty)
}
for (key, value) in [("kind", "tool_call"), ("kind", "other"), ("status", "queued"), ("status", "streaming")] {
var wrong = row, fields = row.metadata!.objectValue!
fields[key] = .string(value); wrong.metadata = .object(fields)
#expect(wrong.agentAttachments.isEmpty)
}
var descriptors: [JSONValue] = [sampleAttachment().metadataValue]
for key in ["workspacePath", "sourcePath", "privatePath", "url", "sourceURL", "downloadUrl", "external", "future"] {
var fields = image.metadataValue.objectValue!; fields[key] = .string("/private/synthetic.png")
descriptors.append(.object(fields))
}
for (key, value) in [("filename", "../private.png"), ("filename", "https://example.invalid/a.png"), ("mimeType", "image/svg+xml"), ("sha256", "bad")] {
var fields = image.metadataValue.objectValue!; fields[key] = .string(value)
descriptors.append(.object(fields))
}
for descriptor in descriptors {
row.metadata = .object(["kind": .string("message"), "status": .string("complete"), "agentAttachments": .array([descriptor])])
#expect(row.agentAttachments.isEmpty)
row.metadata = .object(["kind": .string("message"), "status": .string("complete"), "agentAttachments": .array([image.metadataValue, descriptor])])
#expect(row.agentAttachments.isEmpty) // No valid prefix from an invalid output set.
}
let maximum = (0..<8).map { index in
var item = image; item.id = "image-\(index)"; item.sizeBytes = 4 * 1024 * 1024
return item
}
row.metadata = .object(["kind": .string("message"), "status": .string("complete"), "agentAttachments": .array(maximum.map(\.metadataValue))])
#expect(row.agentAttachments == maximum) // Exactly eight IDs and 32 MiB.
let tooMany = (0..<9).map { index in
var item = image; item.id = "image-\(index)"
return item
}
var tooLarge = maximum; tooLarge[0].sizeBytes += 1
for invalid in [tooMany, [image, image], tooLarge] {
row.role = .assistant
row.metadata = .object(["kind": .string("message"), "status": .string("complete"), "agentAttachments": .array(invalid.map(\.metadataValue))])
#expect(row.agentAttachments.isEmpty)
row.role = .user
#expect(row.agentAttachments == invalid) // These new output limits do not change legacy input parsing.
}
row.role = .user
row.metadata = .object(["agentAttachments": .array([sampleAttachment().metadataValue])])
#expect(row.agentAttachments == [sampleAttachment()])
}
@Test @MainActor func imageOnlyAssistantLiveAndHistoryUseSameMergeAndAuthenticatedLoader() async throws {
let png = try #require(UIGraphicsImageRenderer(size: CGSize(width: 24, height: 16)).image { context in
UIColor.systemTeal.setFill(); context.fill(CGRect(x: 0, y: 0, width: 24, height: 16))
}.pngData())
let attachment = AgentAttachment(id: "output/opaque", kind: .image, filename: "landscape.png", mimeType: "image/png", sizeBytes: png.count,
sha256: SHA256.hash(data: png).map { String(format: "%02x", $0) }.joined())
let row = Message(id: "outbound:1", createdAt: Date(timeIntervalSince1970: 1), role: .assistant, content: "", metadata: .object([
"kind": .string("message"), "status": .string("complete"), "entryId": .number(1), "agentAttachments": .array([attachment.metadataValue])]))
let page = AgentPage(conversationId: "fixture-root", messages: [row], cursor: "1", hasMore: false)
let encoder = JSONEncoder(); encoder.dateEncodingStrategy = .iso8601
let bytes = try encoder.encode(page)
var envelope = try JSONSerialization.jsonObject(with: bytes) as! [String: Any]; envelope["type"] = "messages"
let decodedEvent = try AgentEvent.decode(JSONSerialization.data(withJSONObject: envelope))
let event = try #require(decodedEvent)
let room = SybilAgentRoom(); defer { room.stop() }
try room.receive(event); room.flush()
var history = AgentRoomState()
try history.apply(AgentCoding.decoder().decode(AgentPage.self, from: bytes))
// Replay must not duplicate a completed output; no current capability is needed.
try room.receive(event); room.flush()
#expect(AgentTranscript(messages: room.messages).displayed == AgentTranscript(messages: history.messages).displayed)
#expect(room.messages.count == 1); #expect(room.messages[0].content.isEmpty)
#expect(room.messages[0].agentAttachments == [attachment])
var presentation = AgentPresentation()
presentation.receive(event, before: AgentRoomState(), after: history, replay: false, now: Date())
#expect(presentation.suppressed)
AttachmentURLProtocol.install { request in
#expect(request.url!.absoluteString == "https://fixture.invalid/api/v1/agent/attachments/output%2Fopaque/content")
#expect(request.httpMethod == "GET")
#expect(request.value(forHTTPHeaderField: "Authorization") == "Bearer synthetic-test")
return (200, ["Content-Length": String(png.count)], png)
}
let session = attachmentSession(); defer { session.invalidateAndCancel() }
let client = SybilAPIClient(configuration: .init(baseURL: URL(string: "https://fixture.invalid/api")!, authToken: "synthetic-test"), session: session)
let downloads = AgentAttachmentDownloads()
#expect(try await downloads.data(for: attachment, client: client) == png)
AttachmentURLProtocol.install { _ in Issue.record("Verified output should be cached"); return (500, [:], Data()) }
#expect(try await downloads.data(for: attachment, client: client) == png)
AttachmentURLProtocol.install { _ in (200, [:], Data(repeating: 0, count: png.count)) }
await #expect(throws: AgentAttachmentIssue.self) { try await client.agentDownload(attachment) }
let unauthenticated = SybilAPIClient(configuration: .init(baseURL: URL(string: "https://fixture.invalid/api")!), session: session)
await #expect(throws: AgentAttachmentIssue.self) { try await unauthenticated.agentDownload(attachment) }
}
@Test func additiveCapabilityAndDistinctMetadata() throws { @Test func additiveCapabilityAndDistinctMetadata() throws {
let old = try AgentCoding.decoder().decode(AgentDescriptor.self, from: descriptorData(capability: false)) let old = try AgentCoding.decoder().decode(AgentDescriptor.self, from: descriptorData(capability: false))
#expect(old.capabilities == nil) #expect(old.capabilities == nil)
@@ -347,6 +347,33 @@ struct AgentTests {
#expect(presentation.isTyping(now: now.addingTimeInterval(7.3))) #expect(presentation.isTyping(now: now.addingTimeInterval(7.3)))
} }
@Test func outputImageActivityKeepsImageReplyTypingSuppressed() throws {
var presentation = AgentPresentation(), state = AgentRoomState()
let now = Date(timeIntervalSince1970: 100)
state.activity.isThinking = true
presentation.sent(now: now, activity: state.activity)
#expect(presentation.isTyping(now: now.addingTimeInterval(0.3)))
let image = AgentAttachment(id: "image", kind: .image, filename: "image.png", mimeType: "image/png", sizeBytes: 12, sha256: String(repeating: "a", count: 64))
var reply = agentMessage("image-reply", entry: "1", info: ["status": .string("complete"), "agentAttachments": .array([image.metadataValue])])
reply.content = ""
let page = agentPage([reply], cursor: "1")
var after = state; try after.apply(page)
presentation.receive(.messages(page), before: state, after: after, replay: false, now: now.addingTimeInterval(1))
state = after
#expect(presentation.suppressed)
after.activity.tools = [.init(id: "output-activity", name: "output_image", status: "initiated")]
presentation.receive(.activity("root", after.activity), before: state, after: after, replay: false, now: now.addingTimeInterval(2))
state = after
#expect(!presentation.isTyping(now: now.addingTimeInterval(3)))
let tool = agentPage([agentMessage("output-tool", entry: "2", role: .tool, info: ["toolName": .string("output_image"), "status": .string("initiated")])], cursor: "2")
try after.apply(tool)
presentation.receive(.messages(tool), before: state, after: after, replay: false, now: now.addingTimeInterval(4))
#expect(presentation.suppressed)
#expect(!presentation.isTyping(now: now.addingTimeInterval(5)))
}
@Test func socketURLAndAuthentication() throws { @Test func socketURLAndAuthentication() throws {
let config = APIConfiguration(baseURL: URL(string: "https://sybil.bajor.cloud/api/?accidental=query")!, authToken: " token ") let config = APIConfiguration(baseURL: URL(string: "https://sybil.bajor.cloud/api/?accidental=query")!, authToken: " token ")
#expect(try config.agentWebSocketURL().absoluteString == "wss://sybil.bajor.cloud/api/v1/agent/ws") #expect(try config.agentWebSocketURL().absoluteString == "wss://sybil.bajor.cloud/api/v1/agent/ws")
@@ -6,6 +6,57 @@ import UIKit
@Suite @MainActor @Suite @MainActor
struct MessagePresentationTests { struct MessagePresentationTests {
@Test func agentMarkdownKeepsAltAndFormattingWithoutImageOrHTMLNodes() {
let source = """
**Caption** ![Inline *landscape*](https://images.invalid/inline.png)
![Reference landscape][photo]
[![Linked landscape](//images.invalid/linked.png)](https://example.invalid)
> ![Nested ![inner](file:///private/image.png)](data:image/png;base64,AAAA)
<picture><source srcset="https://images.invalid/source.png"><img src="https://images.invalid/html.png" alt="HTML landscape" title="a > b"></picture>
[Useful link](https://example.invalid)
[photo]: https://images.invalid/reference.png
"""
let content = AgentMessageMarkdown.content(source)
let html = content.renderHTML(), plain = content.renderPlainText()
for label in ["Inline landscape", "Reference landscape", "Linked landscape", "Nested inner", "HTML landscape"] {
#expect(plain.contains(label))
}
#expect(!html.contains("<img")); #expect(!html.contains("<picture")); #expect(!html.contains("<source"))
#expect(!html.contains("images.invalid")); #expect(!html.contains("/private/"))
#expect(html.contains("<strong>Caption</strong>"))
#expect(html.contains(#"href="https://example.invalid""#))
// Normal chat still uses unmodified MarkdownUI content.
#expect(MarkdownContent(source).renderHTML().contains("<img"))
#expect(AgentMessageMarkdown.content("<img src='https://images.invalid/a.png' alt='HTML landscape'>\n").renderHTML() == "<p>HTML landscape</p>\n")
}
@Test func agentMarkdownRetainsCodeTablesAndEscapesImageLikeAltText() {
let source = """
`![literal](https://images.invalid/code.png)`
```html
<img src="literal-code">
```
| Image | Value |
| --- | --- |
| ![Table alt](https://images.invalid/table.png) | **Kept** |
<IMG ALT='![literal alt](https://images.invalid/alt.png)' SRC='ignored'>
"""
let html = AgentMessageMarkdown.content(source).renderHTML()
#expect(html.contains("<table>")); #expect(html.contains("Table alt"))
#expect(html.contains("<strong>Kept</strong>"))
#expect(html.contains("&lt;img src=")); #expect(html.contains("![literal]"))
#expect(html.contains("![literal alt]")); #expect(!html.contains("<img"))
}
@Test func userMarkupRemainsLiteral() { @Test func userMarkupRemainsLiteral() {
let source = "**bold** [link](https://example.invalid) `code`\n- item\n<script>literal</script>" let source = "**bold** [link](https://example.invalid) `code`\n- item\n<script>literal</script>"
let content = SybilMessageText.literalContent(source) let content = SybilMessageText.literalContent(source)
+20
View File
@@ -3,6 +3,7 @@
Run with Python + aiohttp, then xcodebuild test -scheme SybilAgentUI. Run with Python + aiohttp, then xcodebuild test -scheme SybilAgentUI.
""" """
import asyncio import asyncio
import base64
import datetime import datetime
import json import json
import hashlib import hashlib
@@ -23,6 +24,8 @@ reply_tasks = set()
attachments = {} attachments = {}
uploads = {} uploads = {}
attachment_capability = True attachment_capability = True
assistant_image_capability = False
fail_image_download = False
next_upload_error = None next_upload_error = None
blocked_proxy = False blocked_proxy = False
transcript_comparison = False transcript_comparison = False
@@ -95,6 +98,7 @@ async def reply(content):
async def route(request): async def route(request):
global read_cursor, cursor, busy, attachment_capability, next_upload_error, blocked_proxy, transcript_comparison, push_available, auth_required global read_cursor, cursor, busy, attachment_capability, next_upload_error, blocked_proxy, transcript_comparison, push_available, auth_required
global assistant_image_capability, fail_image_download
path = request.path path = request.path
log.append(dict(method=request.method, path=path, query=dict(request.query))) log.append(dict(method=request.method, path=path, query=dict(request.query)))
if path == '/control': if path == '/control':
@@ -106,6 +110,7 @@ async def route(request):
reply_tasks.clear() reply_tasks.clear()
messages.clear(); requests.clear(); attempts.clear(); cursor = 0; read_cursor = 0; busy = False messages.clear(); requests.clear(); attempts.clear(); cursor = 0; read_cursor = 0; busy = False
attachment_capability = True; next_upload_error = None; blocked_proxy = False; transcript_comparison = False attachment_capability = True; next_upload_error = None; blocked_proxy = False; transcript_comparison = False
assistant_image_capability = False; fail_image_download = False
push_installations.clear(); push_retired.clear(); push_requests.clear(); log.clear() push_installations.clear(); push_retired.clear(); push_requests.clear(); log.clear()
push_available = True; auth_required = False push_available = True; auth_required = False
for i in range(body.get('count', 3)): for i in range(body.get('count', 3)):
@@ -120,6 +125,16 @@ async def route(request):
transcript_comparison = True transcript_comparison = True
elif action == 'capability': elif action == 'capability':
attachment_capability = body.get('enabled', True) attachment_capability = body.get('enabled', True)
assistant_image_capability = body.get('assistantImages', False)
elif action == 'assistant-image':
# Synthetic output supplied by XCTest, never a workspace file or provider.
data = base64.b64decode(body['png'], validate=True)
descriptor = dict(id='fixture-output-image', kind='image', filename='fixture-landscape.png',
mimeType='image/png', sizeBytes=len(data), sha256=hashlib.sha256(data).hexdigest())
attachments[descriptor['id']] = dict(descriptor=descriptor, data=data)
fail_image_download = body.get('failFirstDownload', False)
row = message(body.get('content', ''), info={'agentAttachments': [descriptor]})
await broadcast(dict(type='messages', **page([row])))
elif action == 'upload-error': elif action == 'upload-error':
next_upload_error = body next_upload_error = body
elif action == 'blocked-proxy': elif action == 'blocked-proxy':
@@ -190,6 +205,8 @@ async def route(request):
descriptor = dict(id='fixture-root', title='Sybil Agent', type='persistent-agent', model='fixture', protocol='pi-room.v1', websocket='/v1/agent/ws', activity=activity()) descriptor = dict(id='fixture-root', title='Sybil Agent', type='persistent-agent', model='fixture', protocol='pi-room.v1', websocket='/v1/agent/ws', activity=activity())
if attachment_capability: if attachment_capability:
descriptor['capabilities'] = {'attachments': CAPABILITY} descriptor['capabilities'] = {'attachments': CAPABILITY}
if assistant_image_capability:
descriptor.setdefault('capabilities', {})['assistantImages'] = dict(version=1)
return web.json_response(descriptor) return web.json_response(descriptor)
if path == '/api/v1/agent/push/config': if path == '/api/v1/agent/push/config':
if not push_available: if not push_available:
@@ -263,6 +280,9 @@ async def route(request):
uploads[upload_id] = (identity, attachment_id) uploads[upload_id] = (identity, attachment_id)
return web.json_response({'attachment': descriptor}, status=201) return web.json_response({'attachment': descriptor}, status=201)
if path.startswith('/api/v1/agent/attachments/') and path.endswith('/content'): if path.startswith('/api/v1/agent/attachments/') and path.endswith('/content'):
if fail_image_download:
fail_image_download = False
return attachment_error(503, 'fixture_retry', 'Synthetic image unavailable. Retry the download.')
attachment_id = path.removeprefix('/api/v1/agent/attachments/').removesuffix('/content') attachment_id = path.removeprefix('/api/v1/agent/attachments/').removesuffix('/content')
stored = attachments.get(attachment_id) stored = attachments.get(attachment_id)
if stored is None: if stored is None:
@@ -7,6 +7,51 @@ import SwiftUI
/// not GPU presentation, touch latency, or physical-device frame rate. /// not GPU presentation, touch latency, or physical-device frame rate.
@MainActor @MainActor
final class AgentPerformanceTests: XCTestCase { final class AgentPerformanceTests: XCTestCase {
func testEntranceGrowsOccupiedHeight() async throws {
XCTAssertFalse(UIAccessibility.isReduceMotionEnabled, "This check must exercise the entrance animation")
let room = SybilAgentRoom()
let rows = (1...3).map { index in
Message(id: "entrance-\(index)", createdAt: Date(timeIntervalSince1970: 1_700_000_000),
role: .assistant, content: "Existing neighbor \(index)")
}
try room.receive(.messages(AgentPage(conversationId: "entrance", messages: rows,
cursor: "3", beforeCursor: nil, hasMore: false)), replay: true)
room.flush()
let host = UIHostingController(rootView: SybilAgentView(room: room))
let scene = try XCTUnwrap(UIApplication.shared.connectedScenes.compactMap { $0 as? UIWindowScene }.first)
let window = UIWindow(windowScene: scene)
window.rootViewController = host
window.makeKeyAndVisible()
defer { window.isHidden = true; window.rootViewController = nil; room.stop() }
host.view.frame = window.bounds
host.view.layoutIfNeeded()
try await Task.sleep(for: .milliseconds(500))
func transcript(in view: UIView) -> UIScrollView? {
if let scroll = view as? UIScrollView, scroll.contentSize.height > 100 { return scroll }
return view.subviews.lazy.compactMap { transcript(in: $0) }.first
}
let scroll = try XCTUnwrap(transcript(in: host.view))
let before = scroll.contentSize.height
let reply = Message(id: "entrance-4", createdAt: rows[0].createdAt, role: .assistant,
content: "Incoming reply\n\nIts occupied space should grow with the bubble.")
try room.receive(.messages(AgentPage(conversationId: "entrance", messages: [reply],
cursor: "4", beforeCursor: nil, hasMore: false)))
room.flush()
var heights: [CGFloat] = []
for _ in 0..<32 {
try await Task.sleep(for: .milliseconds(16))
heights.append(scroll.contentSize.height)
}
let after = try XCTUnwrap(heights.last)
// All four short rows fit onscreen, so these are measured heights, not
// estimates for unmaterialized lazy history. No frame-rate claim.
XCTAssertLessThan(after, scroll.bounds.height)
XCTAssertGreaterThan(after - before, 50)
let intermediate = Set(heights.filter { $0 > before + 2 && $0 < after - 2 }.map { Int($0.rounded()) })
XCTAssertGreaterThanOrEqual(intermediate.count, 3, "Neighbors must get intermediate layout heights, not a full-height insertion")
print("AGENT_ENTRANCE_HEIGHT before=\(before) samples=\(heights)")
}
func testThinkingScrollResponsiveness() async throws { func testThinkingScrollResponsiveness() async throws {
let room = SybilAgentRoom() let room = SybilAgentRoom()
var rows: [Message] = [] var rows: [Message] = []
+71 -4
View File
@@ -65,6 +65,59 @@ final class AgentUITests: XCTestCase {
return try XCTUnwrap(json["attachment"] as? [String: Any]) return try XCTUnwrap(json["attachment"] as? [String: Any])
} }
func testAssistantImageOnlyLiveRetryAndHistoryWithoutInputCapability() async throws {
app.terminate()
try await control(["action": "reset", "count": 0])
try await control(["action": "capability", "enabled": false, "assistantImages": true])
try await control(["action": "push-available", "enabled": false])
app.launch(); openAgent()
XCTAssertFalse(app.buttons["agent-attach"].exists)
let image = UIGraphicsImageRenderer(size: CGSize(width: 320, height: 180)).image { context in
UIColor.systemTeal.setFill(); context.fill(CGRect(x: 0, y: 0, width: 320, height: 180))
UIColor.systemYellow.setFill(); context.cgContext.fillEllipse(in: CGRect(x: 225, y: 25, width: 50, height: 50))
UIColor.systemIndigo.setFill()
context.cgContext.move(to: CGPoint(x: 0, y: 180)); context.cgContext.addLine(to: CGPoint(x: 120, y: 55))
context.cgContext.addLine(to: CGPoint(x: 250, y: 180)); context.cgContext.fillPath()
}
try await control(["action": "reply", "content": "![Markdown landscape](http://127.0.0.1:8799/blocked-markdown.png)\n\n<img src='http://127.0.0.1:8799/blocked-html.png' alt='HTML landscape'>"])
try await control(["action": "assistant-image", "content": "", "png": try XCTUnwrap(image.pngData()).base64EncodedString(), "failFirstDownload": true])
XCTAssertTrue(app.staticTexts["fixture-landscape.png"].waitForExistence(timeout: 8))
XCTAssertTrue(app.buttons["Retry download"].waitForExistence(timeout: 8))
let imageFrame = app.otherElements["agent-image-frame"]
XCTAssertTrue(imageFrame.exists)
let placeholder = imageFrame.frame
XCTAssertGreaterThan(placeholder.width, 0)
XCTAssertEqual(placeholder.height, 180, accuracy: 1)
app.buttons["Retry download"].tap()
let preview = app.images["agent-image-preview"]
XCTAssertTrue(preview.waitForExistence(timeout: 8))
XCTAssertEqual(preview.label, "Image: fixture-landscape.png")
XCTAssertLessThanOrEqual(preview.frame.height, 180)
XCTAssertEqual(imageFrame.frame.width, placeholder.width, accuracy: 1)
XCTAssertEqual(imageFrame.frame.height, placeholder.height, accuracy: 1)
let loadedSize = imageFrame.frame.size
XCTAssertTrue(app.staticTexts["Markdown landscape"].exists)
XCTAssertTrue(app.staticTexts["HTML landscape"].exists)
XCTAssertFalse(app.buttons.matching(NSPredicate(format: "label BEGINSWITH %@", "Copy workspace path:")).firstMatch.exists)
capture("agent-assistant-image-live")
// Previously saved output remains visible when both capabilities disappear.
try await control(["action": "capability", "enabled": false, "assistantImages": false])
app.terminate(); app.launch(); openAgent()
XCTAssertTrue(preview.waitForExistence(timeout: 8))
XCTAssertEqual(imageFrame.frame.width, placeholder.width, accuracy: 1)
XCTAssertEqual(imageFrame.frame.height, placeholder.height, accuracy: 1)
print("Assistant image frame sizes: placeholder=\(placeholder.size), live=\(loadedSize), history=\(imageFrame.frame.size)")
XCTAssertEqual(app.staticTexts.matching(NSPredicate(format: "label == %@", "fixture-landscape.png")).count, 1)
XCTAssertFalse(app.buttons["agent-attach"].exists)
capture("agent-assistant-image-history")
let state = try await control(["action": "state"])
let log = try XCTUnwrap(state["log"] as? [[String: Any]])
XCTAssertFalse(log.contains { ($0["path"] as? String)?.hasPrefix("/blocked-") == true })
XCTAssertGreaterThanOrEqual(log.filter { ($0["path"] as? String) == "/api/v1/agent/attachments/fixture-output-image/content" }.count, 3)
XCTAssertTrue((state["requests"] as? [String: Any])?.isEmpty == true)
}
func testAttachmentHistoryDownloadAndReload() async throws { func testAttachmentHistoryDownloadAndReload() async throws {
let original = Data("Synthetic original text. <script>plain text only</script>".utf8) let original = Data("Synthetic original text. <script>plain text only</script>".utf8)
let file = try await uploadOriginal(name: "fixture-original.txt", mime: "text/plain", bytes: original) let file = try await uploadOriginal(name: "fixture-original.txt", mime: "text/plain", bytes: original)
@@ -313,7 +366,8 @@ final class AgentUITests: XCTestCase {
XCTAssertTrue(app.staticTexts["History message 160: local simulator fixture."].isHittable) XCTAssertTrue(app.staticTexts["History message 160: local simulator fixture."].isHittable)
scroll.swipeDown() scroll.swipeDown()
scroll.swipeDown() scroll.swipeDown()
XCTAssertTrue(app.buttons["agent-jump-latest"].waitForExistence(timeout: 3)) XCTAssertFalse(app.staticTexts["History message 160: local simulator fixture."].isHittable)
XCTAssertFalse(app.buttons["agent-jump-latest"].exists)
let anchor = try XCTUnwrap(app.staticTexts.matching(NSPredicate(format: "label BEGINSWITH %@", "History message")).allElementsBoundByIndex.first { $0.isHittable }) let anchor = try XCTUnwrap(app.staticTexts.matching(NSPredicate(format: "label BEGINSWITH %@", "History message")).allElementsBoundByIndex.first { $0.isHittable })
let before = anchor.frame.minY let before = anchor.frame.minY
try await control(["action": "reply", "content": "Reply while reading earlier history"]) try await control(["action": "reply", "content": "Reply while reading earlier history"])
@@ -327,14 +381,27 @@ final class AgentUITests: XCTestCase {
XCTAssertTrue(app.buttons["Load earlier messages"].isHittable) XCTAssertTrue(app.buttons["Load earlier messages"].isHittable)
app.buttons["Load earlier messages"].tap() app.buttons["Load earlier messages"].tap()
// Loading older lazy history may move the viewport. It must still load // Loading older lazy history may move the viewport. It must still load
// the remaining history and preserve Latest/keyboard behavior below. // the remaining history and preserve scroll-to-latest/keyboard behavior below.
XCTAssertFalse(app.buttons["Load earlier messages"].exists) XCTAssertFalse(app.buttons["Load earlier messages"].exists)
capture("agent-older-history-loaded") capture("agent-older-history-loaded")
app.buttons["agent-jump-latest"].tap() let latest = app.staticTexts["Reply while reading earlier history"]
XCTAssertTrue(app.staticTexts["Reply while reading earlier history"].isHittable) for _ in 0..<30 {
if latest.isHittable { break }
scroll.swipeUp(velocity: .fast)
}
scroll.swipeUp(velocity: .slow)
XCTAssertTrue(latest.isHittable)
composer.tap(); composer.typeText("A multiline draft\nwith a second line\nand a third line") composer.tap(); composer.typeText("A multiline draft\nwith a second line\nand a third line")
XCTAssertTrue(app.staticTexts["Reply while reading earlier history"].isHittable) XCTAssertTrue(app.staticTexts["Reply while reading earlier history"].isHittable)
capture("agent-multiline-keyboard") capture("agent-multiline-keyboard")
// Sending remains the explicit navigation path while reading history.
scroll.swipeDown(velocity: .slow)
scroll.swipeDown(velocity: .slow)
XCTAssertFalse(latest.isHittable)
app.buttons["agent-send"].tap()
let sent = app.staticTexts["A multiline draft\nwith a second line\nand a third line"]
let followed = XCTNSPredicateExpectation(predicate: NSPredicate(format: "hittable == true"), object: sent)
await fulfillment(of: [followed], timeout: 5)
} }
func testBackgroundUnreadReadReconciliationAndReplay() async throws { func testBackgroundUnreadReadReconciliationAndReplay() async throws {
@@ -41,6 +41,17 @@ final class TranscriptFadeUITests: XCTestCase {
app.scrollViews["agent-transcript"] app.scrollViews["agent-transcript"]
} }
private func scrollToLatest(_ latest: XCUIElement) {
for _ in 0..<30 {
if latest.isHittable { break }
transcript.swipeUp(velocity: .fast)
}
// Finish at the physical end, including the reply metadata and spacing.
transcript.swipeUp(velocity: .slow)
XCTAssertTrue(latest.isHittable)
XCTAssertFalse(app.buttons["agent-jump-latest"].exists)
}
private func capture(_ name: String) { private func capture(_ name: String) {
let attachment = XCTAttachment(screenshot: XCUIScreen.main.screenshot()) let attachment = XCTAttachment(screenshot: XCUIScreen.main.screenshot())
attachment.name = name; attachment.lifetime = .keepAlways; add(attachment) attachment.name = name; attachment.lifetime = .keepAlways; add(attachment)
@@ -146,7 +157,7 @@ final class TranscriptFadeUITests: XCTestCase {
let scroll = transcript let scroll = transcript
XCTAssertTrue(scroll.exists) XCTAssertTrue(scroll.exists)
scroll.swipeDown(velocity: .slow) scroll.swipeDown(velocity: .slow)
XCTAssertTrue(app.buttons["agent-jump-latest"].waitForExistence(timeout: 5)) XCTAssertFalse(app.staticTexts["History message 160: local simulator fixture."].isHittable)
capture("agent-content-crossing-shared-fade") capture("agent-content-crossing-shared-fade")
XCTAssertTrue(composer.isHittable) XCTAssertTrue(composer.isHittable)
XCTAssertTrue(app.buttons["agent-attach"].isHittable) XCTAssertTrue(app.buttons["agent-attach"].isHittable)
@@ -162,7 +173,8 @@ final class TranscriptFadeUITests: XCTestCase {
XCTAssertTrue(app.staticTexts["History message 160: local simulator fixture."].isHittable) XCTAssertTrue(app.staticTexts["History message 160: local simulator fixture."].isHittable)
scroll.swipeDown(velocity: .slow) scroll.swipeDown(velocity: .slow)
scroll.swipeDown(velocity: .slow) scroll.swipeDown(velocity: .slow)
XCTAssertTrue(app.buttons["agent-jump-latest"].waitForExistence(timeout: 5)) XCTAssertFalse(app.staticTexts["History message 160: local simulator fixture."].isHittable)
XCTAssertFalse(app.buttons["agent-jump-latest"].exists)
// Stay below the header's content clearance, where text is readable. // Stay below the header's content clearance, where text is readable.
let visibleTop = scroll.frame.minY + 100 let visibleTop = scroll.frame.minY + 100
let anchor = try XCTUnwrap(app.staticTexts.matching(NSPredicate(format: "label BEGINSWITH %@", "History message")).allElementsBoundByIndex.first { let anchor = try XCTUnwrap(app.staticTexts.matching(NSPredicate(format: "label BEGINSWITH %@", "History message")).allElementsBoundByIndex.first {
@@ -171,8 +183,9 @@ final class TranscriptFadeUITests: XCTestCase {
let before = anchor.frame.minY let before = anchor.frame.minY
let stateBefore = try await control(["action": "state"]) let stateBefore = try await control(["action": "state"])
try await control(["action": "reply", "content": "Fade regression reply while reading"]) try await control(["action": "reply", "content": "Fade regression reply while reading"])
let unread = XCTNSPredicateExpectation(predicate: NSPredicate(format: "label CONTAINS %@", "unread"), object: app.buttons["agent-jump-latest"]) // The removed button no longer exposes unread state. Allow the entrance
await fulfillment(of: [unread], timeout: 8) // and read debounce to settle, then inspect the fixture's actual cursor.
try await Task.sleep(for: .milliseconds(500))
XCTAssertEqual(anchor.frame.minY, before, accuracy: 2) XCTAssertEqual(anchor.frame.minY, before, accuracy: 2)
let stateAfter = try await control(["action": "state"]) let stateAfter = try await control(["action": "state"])
XCTAssertEqual(stateAfter["readCursor"] as? Int, stateBefore["readCursor"] as? Int) XCTAssertEqual(stateAfter["readCursor"] as? Int, stateBefore["readCursor"] as? Int)
@@ -191,8 +204,8 @@ final class TranscriptFadeUITests: XCTestCase {
XCTAssertFalse(load.exists) XCTAssertFalse(load.exists)
capture("agent-fade-older-history-loaded") capture("agent-fade-older-history-loaded")
app.buttons["agent-jump-latest"].tap()
let latest = app.staticTexts["Fade regression reply while reading"] let latest = app.staticTexts["Fade regression reply while reading"]
scrollToLatest(latest)
let followed = XCTNSPredicateExpectation(predicate: NSPredicate(format: "hittable == true"), object: latest) let followed = XCTNSPredicateExpectation(predicate: NSPredicate(format: "hittable == true"), object: latest)
await fulfillment(of: [followed], timeout: 5) await fulfillment(of: [followed], timeout: 5)
composer.tap() composer.tap()
@@ -217,14 +230,14 @@ final class TranscriptFadeUITests: XCTestCase {
let keyboardRead = try await control(["action": "state"]) let keyboardRead = try await control(["action": "state"])
XCTAssertEqual(keyboardRead["readCursor"] as? Int, keyboardRead["cursor"] as? Int) XCTAssertEqual(keyboardRead["readCursor"] as? Int, keyboardRead["cursor"] as? Int)
// Interactive dismissal is user navigation; Latest must still return to // Interactive dismissal is user navigation; scrolling must still return
// the bottom, and presenting the keyboard again must keep it visible. // to the bottom, and presenting the keyboard again must keep it visible.
let dismissStart = scroll.coordinate(withNormalizedOffset: CGVector(dx: 0.5, dy: 0.9)) let dismissStart = scroll.coordinate(withNormalizedOffset: CGVector(dx: 0.5, dy: 0.9))
let dismissEnd = app.coordinate(withNormalizedOffset: CGVector(dx: 0.5, dy: 0.98)) let dismissEnd = app.coordinate(withNormalizedOffset: CGVector(dx: 0.5, dy: 0.98))
dismissStart.press(forDuration: 0.1, thenDragTo: dismissEnd) dismissStart.press(forDuration: 0.1, thenDragTo: dismissEnd)
let dismissed = XCTNSPredicateExpectation(predicate: NSPredicate(format: "exists == false"), object: app.keyboards.firstMatch) let dismissed = XCTNSPredicateExpectation(predicate: NSPredicate(format: "exists == false"), object: app.keyboards.firstMatch)
await fulfillment(of: [dismissed], timeout: 5) await fulfillment(of: [dismissed], timeout: 5)
if app.buttons["agent-jump-latest"].exists { app.buttons["agent-jump-latest"].tap() } scrollToLatest(keyboardReply)
XCTAssertTrue(keyboardReply.isHittable) XCTAssertTrue(keyboardReply.isHittable)
composer.tap() composer.tap()
XCTAssertTrue(app.keyboards.firstMatch.waitForExistence(timeout: 5)) XCTAssertTrue(app.keyboards.firstMatch.waitForExistence(timeout: 5))
+5 -4
View File
@@ -18,11 +18,12 @@ export type AgentProxyJson = (path: string, method?: string, body?: unknown, sig
function apiError(statusCode: number, code: string, message: string) { function apiError(statusCode: number, code: string, message: string) {
return Object.assign(new Error(message), { statusCode, code }); return Object.assign(new Error(message), { statusCode, code });
} }
export async function requireAgentAttachments(proxyJson: AgentProxyJson, signal?: AbortSignal) { export async function requireAgentAttachments(proxyJson: AgentProxyJson, signal?: AbortSignal, download = false) {
if (!env.ADMIN_TOKEN) throw apiError(503, 'attachments_unavailable', 'Attachments require authenticated Sybil access.'); if (!env.ADMIN_TOKEN) throw apiError(503, 'attachments_unavailable', 'Attachments require authenticated Sybil access.');
const descriptor = await proxyJson('/api/conversations/root', 'GET', undefined, signal); const descriptor = await proxyJson('/api/conversations/root', 'GET', undefined, signal);
const value = descriptor.data as { capabilities?: { attachments?: { version?: number } } } | null; const value = descriptor.data as { capabilities?: { attachments?: { version?: number }; assistantImages?: { version?: number } } } | null;
if (descriptor.status !== 200 || value?.capabilities?.attachments?.version !== 1) if (descriptor.status !== 200 || !(value?.capabilities?.attachments?.version === 1
|| (download && value?.capabilities?.assistantImages?.version === 1)))
throw apiError(503, 'attachments_unavailable', 'The Agent does not currently support authenticated attachments.'); throw apiError(503, 'attachments_unavailable', 'The Agent does not currently support authenticated attachments.');
} }
@@ -52,7 +53,7 @@ export async function registerAgentAttachmentRoutes(app: FastifyInstance, option
if (req.raw.aborted || reply.raw.destroyed) { release(); controller.signal.throwIfAborted(); } if (req.raw.aborted || reply.raw.destroyed) { release(); controller.signal.throwIfAborted(); }
if (uploading && Number(req.headers['content-length'] || 0) > AGENT_UPLOAD_LIMIT) if (uploading && Number(req.headers['content-length'] || 0) > AGENT_UPLOAD_LIMIT)
throw apiError(413, 'attachment_limit_exceeded', 'Upload request exceeds its size limit.'); throw apiError(413, 'attachment_limit_exceeded', 'Upload request exceeds its size limit.');
await requireAgentAttachments(options.proxyJson, controller.signal); await requireAgentAttachments(options.proxyJson, controller.signal, !uploading);
controller.signal.throwIfAborted(); controller.signal.throwIfAborted();
}); });
routes.addHook('onResponse', async req => { transfers.get(req)?.release(); }); routes.addHook('onResponse', async req => { transfers.get(req)?.release(); });
+6 -2
View File
@@ -14,7 +14,8 @@ export type MessageText = (message: PublicMessage) => Promise<string>;
const Page = z.object({ conversationId: z.string(), cursor: Cursor, beforeCursor: Cursor, const Page = z.object({ conversationId: z.string(), cursor: Cursor, beforeCursor: Cursor,
messages: z.array(z.unknown()).max(1000) }); messages: z.array(z.unknown()).max(1000) });
const Message = z.object({ id: z.string(), role: z.literal('assistant'), content: z.string(), const Message = z.object({ id: z.string(), role: z.literal('assistant'), content: z.string(),
metadata: z.object({ kind: z.literal('message'), status: z.literal('complete'), entryId: z.number().int().safe() }) }); metadata: z.object({ kind: z.literal('message'), status: z.literal('complete'), entryId: z.number().int().safe(),
agentAttachments: z.unknown().optional() }) });
/** Resolve only an already eligible candidate from one immutable root public entry. */ /** Resolve only an already eligible candidate from one immutable root public entry. */
export function createMessageText(upstream: { baseUrl: string; apiKey?: string }): MessageText { export function createMessageText(upstream: { baseUrl: string; apiKey?: string }): MessageText {
@@ -47,7 +48,10 @@ export function createMessageText(upstream: { baseUrl: string; apiKey?: string }
const matches = page.messages.filter(value => value && typeof value === 'object' && 'id' in value && value.id === candidate.messageId); const matches = page.messages.filter(value => value && typeof value === 'object' && 'id' in value && value.id === candidate.messageId);
const message = matches.length === 1 ? Message.safeParse(matches[0]) : undefined; const message = matches.length === 1 ? Message.safeParse(matches[0]) : undefined;
if (!message?.success || message.data.metadata.entryId !== Number(candidate.cursor)) throw new Error('Public Agent message unavailable'); if (!message?.success || message.data.metadata.entryId !== Number(candidate.cursor)) throw new Error('Public Agent message unavailable');
// Never inspect metadata, tool results, activity, reasoning blocks or linked resources for text. // Attachment messages use the generic alert. Never expose captions, filenames,
// paths, image data or linked resources in an image notification.
if (message.data.metadata.agentAttachments !== undefined) return '';
// Never inspect tool results, activity, reasoning blocks or linked resources for text.
return message.data.content; return message.data.content;
}; };
} }
+3 -2
View File
@@ -43,8 +43,8 @@ export async function registerAgentRoutes(app: FastifyInstance, options: AgentOp
const response = await fetch(new URL('/api/conversations/root', baseUrl), { const response = await fetch(new URL('/api/conversations/root', baseUrl), {
headers: { 'X-Sybil-Client': 'backend-proxy-v1', Authorization: `Bearer ${apiKey}` }, signal: signal ? AbortSignal.any([signal, AbortSignal.timeout(15_000)]) : AbortSignal.timeout(15_000), redirect: 'error', headers: { 'X-Sybil-Client': 'backend-proxy-v1', Authorization: `Bearer ${apiKey}` }, signal: signal ? AbortSignal.any([signal, AbortSignal.timeout(15_000)]) : AbortSignal.timeout(15_000), redirect: 'error',
}); });
const descriptor = await response.json() as { capabilities?: { attachments?: unknown } }; const descriptor = await response.json() as { capabilities?: { attachments?: unknown; assistantImages?: unknown } };
if (!response.ok || descriptor?.capabilities?.attachments) throw new Error('Private runtime'); if (!response.ok || descriptor?.capabilities?.attachments || descriptor?.capabilities?.assistantImages) throw new Error('Private runtime');
} catch { throw Object.assign(new Error('Authenticated Sybil access is required for this Agent runtime.'), { statusCode: 503, code: 'attachments_unavailable' }); } } catch { throw Object.assign(new Error('Authenticated Sybil access is required for this Agent runtime.'), { statusCode: 503, code: 'attachments_unavailable' }); }
}; };
const proxyJson = async (path: string, method = 'GET', body?: unknown, signal?: AbortSignal) => { const proxyJson = async (path: string, method = 'GET', body?: unknown, signal?: AbortSignal) => {
@@ -53,6 +53,7 @@ export async function registerAgentRoutes(app: FastifyInstance, options: AgentOp
let response: Response; let response: Response;
try { try {
response = await fetch(new URL(path, baseUrl), { method, response = await fetch(new URL(path, baseUrl), { method,
redirect: 'error',
headers: { 'X-Sybil-Client': 'backend-proxy-v1', Authorization: `Bearer ${apiKey}`, ...(body === undefined ? {} : { 'Content-Type': 'application/json' }) }, headers: { 'X-Sybil-Client': 'backend-proxy-v1', Authorization: `Bearer ${apiKey}`, ...(body === undefined ? {} : { 'Content-Type': 'application/json' }) },
...(body === undefined ? {} : { body: JSON.stringify(body) }), signal: signal ? AbortSignal.any([signal, AbortSignal.timeout(15_000)]) : AbortSignal.timeout(15_000), ...(body === undefined ? {} : { body: JSON.stringify(body) }), signal: signal ? AbortSignal.any([signal, AbortSignal.timeout(15_000)]) : AbortSignal.timeout(15_000),
}); });
+18 -2
View File
@@ -20,14 +20,14 @@ const [{ registerAgentRoutes }, { env }] = await Promise.all([import('../src/age
const auth = { authorization: `Bearer ${process.env.ADMIN_TOKEN}` }, key = 'runtime-private-test-key'; const auth = { authorization: `Bearer ${process.env.ADMIN_TOKEN}` }, key = 'runtime-private-test-key';
const file = { id:'att_12345678-1234-4234-8234-123456789abc',kind:'text',filename:'résumé.txt',mimeType:'text/plain',sizeBytes:4,sha256:'a'.repeat(64),workspacePath:'/data/workspace/uploads/att_12345678-1234-4234-8234-123456789abc/résumé.txt' }; const file = { id:'att_12345678-1234-4234-8234-123456789abc',kind:'text',filename:'résumé.txt',mimeType:'text/plain',sizeBytes:4,sha256:'a'.repeat(64),workspacePath:'/data/workspace/uploads/att_12345678-1234-4234-8234-123456789abc/résumé.txt' };
const bodies: any[] = []; const bodies: any[] = [];
let capable=true, badReceipt=false, errorStatus=0, uploadCalls=0; let capable=true, assistantImages=false, badReceipt=false, errorStatus=0, uploadCalls=0;
let holdDescriptor=false, holdDownload=false;const held:ServerResponse[]=[], closed:ServerResponse[]=[]; let holdDescriptor=false, holdDownload=false;const held:ServerResponse[]=[], closed:ServerResponse[]=[];
const source=createServer(async(req,res)=>{ const source=createServer(async(req,res)=>{
assert.equal(req.headers.authorization,`Bearer ${key}`); assert.equal(req.headers.authorization,`Bearer ${key}`);
const chunks: Buffer[]=[]; const chunks: Buffer[]=[];
try { for await(const chunk of req) chunks.push(chunk); } catch { return; } try { for await(const chunk of req) chunks.push(chunk); } catch { return; }
const bytes=Buffer.concat(chunks);res.setHeader('Content-Type','application/json'); const bytes=Buffer.concat(chunks);res.setHeader('Content-Type','application/json');
if(req.url==='/api/conversations/root'){if(holdDescriptor){held.push(res);res.once('close',()=>closed.push(res));return;}res.end(JSON.stringify({id:'1',...(capable?{capabilities:{attachments:{version:1}}}:{})}));return;} if(req.url==='/api/conversations/root'){if(holdDescriptor){held.push(res);res.once('close',()=>closed.push(res));return;}res.end(JSON.stringify({id:'1',capabilities:{...(capable?{attachments:{version:1}}:{}),...(assistantImages?{assistantImages:{version:1}}:{})}}));return;}
if(errorStatus){res.statusCode=errorStatus;res.end(JSON.stringify({error:true,message:'Storage quota reached',statusCode:errorStatus,code:'attachment_quota_exceeded',remainingBytes:12,quotaBytes:1024,requiredBytes:99}));return;} if(errorStatus){res.statusCode=errorStatus;res.end(JSON.stringify({error:true,message:'Storage quota reached',statusCode:errorStatus,code:'attachment_quota_exceeded',remainingBytes:12,quotaBytes:1024,requiredBytes:99}));return;}
if(req.url?.endsWith('/attachments')){uploadCalls++;assert.match(req.headers['content-type']!,/^multipart\/form-data; boundary=/);assert.ok(bytes.includes(Buffer.from('résumé.txt')));bodies.push(bytes.toString());res.statusCode=201;res.end(JSON.stringify({attachment:file}));return;} if(req.url?.endsWith('/attachments')){uploadCalls++;assert.match(req.headers['content-type']!,/^multipart\/form-data; boundary=/);assert.ok(bytes.includes(Buffer.from('résumé.txt')));bodies.push(bytes.toString());res.statusCode=201;res.end(JSON.stringify({attachment:file}));return;}
if(req.url?.endsWith('/content')){if(holdDownload){held.push(res);res.once('close',()=>closed.push(res));return;}res.writeHead(200,{'Content-Type':'text/plain; charset=utf-8','Content-Length':'4','Content-Disposition':"attachment; filename=notes.txt; filename*=UTF-8''notes.txt"});res.end('data');return;} if(req.url?.endsWith('/content')){if(holdDownload){held.push(res);res.once('close',()=>closed.push(res));return;}res.writeHead(200,{'Content-Type':'text/plain; charset=utf-8','Content-Length':'4','Content-Disposition':"attachment; filename=notes.txt; filename*=UTF-8''notes.txt"});res.end('data');return;}
@@ -57,6 +57,22 @@ test('multipart forwards Unicode filename/uploadId with private auth; quota deta
assert.equal((await app.inject({url:'/v1/agent/attachments/not-an-id/content',headers:auth})).statusCode,404); assert.equal((await app.inject({url:'/v1/agent/attachments/not-an-id/content',headers:auth})).statusCode,404);
}); });
test('assistant-only capability allows authenticated retrieval without granting input uploads or open history',async()=>{
capable=false;assistantImages=true;
const saved=env.ADMIN_TOKEN;
try {
const descriptor=(await app.inject({url:'/v1/agent',headers:auth})).json();
assert.deepEqual(descriptor.capabilities,{assistantImages:{version:1}});
assert.equal((await fetch(base+`/v1/agent/attachments/${file.id}/content`,{headers:auth})).status,200);
assert.equal((await fetch(base+`/v1/agent/attachments/${file.id}/content`)).status,401);
assert.equal((await fetch(base+'/v1/agent/attachments',{method:'POST',headers:auth,body:form()})).status,503);
assert.equal((await app.inject({method:'POST',url:'/v1/agent/messages',headers:auth,payload:{content:'',requestId:'assistant-only',attachments:[{id:file.id}]}})).statusCode,503);
assert.equal((await app.inject({method:'POST',url:'/v1/agent/output_image',headers:auth,payload:{path:'https://never-fetch.invalid/image'}})).statusCode,404);
env.ADMIN_TOKEN=undefined;
for(const url of ['/v1/agent','/v1/agent/messages',`/v1/agent/attachments/${file.id}/content`]) assert.equal((await app.inject({url})).statusCode,503);
} finally {env.ADMIN_TOKEN=saved;capable=true;assistantImages=false;}
});
test('ID-only sends require matching receipts; history retains attachment metadata and raw tools',async()=>{ test('ID-only sends require matching receipts; history retains attachment metadata and raw tools',async()=>{
const payload={content:'',requestId:'send',attachments:[{id:file.id}]}; const payload={content:'',requestId:'send',attachments:[{id:file.id}]};
assert.equal((await app.inject({method:'POST',url:'/v1/agent/messages',headers:auth,payload})).statusCode,202);assert.deepEqual(bodies.at(-1),{...payload,channel:'sybil',whenBusy:'followUp',wait:false}); assert.equal((await app.inject({method:'POST',url:'/v1/agent/messages',headers:auth,payload})).statusCode,202);assert.deepEqual(bodies.at(-1),{...payload,channel:'sybil',whenBusy:'followUp',wait:false});
+5
View File
@@ -41,6 +41,11 @@ test('preview lookup uses only the exact eligible completed root assistant conte
]) { page = invalid; await assert.rejects(load(candidate)); } ]) { page = invalid; await assert.rejects(load(candidate)); }
page = { ...original, messages: [{ ...message, content: '' }] }; page = { ...original, messages: [{ ...message, content: '' }] };
assert.equal(await load(candidate), '', 'empty content never borrows metadata, tool output or reasoning'); assert.equal(await load(candidate), '', 'empty content never borrows metadata, tool output or reasoning');
page = { ...original, messages: [{ ...message, content: 'Caption stays inside authenticated chat.', metadata: { ...message.metadata,
agentAttachments: [{ filename: 'private-name.png', workspacePath: '/private/source.png', url: 'https://never-fetch.invalid/image' }] } }] };
const imageText = await load(candidate);
assert.equal(imageText, '', 'image captions and metadata use only a generic notification');
assert.deepEqual(JSON.parse(previewPayload(imageText, body => ({ body }), 4096).toString()), { body: 'You have a new message.' });
const before = requests.length; const before = requests.length;
for (const messageId of ['tool:1:11', 'outbound:12', 'https://never-fetch.invalid/', 'entry:11:0/../../state']) { for (const messageId of ['tool:1:11', 'outbound:12', 'https://never-fetch.invalid/', 'entry:11:0/../../state']) {
await assert.rejects(load({ ...candidate, messageId })); await assert.rejects(load({ ...candidate, messageId }));
+3 -3
View File
@@ -3554,7 +3554,7 @@ export default function App() {
type="button" type="button"
data-testid="agent-sidebar-item" data-testid="agent-sidebar-item"
aria-current={isAgentMode ? "page" : undefined} aria-current={isAgentMode ? "page" : undefined}
aria-label={`Sybil Agent${agentRoom.unreadCount ? `, ${agentRoom.unreadCount} unread messages` : ""}${isAgentBusy ? ", working" : ""}`} aria-label={`Sybil Agent${agentRoom.unreadCount ? `, ${agentRoom.unreadCount}${agentRoom.unreadCountIsPartial ? " or more" : ""} unread messages` : agentRoom.unreadCountIsPartial ? ", earlier messages may be unread" : ""}${isAgentBusy ? ", working" : ""}`}
onClick={() => { onClick={() => {
setContextMenu(null); setContextMenu(null);
setError(null); setError(null);
@@ -3580,9 +3580,9 @@ export default function App() {
<span className="block truncate text-sm font-semibold">Sybil Agent</span> <span className="block truncate text-sm font-semibold">Sybil Agent</span>
</span> </span>
{isAgentBusy ? <LoaderCircle className="h-4 w-4 shrink-0 animate-spin text-cyan-200" aria-label="Agent working" /> : null} {isAgentBusy ? <LoaderCircle className="h-4 w-4 shrink-0 animate-spin text-cyan-200" aria-label="Agent working" /> : null}
{agentRoom.unreadCount > 0 ? ( {agentRoom.unreadCount > 0 || agentRoom.unreadCountIsPartial ? (
<span data-testid="agent-unread-badge" className="flex h-5 min-w-5 shrink-0 items-center justify-center rounded-full bg-cyan-200 px-1.5 text-[11px] font-bold tabular-nums text-slate-950"> <span data-testid="agent-unread-badge" className="flex h-5 min-w-5 shrink-0 items-center justify-center rounded-full bg-cyan-200 px-1.5 text-[11px] font-bold tabular-nums text-slate-950">
{agentRoom.unreadCount} {agentRoom.unreadCount ? `${agentRoom.unreadCount}${agentRoom.unreadCountIsPartial ? "+" : ""}` : "•"}
</span> </span>
) : null} ) : null}
</button> </button>
+20 -11
View File
@@ -1,19 +1,20 @@
import { useEffect, useRef, useState } from "preact/hooks"; import { useCallback, useEffect, useRef, useState } from "preact/hooks";
import { Copy, Download, FileText, Image, LoaderCircle } from "lucide-preact"; import { Copy, Download, FileText, Image, LoaderCircle } from "lucide-preact";
import { attachmentSize, type AgentAttachment } from "@/lib/agent-attachments"; import { attachmentSize, type AgentAttachment } from "@/lib/agent-attachments";
type Props = { file: AgentAttachment; download(file: AgentAttachment, signal?: AbortSignal): Promise<Blob> }; type Props = { file: AgentAttachment; inline?: boolean; download(file: AgentAttachment, signal?: AbortSignal): Promise<Blob> };
/** Authenticated bytes only; no native credential, remote href, or public file serving. */ /** Authenticated bytes only; no native credential, remote href, or public file serving. */
export function AgentAttachmentCard({ file, download }: Props) { export function AgentAttachmentCard({ file, download, inline = false }: Props) {
const [preview, setPreview] = useState<string>(); const [preview, setPreview] = useState<string>();
const [error, setError] = useState<string>(); const [error, setError] = useState<string>();
const [busy, setBusy] = useState(false); const [busy, setBusy] = useState(false);
const [copied, setCopied] = useState(false); const [copied, setCopied] = useState(false);
const controller = useRef<AbortController>(); const controller = useRef<AbortController>();
useEffect(() => () => { controller.current?.abort(); }, []); const transferring = useRef(false);
useEffect(() => () => { if (preview) URL.revokeObjectURL(preview); }, [preview]); useEffect(() => () => { if (preview) URL.revokeObjectURL(preview); }, [preview]);
const retrieve = async (show: boolean) => { const retrieve = useCallback(async (show: boolean) => {
if (busy) return; if (transferring.current) return;
transferring.current = true;
const request = new AbortController(); controller.current = request; setBusy(true); setError(undefined); const request = new AbortController(); controller.current = request; setBusy(true); setError(undefined);
try { try {
const blob = await download(file, request.signal); if (request.signal.aborted) return; const blob = await download(file, request.signal); if (request.signal.aborted) return;
@@ -24,14 +25,22 @@ export function AgentAttachmentCard({ file, download }: Props) {
setTimeout(() => URL.revokeObjectURL(url), 1000); setTimeout(() => URL.revokeObjectURL(url), 1000);
} }
} catch (failure) { if (!request.signal.aborted) setError(failure instanceof Error ? failure.message : "Download failed."); } } catch (failure) { if (!request.signal.aborted) setError(failure instanceof Error ? failure.message : "Download failed."); }
finally { if (!request.signal.aborted) setBusy(false); } finally { if (controller.current === request) { transferring.current = false; if (!request.signal.aborted) setBusy(false); } }
}; }, [download, file.id, file.sha256, file.mimeType, file.kind, file.sizeBytes, file.filename]);
useEffect(() => {
setPreview(undefined); setError(undefined); setBusy(false); transferring.current = false;
if (inline && file.kind === "image") void retrieve(true);
return () => { controller.current?.abort(); };
}, [inline, retrieve]);
return <div className="agent-attachment"> return <div className="agent-attachment">
<div className="agent-attachment__title"><FileText size={15} /><span>{file.filename}</span><small>{attachmentSize(file.sizeBytes)}</small></div> <div className="agent-attachment__title"><FileText size={15} /><span>{file.filename}</span><small>{attachmentSize(file.sizeBytes)}</small></div>
{file.workspacePath ? <div className="agent-attachment__path"><code title={file.workspacePath}>{file.workspacePath}</code><button type="button" aria-label={`Copy path for ${file.filename}`} title={copied ? "Copied" : "Copy workspace path"} onClick={() => { void navigator.clipboard.writeText(file.workspacePath!).then(() => setCopied(true)).catch(() => setError("Could not copy the path.")); }}><Copy size={13} /></button></div> : null} {!inline && file.workspacePath ? <div className="agent-attachment__path"><code title={file.workspacePath}>{file.workspacePath}</code><button type="button" aria-label={`Copy path for ${file.filename}`} title={copied ? "Copied" : "Copy workspace path"} onClick={() => { void navigator.clipboard.writeText(file.workspacePath!).then(() => setCopied(true)).catch(() => setError("Could not copy the path.")); }}><Copy size={13} /></button></div> : null}
{preview ? <img className="agent-attachment__preview" src={preview} alt={file.filename} /> : null} {inline ? <div className="agent-attachment__image-frame" aria-busy={busy}>
{preview ? <img className="agent-attachment__preview" src={preview} alt={file.filename} onError={() => { setPreview(undefined); setError("Image could not be displayed."); }} />
: <span role="status">{error ? "Image unavailable" : "Loading image…"}</span>}
</div> : preview ? <img className="agent-attachment__preview" src={preview} alt={file.filename} /> : null}
<div className="agent-attachment__actions"> <div className="agent-attachment__actions">
{file.kind === "image" ? <button type="button" disabled={busy} onClick={() => preview ? setPreview(undefined) : void retrieve(true)}><Image size={13} />{preview ? "Hide preview" : "Preview"}</button> : null} {file.kind === "image" && (!inline || error) ? <button type="button" disabled={busy} onClick={() => preview ? setPreview(undefined) : void retrieve(true)}><Image size={13} />{inline ? "Retry image" : preview ? "Hide preview" : "Preview"}</button> : null}
<button type="button" disabled={busy} onClick={() => void retrieve(false)}>{busy ? <LoaderCircle size={13} className="animate-spin" /> : <Download size={13} />}Download</button> <button type="button" disabled={busy} onClick={() => void retrieve(false)}>{busy ? <LoaderCircle size={13} className="animate-spin" /> : <Download size={13} />}Download</button>
</div> </div>
{error ? <span className="agent-attachment__error" role="alert">{error} Retry using the buttons above.</span> : null} {error ? <span className="agent-attachment__error" role="alert">{error} Retry using the buttons above.</span> : null}
+28 -25
View File
@@ -45,6 +45,32 @@
.agent-room__entry { .agent-room__entry {
flex-shrink:0; flex-shrink:0;
min-width:0; min-width:0;
/* Retain chronological DOM/selection/a11y; skip offscreen layout and paint.
This is not DOM virtualization. Remember measured heights when skipped. */
content-visibility:auto;
contain-intrinsic-size:auto 160px;
}
.agent-room__history {
display:grid;
justify-items:center;
gap:8px;
flex-shrink:0;
color:hsl(255 19% 70%);
font-size:12px;
}
.agent-room__history button {
border:1px solid hsl(267 75% 73%/.2);
border-radius:8px;
padding:8px 14px;
background:hsl(261 52% 20%/.56);
cursor:pointer;
}
.agent-room__history button:disabled {
cursor:wait;
opacity:.65;
} }
.agent-room__day { .agent-room__day {
@@ -238,31 +264,6 @@
font-size:9px; font-size:9px;
} }
.agent-room__jump {
position:absolute;
z-index:2;
left:50%;
bottom:14px;
transform:translateX(-50%);
display:flex;
align-items:center;
gap:7px;
white-space:nowrap;
padding:9px 14px;
border:1px solid hsl(266 76% 72%/.43);
border-radius:99px;
background:hsl(256 46% 20%/.96);
color:hsl(263 60% 93%);
box-shadow:0 5px 24px hsl(240 70% 2%/.5);
font-size:11px;
cursor:pointer;
backdrop-filter:blur(14px);
}
.agent-room__jump:hover {
background:hsl(261 48% 27%);
}
.agent-room__composer-area { .agent-room__composer-area {
flex:none; flex:none;
border-top:1px solid hsl(253 40% 40%/.14); border-top:1px solid hsl(253 40% 40%/.14);
@@ -475,4 +476,6 @@
.agent-attachment__actions button { display: flex; align-items: center; gap: 5px; color: #ddd6fe; } .agent-attachment__actions button { display: flex; align-items: center; gap: 5px; color: #ddd6fe; }
.agent-attachment__actions button:disabled { opacity: .5; } .agent-attachment__actions button:disabled { opacity: .5; }
.agent-attachment__preview { display: block; max-height: 260px; max-width: 100%; margin-top: 8px; border-radius: 6px; } .agent-attachment__preview { display: block; max-height: 260px; max-width: 100%; margin-top: 8px; border-radius: 6px; }
.agent-attachment__image-frame { display: flex; align-items: center; justify-content: center; height: 260px; width: 480px; max-width: 100%; margin-top: 8px; }
.agent-attachment__image-frame .agent-attachment__preview { width: 100%; height: 100%; object-fit: contain; margin: 0; }
.agent-attachment__error { display: block; width: 100%; color: #fda4af; font-size: 12px; margin-top: 5px; } .agent-attachment__error { display: block; width: 100%; color: #fda4af; font-size: 12px; margin-top: 5px; }
+45 -22
View File
@@ -1,5 +1,6 @@
import { useEffect, useLayoutEffect, useMemo, useRef, useState } from "preact/hooks"; import { useCallback, useEffect, useLayoutEffect, useMemo, useRef, useState } from "preact/hooks";
import { ArrowDown, Clock3, LoaderCircle, Menu, Paperclip, X, RefreshCw, SendHorizontal, WifiOff } from "lucide-preact"; import { memo } from "preact/compat";
import { Clock3, LoaderCircle, Menu, Paperclip, X, RefreshCw, SendHorizontal, WifiOff } from "lucide-preact";
import type { Message } from "@/lib/api"; import type { Message } from "@/lib/api";
import type { AgentRoomHookResult } from "@/hooks/use-agent-room"; import type { AgentRoomHookResult } from "@/hooks/use-agent-room";
import { agentTranscriptMessages } from "@/lib/agent-room-presentation"; import { agentTranscriptMessages } from "@/lib/agent-room-presentation";
@@ -49,7 +50,7 @@ function toolName(value: unknown) {
return typeof value === "string" ? value.replace(/_/g, " ") : "Tool"; return typeof value === "string" ? value.replace(/_/g, " ") : "Tool";
} }
function AgentBubble({ message, animate, onRetry, download }: { message: Message; animate: boolean; onRetry: (id: string) => void; download: AgentRoomHookResult["downloadAttachment"] }) { const AgentBubble = memo(function AgentBubble({ message, animate, onRetry, download }: { message: Message; animate: boolean; onRetry: (id: string) => void; download: AgentRoomHookResult["downloadAttachment"] }) {
const own = message.role === "user"; const own = message.role === "user";
const info = metadata(message); const info = metadata(message);
const status = typeof info.status === "string" ? info.status : "complete"; const status = typeof info.status === "string" ? info.status : "complete";
@@ -60,8 +61,8 @@ function AgentBubble({ message, animate, onRetry, download }: { message: Message
{!own ? <span className="agent-room__message-avatar" aria-hidden="true"><img src="/character-still.png" alt="" draggable={false} /></span> : null} {!own ? <span className="agent-room__message-avatar" aria-hidden="true"><img src="/character-still.png" alt="" draggable={false} /></span> : null}
<div className="agent-room__message-content"> <div className="agent-room__message-content">
<div className={`message-bubble${own ? " message-bubble--user" : ""}`}> <div className={`message-bubble${own ? " message-bubble--user" : ""}`}>
{own ? <p className="agent-room__plain-message">{message.content}</p> : <MarkdownContent markdown={message.content} openLinksInNewTab />} {own ? <p className="agent-room__plain-message">{message.content}</p> : <MarkdownContent markdown={message.content} openLinksInNewTab allowImages={false} />}
{agentAttachments(message.metadata).map(file => <AgentAttachmentCard key={file.id} file={file} download={download} />)} {agentAttachments(message.metadata, message.role).map(file => <AgentAttachmentCard key={file.id} file={file} download={download} inline={message.role === "assistant"} />)}
</div> </div>
{own && failed && typeof info.error === "string" ? <div className="agent-attachment__error" role="alert">{info.error}</div> : null} {own && failed && typeof info.error === "string" ? <div className="agent-attachment__error" role="alert">{info.error}</div> : null}
<div className="agent-room__message-meta"> <div className="agent-room__message-meta">
@@ -77,7 +78,7 @@ function AgentBubble({ message, animate, onRetry, download }: { message: Message
</div> </div>
</article> </article>
); );
} });
export function AgentConversation({ room, onOpenSidebar, isVisible = true }: Props) { export function AgentConversation({ room, onOpenSidebar, isVisible = true }: Props) {
const [draft, setDraft] = useState(savedDraft); const [draft, setDraft] = useState(savedDraft);
@@ -98,7 +99,7 @@ export function AgentConversation({ room, onOpenSidebar, isVisible = true }: Pro
const entryAnimationsRef = useRef(new Set<Animation>()); const entryAnimationsRef = useRef(new Set<Animation>());
const typingHeightRef = useRef(0); const typingHeightRef = useRef(0);
const historyReadyRef = useRef(!room.isLoading); const historyReadyRef = useRef(!room.isLoading);
const [atBottom, setAtBottom] = useState(true); const historyBeforeRef = useRef(room.historyBeforeCursor);
const scrollerRef = useRef<HTMLDivElement>(null); const scrollerRef = useRef<HTMLDivElement>(null);
const timelineRef = useRef<HTMLDivElement>(null); const timelineRef = useRef<HTMLDivElement>(null);
const textareaRef = useRef<HTMLTextAreaElement>(null); const textareaRef = useRef<HTMLTextAreaElement>(null);
@@ -131,9 +132,20 @@ export function AgentConversation({ room, onOpenSidebar, isVisible = true }: Pro
const element = scrollerRef.current; const element = scrollerRef.current;
if (!element || pinnedRef.current) { anchorRef.current = null; return; } if (!element || pinnedRef.current) { anchorRef.current = null; return; }
const top = element.getBoundingClientRect().top; const top = element.getBoundingClientRect().top;
// Anchor to layout, independently of the bubble's entrance transform. // Find the first visible row without laying out every offscreen message.
const node = Array.from(element.querySelectorAll<HTMLElement>("[data-agent-entry-id]")).find(item => item.getBoundingClientRect().bottom > top + 8); const entries = element.querySelectorAll<HTMLElement>("[data-agent-entry-id]");
anchorRef.current = node ? { id: node.dataset.agentEntryId!, offset: node.getBoundingClientRect().top - top } : null; // Probe row boxes without measuring every skipped subtree in a long history.
let low = 0, high = entries.length;
while (low < high) {
const mid = (low + high) >>> 1;
if (entries[mid].getBoundingClientRect().bottom <= top + 8) low = mid + 1;
else high = mid;
}
const node = entries[low];
// The first row can lose its day separator when older rows arrive. Anchor
// the message itself so that changing that separator cannot shift its text.
const message = node?.querySelector<HTMLElement>("[data-agent-message-id]");
anchorRef.current = node && message ? { id: node.dataset.agentEntryId!, offset: message.getBoundingClientRect().top - top } : null;
}; };
const restoreViewport = () => { const restoreViewport = () => {
const element = scrollerRef.current; const element = scrollerRef.current;
@@ -141,17 +153,19 @@ export function AgentConversation({ room, onOpenSidebar, isVisible = true }: Pro
if (pinnedRef.current) element.scrollTop = element.scrollHeight; if (pinnedRef.current) element.scrollTop = element.scrollHeight;
else if (anchorRef.current) { else if (anchorRef.current) {
const anchor = anchorRef.current; const anchor = anchorRef.current;
const node = Array.from(element.querySelectorAll<HTMLElement>("[data-agent-entry-id]")).find(item => item.dataset.agentEntryId === anchor.id); const node = element.querySelector<HTMLElement>(`[data-agent-entry-id="${CSS.escape(anchor.id)}"]`);
if (node) element.scrollTop += node.getBoundingClientRect().top - element.getBoundingClientRect().top - anchor.offset; const message = node?.querySelector<HTMLElement>("[data-agent-message-id]");
if (message) element.scrollTop += message.getBoundingClientRect().top - element.getBoundingClientRect().top - anchor.offset;
} }
restoredScrollTopRef.current = element.scrollTop; restoredScrollTopRef.current = element.scrollTop;
rememberAnchor(); // Keep the user's key/offset through successive intrinsic-size corrections.
// Recapturing here can anchor a newly revealed older row while its neighbors
// are still being measured, causing the original reading position to drift.
scheduleRead(); scheduleRead();
}; };
const jumpToLatest = () => { const jumpToLatest = () => {
pinnedRef.current = true; pinnedRef.current = true;
anchorRef.current = null; anchorRef.current = null;
setAtBottom(true);
if (scrollerRef.current) { if (scrollerRef.current) {
scrollerRef.current.scrollTop = scrollerRef.current.scrollHeight; scrollerRef.current.scrollTop = scrollerRef.current.scrollHeight;
restoredScrollTopRef.current = scrollerRef.current.scrollTop; restoredScrollTopRef.current = scrollerRef.current.scrollTop;
@@ -197,7 +211,7 @@ export function AgentConversation({ room, onOpenSidebar, isVisible = true }: Pro
useLayoutEffect(() => { useLayoutEffect(() => {
restoreViewport(); restoreViewport();
}, [room.messages, room.isTyping, room.activity?.state, room.isLoading, isVisible]); }, [room.messages, room.isTyping, room.activity?.state, room.isLoading, room.isLoadingOlder, room.olderError, room.hasOlder, isVisible]);
useEffect(() => { useEffect(() => {
mountedRef.current = true; mountedRef.current = true;
@@ -231,10 +245,10 @@ export function AgentConversation({ room, onOpenSidebar, isVisible = true }: Pro
setDraft(value); setDraft(value);
}; };
const retryMessage = (id: string) => { const retryMessage = useCallback((id: string) => {
// Admission errors remain attached to their own bubble. // Admission errors remain attached to their own bubble.
void room.retryMessage(id).catch(() => {}); void room.retryMessage(id).catch(() => {});
}; }, [room.retryMessage]);
const submit = () => { const submit = () => {
const content = draftRef.current.trim(); const content = draftRef.current.trim();
if ((!content && !uploads.length) || pendingUploads || (uploads.length && !room.attachmentCapability)) return; if ((!content && !uploads.length) || pendingUploads || (uploads.length && !room.attachmentCapability)) return;
@@ -247,9 +261,14 @@ export function AgentConversation({ room, onOpenSidebar, isVisible = true }: Pro
const messages = useMemo(() => { const messages = useMemo(() => {
const transcript = agentTranscriptMessages(room.messages); const transcript = agentTranscriptMessages(room.messages);
const historyChanged = historyBeforeRef.current !== room.historyBeforeCursor;
historyBeforeRef.current = room.historyBeforeCursor;
let previousDay = ""; let previousDay = "";
for (const message of transcript) { let lastSeen = -1;
if (!seenMessageIdsRef.current.has(message.id) && historyReadyRef.current && !room.isLoading) { transcript.forEach((message, index) => { if (seenMessageIdsRef.current.has(message.id)) lastSeen = index; });
for (const [index, message] of transcript.entries()) {
// Historical prepends keep their measured space; only new tail rows enter.
if (index > lastSeen && !(historyChanged && lastSeen < 0) && !seenMessageIdsRef.current.has(message.id) && historyReadyRef.current && !room.isLoading) {
enteringMessageIdsRef.current.add(message.id); enteringMessageIdsRef.current.add(message.id);
} }
seenMessageIdsRef.current.add(message.id); seenMessageIdsRef.current.add(message.id);
@@ -265,7 +284,7 @@ export function AgentConversation({ room, onOpenSidebar, isVisible = true }: Pro
</div> </div>
); );
}); });
}, [room.messages, room.isLoading, room.retryMessage, room.downloadAttachment]); }, [room.messages, room.isLoading, room.historyBeforeCursor, room.retryMessage, room.downloadAttachment]);
const activity = room.activity; const activity = room.activity;
const running = room.isTyping; const running = room.isTyping;
@@ -305,19 +324,23 @@ export function AgentConversation({ room, onOpenSidebar, isVisible = true }: Pro
&& Math.abs(scrollerRef.current.scrollTop - restoredScrollTopRef.current) < 1) return; && Math.abs(scrollerRef.current.scrollTop - restoredScrollTopRef.current) < 1) return;
const bottom = distanceFromBottom() <= 48; const bottom = distanceFromBottom() <= 48;
pinnedRef.current = bottom; pinnedRef.current = bottom;
setAtBottom(current => current === bottom ? current : bottom);
rememberAnchor(); rememberAnchor();
if (bottom) scheduleRead(); if (bottom) scheduleRead();
}} }}
> >
<div ref={timelineRef} className="agent-room__timeline"> <div ref={timelineRef} className="agent-room__timeline">
{room.isLoading && !messages.length ? <div className="agent-room__empty" role="status"><LoaderCircle className="h-5 w-5 animate-spin" aria-hidden="true" /><span className="sr-only">Loading conversation</span></div> : null} {room.isLoading && !messages.length ? <div className="agent-room__empty" role="status"><LoaderCircle className="h-5 w-5 animate-spin" aria-hidden="true" /><span className="sr-only">Loading conversation</span></div> : null}
{room.hasOlder ? <div className="agent-room__history">
<button type="button" disabled={room.isLoadingOlder} onClick={() => {
pinnedRef.current = false; rememberAnchor(); room.loadOlder();
}}>{room.isLoadingOlder ? "Loading older messages…" : "Load older messages"}</button>
{room.olderError ? <span role="alert">{room.olderError} Try loading again.</span> : null}
</div> : null}
{messages} {messages}
{running ? <div className="agent-room__typing" role="status" aria-live="polite"><span className="agent-room__typing-dots" aria-hidden="true"><i /><i /><i /></span><span>{workingLabel}</span>{(activity?.backgroundWorkers || 0) > 1 ? <span className="agent-room__worker-count">{activity!.backgroundWorkers} workers</span> : null}</div> : null} {running ? <div className="agent-room__typing" role="status" aria-live="polite"><span className="agent-room__typing-dots" aria-hidden="true"><i /><i /><i /></span><span>{workingLabel}</span>{(activity?.backgroundWorkers || 0) > 1 ? <span className="agent-room__worker-count">{activity!.backgroundWorkers} workers</span> : null}</div> : null}
{!running && waiting && activity?.wait ? <div className="agent-room__waiting" role="status"><Clock3 size={13} /><span>Checking back at {timeFormat.format(new Date(activity.wait.due))}</span></div> : null} {!running && waiting && activity?.wait ? <div className="agent-room__waiting" role="status"><Clock3 size={13} /><span>Checking back at {timeFormat.format(new Date(activity.wait.due))}</span></div> : null}
</div> </div>
</div> </div>
{!atBottom ? <button className="agent-room__jump" onClick={jumpToLatest}><ArrowDown size={15} /><span>{room.unreadCount ? `${room.unreadCount} new message${room.unreadCount === 1 ? "" : "s"}` : "Jump to latest"}</span></button> : null}
</div> </div>
<div className="agent-room__composer-area" <div className="agent-room__composer-area"
@@ -11,6 +11,8 @@ type Props = {
mode?: MarkdownMode; mode?: MarkdownMode;
resolveCitationIndex?: (href: string) => number | undefined; resolveCitationIndex?: (href: string) => number | undefined;
openLinksInNewTab?: boolean; openLinksInNewTab?: boolean;
/** Agent images are delivered only through authenticated attachment IDs. */
allowImages?: boolean;
}; };
function replaceMarkdownLinksWithCitationTokens(markdown: string, resolveCitationIndex?: (href: string) => number | undefined) { function replaceMarkdownLinksWithCitationTokens(markdown: string, resolveCitationIndex?: (href: string) => number | undefined) {
@@ -35,24 +37,28 @@ function setNewTabLinkAttributes(currentNode: Element) {
currentNode.setAttribute("rel", "noopener noreferrer"); currentNode.setAttribute("rel", "noopener noreferrer");
} }
function renderMarkdown(markdown: string, openLinksInNewTab: boolean) { function renderMarkdown(markdown: string, openLinksInNewTab: boolean, allowImages: boolean) {
const rawHtml = marked.parse(markdown, { gfm: true, breaks: true, renderer: markdownRenderer }) as string; const rawHtml = marked.parse(markdown, { gfm: true, breaks: true, renderer: markdownRenderer }) as string;
if (!openLinksInNewTab) return DOMPurify.sanitize(rawHtml, { ADD_ATTR: ["class", "target", "rel"] }); const config = { ADD_ATTR: ["class", "target", "rel"], ...(!allowImages ? {
FORBID_TAGS: ["img", "picture", "source", "video", "audio", "svg", "math", "style"],
FORBID_ATTR: ["style", "srcset", "background", "poster"],
} : {}) };
if (!openLinksInNewTab) return DOMPurify.sanitize(rawHtml, config);
DOMPurify.addHook("afterSanitizeAttributes", setNewTabLinkAttributes); DOMPurify.addHook("afterSanitizeAttributes", setNewTabLinkAttributes);
try { try {
return DOMPurify.sanitize(rawHtml, { ADD_ATTR: ["class", "target", "rel"] }); return DOMPurify.sanitize(rawHtml, config);
} finally { } finally {
DOMPurify.removeHook("afterSanitizeAttributes", setNewTabLinkAttributes); DOMPurify.removeHook("afterSanitizeAttributes", setNewTabLinkAttributes);
} }
} }
export function MarkdownContent({ markdown, className, mode = "default", resolveCitationIndex, openLinksInNewTab = false }: Props) { export function MarkdownContent({ markdown, className, mode = "default", resolveCitationIndex, openLinksInNewTab = false, allowImages = true }: Props) {
const html = useMemo(() => { const html = useMemo(() => {
const prepared = const prepared =
mode === "citationTokens" ? replaceMarkdownLinksWithCitationTokens(markdown, resolveCitationIndex) : markdown; mode === "citationTokens" ? replaceMarkdownLinksWithCitationTokens(markdown, resolveCitationIndex) : markdown;
return renderMarkdown(prepared, openLinksInNewTab); return renderMarkdown(prepared, openLinksInNewTab, allowImages);
}, [markdown, mode, openLinksInNewTab, resolveCitationIndex]); }, [markdown, mode, openLinksInNewTab, resolveCitationIndex, allowImages]);
return <div className={cn("md-content", className)} dangerouslySetInnerHTML={{ __html: html }} />; return <div className={cn("md-content", className)} dangerouslySetInnerHTML={{ __html: html }} />;
} }
+37 -9
View File
@@ -2,7 +2,7 @@ import { useCallback, useEffect, useMemo, useRef, useState } from "preact/hooks"
import { getApiBaseUrl, getAuthToken, type Message } from "@/lib/api"; import { getApiBaseUrl, getAuthToken, type Message } from "@/lib/api";
import { import {
AGENT_ADMISSION_STORAGE_KEY, AgentRoomClient, agentAuthFrame, agentDisplayMessages, agentUnreadCount, AGENT_ADMISSION_STORAGE_KEY, AgentRoomClient, agentAuthFrame, agentDisplayMessages, agentUnreadCount,
applyAgentEvent, applyAgentReadState, applyAgentEvent, applyAgentReadState, applyAgentHistoryPage, agentUnreadCountIsPartial,
createAgentReadSession, createAgentRoomState, parseAgentRoomEvent, syncAgentHistory, createAgentReadSession, createAgentRoomState, parseAgentRoomEvent, syncAgentHistory,
type AgentActivity, type AgentRoomState, type AgentActivity, type AgentRoomState,
} from "@/lib/agent-room"; } from "@/lib/agent-room";
@@ -16,6 +16,7 @@ export type AgentRoomConnection = "connecting" | "connected" | "reconnecting" |
export type AgentRoomHookResult = { export type AgentRoomHookResult = {
messages: Message[]; activity: AgentActivity | null; unreadCount: number; messages: Message[]; activity: AgentActivity | null; unreadCount: number;
connection: AgentRoomConnection; error: string | null; isLoading: boolean; isTyping: boolean; connection: AgentRoomConnection; error: string | null; isLoading: boolean; isTyping: boolean;
hasOlder: boolean; historyBeforeCursor: string | null; isLoadingOlder: boolean; olderError: string | null; loadOlder(): void; unreadCountIsPartial: boolean;
attachmentCapability?: AgentAttachmentCapability; attachmentCapability?: AgentAttachmentCapability;
uploadAttachment(file: File, uploadId: string, signal: AbortSignal): Promise<AgentAttachment>; uploadAttachment(file: File, uploadId: string, signal: AbortSignal): Promise<AgentAttachment>;
downloadAttachment(file: AgentAttachment, signal?: AbortSignal): Promise<Blob>; downloadAttachment(file: AgentAttachment, signal?: AbortSignal): Promise<Blob>;
@@ -33,6 +34,9 @@ export function useAgentRoom(enabled: boolean): AgentRoomHookResult {
const [attachmentCapability, setAttachmentCapability] = useState<AgentAttachmentCapability>(); const [attachmentCapability, setAttachmentCapability] = useState<AgentAttachmentCapability>();
const [error, setError] = useState<string | null>(null); const [error, setError] = useState<string | null>(null);
const [isLoading, setLoading] = useState(enabled); const [isLoading, setLoading] = useState(enabled);
const [isLoadingOlder, setLoadingOlder] = useState(false);
const [olderError, setOlderError] = useState<string | null>(null);
const older = useRef<(() => Promise<void>) | null>(null);
const [revision, setRevision] = useState(0); const [revision, setRevision] = useState(0);
const client = useRef<AgentRoomClient | null>(null); const client = useRef<AgentRoomClient | null>(null);
const [, renderUploads] = useState(0); const [, renderUploads] = useState(0);
@@ -81,9 +85,10 @@ export function useAgentRoom(enabled: boolean): AgentRoomHookResult {
} }
useEffect(() => { useEffect(() => {
setLoadingOlder(false); setOlderError(null);
if (!enabled) { if (!enabled) {
sourceMismatch.current = false; sourceMismatch.current = false;
client.current = null; mark.current = null; setAttachmentCapability(undefined); client.current = null; mark.current = null; older.current = null; setAttachmentCapability(undefined);
update(() => createAgentRoomState()); setLoading(false); setConnection("offline"); setError(null); update(() => createAgentRoomState()); setLoading(false); setConnection("offline"); setError(null);
return; return;
} }
@@ -96,6 +101,7 @@ export function useAgentRoom(enabled: boolean): AgentRoomHookResult {
let reconnectAttempt = 0; let reconnectAttempt = 0;
let started = false; let started = false;
let maintaining = false; let maintaining = false;
let loadingOlder = false;
let lastPacket = Date.now(); let lastPacket = Date.now();
let socketAuthenticated = false; let socketAuthenticated = false;
let authFailed = false; let authFailed = false;
@@ -110,7 +116,7 @@ export function useAgentRoom(enabled: boolean): AgentRoomHookResult {
setView({ room: next, isTyping }); setView({ room: next, isTyping });
}, },
}); });
setLoading(current.current.conversationId === null); setConnection("connecting"); setError(null); setLoading(!current.current.historyLoaded); setConnection("connecting"); setError(null);
const reportError = (failure: unknown) => { if (active && !controller.signal.aborted) setError(errorText(failure)); }; const reportError = (failure: unknown) => { if (active && !controller.signal.aborted) setError(errorText(failure)); };
const readSession = createAgentReadSession({ const readSession = createAgentReadSession({
initial: current.current, initial: current.current,
@@ -119,6 +125,7 @@ export function useAgentRoom(enabled: boolean): AgentRoomHookResult {
sourceChanged = true; sourceMismatch.current = true; mark.current = null; sourceChanged = true; sourceMismatch.current = true; mark.current = null;
clearTimeout(reconnectTimer); clearTimeout(startupTimer); clearTimeout(socketTimeout); clearTimeout(reconnectTimer); clearTimeout(startupTimer); clearTimeout(socketTimeout);
presenter.dispose(); controller.abort(); socket?.close(); presenter.dispose(); controller.abort(); socket?.close();
older.current = null; setLoadingOlder(false);
setError(failure.message); setConnection("offline"); setLoading(false); setError(failure.message); setConnection("offline"); setLoading(false);
}, },
apply: read => { apply: read => {
@@ -139,6 +146,22 @@ export function useAgentRoom(enabled: boolean): AgentRoomHookResult {
const descriptor = await api.descriptor(controller.signal); const descriptor = await api.descriptor(controller.signal);
if (active && !sourceChanged) { setAttachmentCapability(descriptor.capabilities?.attachments?.version === 1 ? descriptor.capabilities.attachments : undefined); presenter.push({ type: "activity", conversationId: descriptor.id, activity: descriptor.activity }); } if (active && !sourceChanged) { setAttachmentCapability(descriptor.capabilities?.attachments?.version === 1 ? descriptor.capabilities.attachments : undefined); presenter.push({ type: "activity", conversationId: descriptor.id, activity: descriptor.activity }); }
}; };
older.current = async () => {
const before = current.current.beforeCursor;
if (!active || sourceChanged || !started || loadingOlder || !current.current.hasOlder || before === null) return;
loadingOlder = true; setLoadingOlder(true); setOlderError(null);
try {
const page = await api.history(before, controller.signal);
if (!active || sourceChanged || controller.signal.aborted) return;
presenter.update(state => applyAgentHistoryPage(state, page, before));
presenter.flush();
} catch (failure) {
if (active && !sourceChanged && !controller.signal.aborted) setOlderError(errorText(failure));
} finally {
loadingOlder = false;
if (active && !sourceChanged) setLoadingOlder(false);
}
};
const connect = () => { const connect = () => {
if (!active || authFailed || sourceChanged) return; if (!active || authFailed || sourceChanged) return;
@@ -191,9 +214,11 @@ export function useAgentRoom(enabled: boolean): AgentRoomHookResult {
const start = async () => { const start = async () => {
try { try {
await Promise.all([refreshRead(), refreshActivity()]); await Promise.all([refreshRead(), refreshActivity()]);
// The first start begins at zero, not the latest page. Unread counts if (!current.current.historyLoaded) {
// therefore include older public replies even after a long absence. const page = await api.history(undefined, controller.signal);
await catchUp(!started); if (!active || sourceChanged || controller.signal.aborted) return;
presenter.update(state => applyAgentHistoryPage(state, page));
} else await catchUp(!started);
if (!active || sourceChanged) return; if (!active || sourceChanged) return;
presenter.flush(); presenter.flush();
started = true; setLoading(false); connect(); started = true; setLoading(false); connect();
@@ -231,18 +256,19 @@ export function useAgentRoom(enabled: boolean): AgentRoomHookResult {
window.removeEventListener("focus", focus); document.removeEventListener("visibilitychange", focus); window.removeEventListener("focus", focus); document.removeEventListener("visibilitychange", focus);
socket?.close(); socket?.close();
if (client.current === api) client.current = null; if (client.current === api) client.current = null;
mark.current = null; mark.current = null; older.current = null;
}; };
}, [enabled, revision, update]); }, [enabled, revision, update]);
const renderedCursor = room.cursor; const renderedCursor = room.cursor;
const renderedSourceId = room.sourceId; const renderedSourceId = room.sourceId;
const hasRenderedHistory = room.messages.some(message => message.role === "user" || message.role === "assistant");
const markRead = useCallback(() => { const markRead = useCallback(() => {
if (!enabled || isLoading) return; if (!enabled || isLoading || !hasRenderedHistory) return;
// This callback belongs to this rendered snapshot. Reading current.current // This callback belongs to this rendered snapshot. Reading current.current
// here would incorrectly mark a socket message that has not rendered yet. // here would incorrectly mark a socket message that has not rendered yet.
mark.current?.(renderedCursor, renderedSourceId); mark.current?.(renderedCursor, renderedSourceId);
}, [enabled, isLoading, renderedCursor, renderedSourceId]); }, [enabled, isLoading, hasRenderedHistory, renderedCursor, renderedSourceId]);
const sendMessage = useCallback(async (content: string, attachments?: AgentAttachment[], onPersisted?: () => void) => { const sendMessage = useCallback(async (content: string, attachments?: AgentAttachment[], onPersisted?: () => void) => {
if (!enabled || !outbox.current) throw new Error("Agent connection is unavailable."); if (!enabled || !outbox.current) throw new Error("Agent connection is unavailable.");
await outbox.current.send(content, attachments, () => { uploadQueue.current!.clear(); onPersisted?.(); }); await outbox.current.send(content, attachments, () => { uploadQueue.current!.clear(); onPersisted?.(); });
@@ -263,7 +289,9 @@ export function useAgentRoom(enabled: boolean): AgentRoomHookResult {
if (sourceMismatch.current) { window.location.reload(); return; } if (sourceMismatch.current) { window.location.reload(); return; }
setRevision(value => value + 1); setRevision(value => value + 1);
}, []); }, []);
const loadOlder = useCallback(() => { void older.current?.(); }, []);
const messages = useMemo(() => outbox.current?.display(agentDisplayMessages(room)) || agentDisplayMessages(room), [room.messages, room.activity, outboxRevision]); const messages = useMemo(() => outbox.current?.display(agentDisplayMessages(room)) || agentDisplayMessages(room), [room.messages, room.activity, outboxRevision]);
return { messages: enabled ? messages : [], activity: room.activity, unreadCount: agentUnreadCount(room), connection, error, isLoading, return { messages: enabled ? messages : [], activity: room.activity, unreadCount: agentUnreadCount(room), connection, error, isLoading,
hasOlder: room.hasOlder, historyBeforeCursor: room.beforeCursor, isLoadingOlder, olderError, loadOlder, unreadCountIsPartial: agentUnreadCountIsPartial(room),
attachmentCapability, uploadQueue: uploadQueue.current!, uploadAttachment, downloadAttachment, isTyping: enabled && view.isTyping, sendMessage, retryMessage, markRead, retry }; attachmentCapability, uploadQueue: uploadQueue.current!, uploadAttachment, downloadAttachment, isTyping: enabled && view.isTyping, sendMessage, retryMessage, markRead, retry };
} }
+15 -2
View File
@@ -19,10 +19,23 @@ export function parseAgentAttachment(value: unknown): AgentAttachment {
return { id: item.id, kind: item.kind, filename: item.filename, mimeType: item.mimeType, sizeBytes: item.sizeBytes, sha256: item.sha256, return { id: item.id, kind: item.kind, filename: item.filename, mimeType: item.mimeType, sizeBytes: item.sizeBytes, sha256: item.sha256,
...(item.workspacePath ? { workspacePath: item.workspacePath } : {}) }; ...(item.workspacePath ? { workspacePath: item.workspacePath } : {}) };
} }
export function agentAttachments(metadata: unknown): AgentAttachment[] { export function agentAttachments(metadata: unknown, role = "user"): AgentAttachment[] {
if (role !== "user" && role !== "assistant") return [];
const values = (metadata as { agentAttachments?: unknown } | null)?.agentAttachments; const values = (metadata as { agentAttachments?: unknown } | null)?.agentAttachments;
if (!Array.isArray(values) || values.length > 8) return []; if (!Array.isArray(values) || values.length > 8) return [];
try { return values.map(parseAgentAttachment); } catch { return []; } try {
const files = values.map(value => {
// Source paths are not part of the assistant output contract, even if an
// older/malformed server supplies one. Only the ID authorizes retrieval.
const file = parseAgentAttachment(role === "assistant" ? { ...value, workspacePath: undefined } : value);
if (role === "assistant" && (file.kind !== "image" || !["image/png", "image/jpeg", "image/gif", "image/webp"].includes(file.mimeType)
|| file.sizeBytes > 6 * 1024 * 1024 || file.filename.length > 255 || /[\\/\u0000-\u001f\u007f]/.test(file.filename)))
throw new Error("Invalid assistant image.");
return file;
});
if (new Set(files.map(file => file.id)).size !== files.length || files.reduce((total, file) => total + file.sizeBytes, 0) > 32 * 1024 * 1024) return [];
return files;
} catch { return []; }
} }
export function attachmentCaption(content: string, files: AgentAttachment[]): string { export function attachmentCaption(content: string, files: AgentAttachment[]): string {
return content.trim() ? content : `Attached: ${files.map(file => file.filename).join(", ")}.`; return content.trim() ? content : `Attached: ${files.map(file => file.filename).join(", ")}.`;
+1 -1
View File
@@ -20,7 +20,7 @@ export type AgentRoomPresenterOptions = {
typingDelayMs?: number; typingDelayMs?: number;
clock?: Scheduler; clock?: Scheduler;
}; };
const quietTools = new Set(['user_message', 'notify_parent', 'clock_wait', 'event_decision']); const quietTools = new Set(['user_message', 'output_image', 'notify_parent', 'clock_wait', 'event_decision']);
const data = (message: Message): Record<string, unknown> => message.metadata && typeof message.metadata === 'object' const data = (message: Message): Record<string, unknown> => message.metadata && typeof message.metadata === 'object'
&& !Array.isArray(message.metadata) ? message.metadata as Record<string, unknown> : {}; && !Array.isArray(message.metadata) ? message.metadata as Record<string, unknown> : {};
const entryId = (message: Message): bigint | undefined => { const entryId = (message: Message): bigint | undefined => {
+67 -7
View File
@@ -18,12 +18,13 @@ export type AgentMessagesPage = {
}; };
export type AgentDescriptor = { export type AgentDescriptor = {
id: string; title: string; model: string; activity: AgentActivity; id: string; title: string; model: string; activity: AgentActivity;
websocket: string; protocol: string; capabilities?: { attachments?: AgentAttachmentCapability }; websocket: string; protocol: string; capabilities?: { attachments?: AgentAttachmentCapability; assistantImages?: { version: 1 } };
}; };
export type AgentReadState = { cursor: string; updatedAt: string | null; sourceId?: string | null }; export type AgentReadState = { cursor: string; updatedAt: string | null; sourceId?: string | null };
export type AgentRoomState = { export type AgentRoomState = {
conversationId: string | null; messages: Message[]; cursor: string; conversationId: string | null; messages: Message[]; cursor: string;
readCursor: string; activity: AgentActivity | null; readCursor: string; activity: AgentActivity | null;
historyLoaded: boolean; beforeCursor: string | null; hasOlder: boolean;
/** undefined before the first read snapshot; null pins a legacy source-less session. */ /** undefined before the first read snapshot; null pins a legacy source-less session. */
sourceId?: string | null; sourceId?: string | null;
}; };
@@ -84,6 +85,7 @@ export function mergeAgentMessages(previous: readonly Message[], incoming: reado
} }
export const createAgentRoomState = (): AgentRoomState => ({ export const createAgentRoomState = (): AgentRoomState => ({
conversationId: null, messages: [], cursor: "0", readCursor: "0", activity: null, conversationId: null, messages: [], cursor: "0", readCursor: "0", activity: null,
historyLoaded: false, beforeCursor: null, hasOlder: false,
}); });
export class AgentReadSourceChangedError extends Error { export class AgentReadSourceChangedError extends Error {
constructor() { super("Agent history changed. Reload the page before marking messages read."); this.name = "AgentReadSourceChangedError"; } constructor() { super("Agent history changed. Reload the page before marking messages read."); this.name = "AgentReadSourceChangedError"; }
@@ -102,6 +104,25 @@ export function applyAgentPage(state: AgentRoomState, page: AgentMessagesPage):
const cursor = page.cursor === null ? state.cursor : advanceAgentCursor(state.cursor, page.cursor); const cursor = page.cursor === null ? state.cursor : advanceAgentCursor(state.cursor, page.cursor);
return { ...state, conversationId: page.conversationId, messages: mergeAgentMessages(state.messages, page.messages), cursor }; return { ...state, conversationId: page.conversationId, messages: mergeAgentMessages(state.messages, page.messages), cursor };
} }
/** Recent/older windows have their own boundary; live replay never moves it. */
export function applyAgentHistoryPage(state: AgentRoomState, page: AgentMessagesPage, before?: string): AgentRoomState {
// A retired or repeated older request cannot overwrite a newer window.
if (before !== undefined && before !== state.beforeCursor) return state;
const boundary = page.beforeCursor === null ? null : normalizeAgentCursor(page.beforeCursor);
const cursor = page.cursor === null ? null : normalizeAgentCursor(page.cursor);
if (page.hasMore && (boundary === null || boundary === "0"
|| (before !== undefined && BigInt(boundary) >= BigInt(before)))) throw new Error("Agent older history did not advance its cursor.");
if (before !== undefined && ((cursor !== null && BigInt(cursor) >= BigInt(before))
|| (boundary !== null && BigInt(boundary) > BigInt(before)))) throw new Error("Agent older history returned an invalid boundary.");
const next = applyAgentPage(state, page);
return { ...next, historyLoaded: true, beforeCursor: boundary, hasOlder: page.hasMore,
// Older snapshots must not replace newer socket edits or advance delivery/read state.
...(before === undefined ? {} : { cursor: state.cursor, messages: mergeAgentMessages(page.messages, state.messages) }),
};
}
export function agentUnreadCountIsPartial(state: AgentRoomState): boolean {
return state.hasOlder && state.beforeCursor !== null && BigInt(state.readCursor) + 1n < BigInt(state.beforeCursor);
}
export function applyAgentReadState(state: AgentRoomState, read: AgentReadState): AgentRoomState { export function applyAgentReadState(state: AgentRoomState, read: AgentReadState): AgentRoomState {
const sourceId = readSource(state, read); const sourceId = readSource(state, read);
return { ...state, sourceId, readCursor: advanceAgentCursor(state.readCursor, read.cursor) }; return { ...state, sourceId, readCursor: advanceAgentCursor(state.readCursor, read.cursor) };
@@ -154,6 +175,8 @@ export class AgentRoomClient {
private readonly baseUrl: string; private readonly baseUrl: string;
private readonly fetcher: typeof globalThis.fetch; private readonly fetcher: typeof globalThis.fetch;
private readonly token: () => string | null; private readonly token: () => string | null;
private activeDownloads = 0;
private readonly downloadWaiters = new Set<() => void>();
constructor(options: AgentRoomClientOptions = {}) { constructor(options: AgentRoomClientOptions = {}) {
this.baseUrl = (options.baseUrl || "/api").replace(/\/+$/, ""); this.baseUrl = (options.baseUrl || "/api").replace(/\/+$/, "");
this.fetcher = (options.fetch || globalThis.fetch).bind(globalThis); this.fetcher = (options.fetch || globalThis.fetch).bind(globalThis);
@@ -178,6 +201,9 @@ export class AgentRoomClient {
messages(after: string, signal?: AbortSignal): Promise<AgentMessagesPage> { messages(after: string, signal?: AbortSignal): Promise<AgentMessagesPage> {
return this.request(`/messages?${new URLSearchParams({ after: normalizeAgentCursor(after), limit: "200" })}`, { signal }); return this.request(`/messages?${new URLSearchParams({ after: normalizeAgentCursor(after), limit: "200" })}`, { signal });
} }
history(before?: string, signal?: AbortSignal): Promise<AgentMessagesPage> {
return this.request(`/messages?${new URLSearchParams({ ...(before === undefined ? {} : { before: normalizeAgentCursor(before) }), limit: "100" })}`, { signal });
}
readState(signal?: AbortSignal): Promise<AgentReadState> { return this.request("/read-state", { signal }); } readState(signal?: AbortSignal): Promise<AgentReadState> { return this.request("/read-state", { signal }); }
markRead(cursor: string, signal?: AbortSignal, sourceId?: string | null): Promise<AgentReadState> { markRead(cursor: string, signal?: AbortSignal, sourceId?: string | null): Promise<AgentReadState> {
return this.request("/read-state", { method: "PUT", body: JSON.stringify({ cursor: normalizeAgentCursor(cursor), ...(agentPushSourceId(sourceId) ? { sourceId } : {}) }), signal }); return this.request("/read-state", { method: "PUT", body: JSON.stringify({ cursor: normalizeAgentCursor(cursor), ...(agentPushSourceId(sourceId) ? { sourceId } : {}) }), signal });
@@ -199,12 +225,46 @@ export class AgentRoomClient {
} }
async download(attachment: AgentAttachment, signal?: AbortSignal): Promise<Blob> { async download(attachment: AgentAttachment, signal?: AbortSignal): Promise<Blob> {
const file = parseAgentAttachment(attachment), headers = new Headers(), token = this.token(); const file = parseAgentAttachment(attachment), headers = new Headers(), token = this.token();
if (token) headers.set("Authorization", `Bearer ${token}`); signal?.throwIfAborted();
const response = await this.fetcher(`${this.baseUrl}/v1/agent/attachments/${encodeURIComponent(file.id)}/content`, { headers, signal, cache: "no-store", redirect: "error" }); while (this.activeDownloads >= 2) {
if (!response.ok) { const error = await response.json().catch(() => null); throw new Error(error?.message || "Attachment download failed."); } await new Promise<void>((resolve, reject) => {
const blob = await response.blob(); const cleanup = () => { this.downloadWaiters.delete(wake); signal?.removeEventListener("abort", abort); };
if (blob.size !== file.sizeBytes || blob.size > 8 * 1024 * 1024) throw new Error("Attachment download was incomplete."); const wake = () => { cleanup(); resolve(); };
return blob; const abort = () => { cleanup(); reject(signal?.reason || new Error("Download canceled.")); };
this.downloadWaiters.add(wake); signal?.addEventListener("abort", abort, { once: true });
if (signal?.aborted) abort();
});
signal?.throwIfAborted();
}
this.activeDownloads++;
try {
if (token !== this.token()) throw new Error("Authentication changed. Retry the download.");
if (token) headers.set("Authorization", `Bearer ${token}`);
const response = await this.fetcher(`${this.baseUrl}/v1/agent/attachments/${encodeURIComponent(file.id)}/content`, { headers, signal, cache: "no-store", redirect: "error" });
if (!response.ok) { await response.body?.cancel(); throw new Error("Attachment download failed. Retry the download."); }
const reader = response.body?.getReader();
if (!reader) throw new Error("Attachment download was incomplete.");
const bytes = new Uint8Array(file.sizeBytes); let size = 0;
try {
for (;;) {
const { done, value } = await reader.read(); if (done) break;
signal?.throwIfAborted();
if (size + value.length > bytes.length) throw new Error("Attachment download exceeds its size limit.");
bytes.set(value, size); size += value.length;
}
} finally { await reader.cancel().catch(() => {}); reader.releaseLock(); }
if (size !== file.sizeBytes) throw new Error("Attachment download was incomplete.");
const sha256 = [...new Uint8Array(await crypto.subtle.digest("SHA-256", bytes))].map(n => n.toString(16).padStart(2, "0")).join("");
if (sha256 !== file.sha256) throw new Error("Attachment checksum did not match.");
if (file.kind === "image" && response.headers.get("content-type")?.split(";")[0].trim() !== file.mimeType)
throw new Error("Attachment image type did not match.");
signal?.throwIfAborted();
if (token !== this.token()) throw new Error("Authentication changed. Retry the download.");
return new Blob([bytes], { type: response.headers.get("content-type") || "application/octet-stream" });
} finally {
this.activeDownloads--;
this.downloadWaiters.values().next().value?.();
}
} }
websocketUrl(pageUrl: string): string { websocketUrl(pageUrl: string): string {
const url = new URL(`${this.baseUrl}/v1/agent/ws`, pageUrl); const url = new URL(`${this.baseUrl}/v1/agent/ws`, pageUrl);
+28 -5
View File
@@ -11,13 +11,15 @@ const browser=await chromium.launch({executablePath:process.env.CHROME_EXECUTABL
...(process.env.BROWSER_NO_SANDBOX === '1' ? {args:['--no-sandbox']} : {})}); ...(process.env.BROWSER_NO_SANDBOX === '1' ? {args:['--no-sandbox']} : {})});
const context=await browser.newContext({viewport:{width:1200,height:900},permissions:['clipboard-write']}); const context=await browser.newContext({viewport:{width:1200,height:900},permissions:['clipboard-write']});
await context.addInitScript(()=>localStorage.setItem('sybil_admin_token','browser-fixture-token')); await context.addInitScript(()=>localStorage.setItem('sybil_admin_token','browser-fixture-token'));
const page=await context.newPage(), errors=[], uploads=[], sends=[], downloads=[]; const page=await context.newPage(), errors=[], uploads=[], sends=[], downloads=[], trackingRequests=[];
page.on('pageerror',e=>errors.push(e.stack || e.message)); page.on('pageerror',e=>errors.push(e.stack || e.message));
const cap={version:1,maxCount:8,maxTotalBytes:33554432,maxImageBytes:6291456,maxFileBytes:8388608,imageMimeTypes:['image/png','image/jpeg','image/gif','image/webp'],quotaBytes:1073741824,usedBytes:0,remainingBytes:1073741824,maxConcurrentUploads:2,maxConcurrentDecodes:1}; const cap={version:1,maxCount:8,maxTotalBytes:33554432,maxImageBytes:6291456,maxFileBytes:8388608,imageMimeTypes:['image/png','image/jpeg','image/gif','image/webp'],quotaBytes:1073741824,usedBytes:0,remainingBytes:1073741824,maxConcurrentUploads:2,maxConcurrentDecodes:1};
const idle={state:'idle',isThinking:false,queued:0,backgroundWorkers:0,tools:[],queuedMessages:[]}; const idle={state:'idle',isThinking:false,queued:0,backgroundWorkers:0,tools:[],queuedMessages:[]};
let failedUpload=true, failedSend=true, live, cursor=0;const history=[]; let failedUpload=true, failedSend=true, live, cursor=0, inputAttachments=true;const history=[];
const image=Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+a2ioAAAAASUVORK5CYII=','base64'); const image=Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+a2ioAAAAASUVORK5CYII=','base64');
const attachment=(name='notes.txt')=>({id:`att_12345678-1234-4234-8234-${name==='picture.png'?'123456789abd':'123456789abc'}`,filename:name,mimeType:name==='picture.png'?'image/png':'text/plain',kind:name==='picture.png'?'image':'text',sizeBytes:name==='picture.png'?image.length:4,sha256:createHash('sha256').update(name==='picture.png'?image:Buffer.from('data')).digest('hex'),workspacePath:`/data/workspace/uploads/test/${name}`}); const attachment=(name='notes.txt')=>({id:`att_12345678-1234-4234-8234-${name==='picture.png'?'123456789abd':'123456789abc'}`,filename:name,mimeType:name==='picture.png'?'image/png':'text/plain',kind:name==='picture.png'?'image':'text',sizeBytes:name==='picture.png'?image.length:4,sha256:createHash('sha256').update(name==='picture.png'?image:Buffer.from('data')).digest('hex'),workspacePath:`/data/workspace/uploads/test/${name}`});
const assistantImage={...attachment('picture.png'),id:'att_12345678-1234-4234-8234-123456789abe',filename:'assistant.png'};delete assistantImage.workspacePath;
await page.route('https://never-fetch.invalid/**',async route=>{trackingRequests.push(route.request().url());await route.abort();});
await page.route('**/api/**',async route=>{ await page.route('**/api/**',async route=>{
const req=route.request(),url=new URL(req.url()),p=url.pathname,m=req.method();let value; const req=route.request(),url=new URL(req.url()),p=url.pathname,m=req.method();let value;
if(p.startsWith('/api/v1/agent'))assert.equal(req.headers().authorization,'Bearer browser-fixture-token'); if(p.startsWith('/api/v1/agent'))assert.equal(req.headers().authorization,'Bearer browser-fixture-token');
@@ -26,7 +28,7 @@ await page.route('**/api/**',async route=>{
if(failedUpload){failedUpload=false;await route.fulfill({status:507,json:{message:'Storage quota reached',code:'attachment_quota_exceeded'}});return;} if(failedUpload){failedUpload=false;await route.fulfill({status:507,json:{message:'Storage quota reached',code:'attachment_quota_exceeded'}});return;}
await route.fulfill({status:201,json:{attachment:attachment(body.includes('picture.png')?'picture.png':'notes.txt')}});return; await route.fulfill({status:201,json:{attachment:attachment(body.includes('picture.png')?'picture.png':'notes.txt')}});return;
} }
if(p.endsWith('/content')){downloads.push(req.url());await route.fulfill({contentType:p.includes('789abd')?'image/png':'text/plain',body:p.includes('789abd')?image:'data'});return;} if(p.endsWith('/content')){downloads.push(req.url());const isImage=p.includes('789abd')||p.includes('789abe');await route.fulfill({contentType:isImage?'image/png':'text/plain',body:isImage?image:'data'});return;}
if(p.endsWith('/agent/messages')&&m==='POST'){ if(p.endsWith('/agent/messages')&&m==='POST'){
const body=req.postDataJSON();sends.push(body); const body=req.postDataJSON();sends.push(body);
if(failedSend){failedSend=false;await route.fulfill({status:503,json:{message:'Temporarily unavailable'}});return;} if(failedSend){failedSend=false;await route.fulfill({status:503,json:{message:'Temporarily unavailable'}});return;}
@@ -37,7 +39,7 @@ await page.route('**/api/**',async route=>{
} }
if(p.endsWith('/agent/messages'))value={conversationId:'1',messages:history,cursor:String(cursor),beforeCursor:null,hasMore:false}; if(p.endsWith('/agent/messages'))value={conversationId:'1',messages:history,cursor:String(cursor),beforeCursor:null,hasMore:false};
else if(p.endsWith('/read-state'))value={cursor:String(cursor),updatedAt:null}; else if(p.endsWith('/read-state'))value={cursor:String(cursor),updatedAt:null};
else if(p.endsWith('/agent'))value={id:'1',title:'Sybil Agent',activity:idle,capabilities:{attachments:cap},protocol:'pi-room.v1',websocket:'/v1/agent/ws'}; else if(p.endsWith('/agent'))value={id:'1',title:'Sybil Agent',activity:idle,capabilities:{...(inputAttachments?{attachments:cap}:{}),assistantImages:{version:1}},protocol:'pi-room.v1',websocket:'/v1/agent/ws'};
else if(p.includes('session'))value={mode:'token',authenticated:true}; else if(p.includes('session'))value={mode:'token',authenticated:true};
else if(p.endsWith('/tools')||p.endsWith('/chat-tools'))value={tools:[]}; else if(p.endsWith('/tools')||p.endsWith('/chat-tools'))value={tools:[]};
else if(p.endsWith('/models'))value={models:[],providers:{}}; else if(p.endsWith('/models'))value={models:[],providers:{}};
@@ -75,7 +77,28 @@ try{
await page.getByText('Uploaded',{exact:true}).waitFor();await page.getByRole('button',{name:'Remove notes.txt'}).click(); await page.getByText('Uploaded',{exact:true}).waitFor();await page.getByRole('button',{name:'Remove notes.txt'}).click();
await page.locator('.agent-room__composer-area').evaluate(el=>{const data=new DataTransfer();data.items.add(new File(['data'],'notes.txt',{type:'text/plain'}));el.dispatchEvent(new DragEvent('drop',{dataTransfer:data,bubbles:true,cancelable:true}));}); await page.locator('.agent-room__composer-area').evaluate(el=>{const data=new DataTransfer();data.items.add(new File(['data'],'notes.txt',{type:'text/plain'}));el.dispatchEvent(new DragEvent('drop',{dataTransfer:data,bubbles:true,cancelable:true}));});
await page.getByText('Uploaded',{exact:true}).waitFor(); await page.getByText('Uploaded',{exact:true}).waitFor();
await page.getByRole('button',{name:'Remove notes.txt'}).click();
// Assistant image-only messages use authenticated bytes automatically, with no source path.
const assistant={id:'outbound:900',role:'assistant',createdAt:new Date().toISOString(),content:'',metadata:{kind:'message',status:'complete',entryId:++cursor,agentAttachments:[assistantImage]}};history.push(assistant);
live.send(JSON.stringify({type:'messages',conversationId:'1',messages:[assistant],cursor:String(cursor),beforeCursor:String(cursor),hasMore:false}));
const assistantBubble=page.locator('[data-agent-message-id="outbound:900"]'),inlineImage=assistantBubble.locator('.agent-attachment__preview');
await inlineImage.waitFor();await inlineImage.evaluate(img=>img.decode());
assert.ok((await inlineImage.getAttribute('src')).startsWith('blob:'));
assert.equal(await assistantBubble.getByRole('button',{name:'Preview',exact:true}).count(),0);
assert.equal(await assistantBubble.locator('.agent-attachment__path').count(),0);
const tracking={id:'outbound:901',role:'assistant',createdAt:new Date().toISOString(),content:'Remote images must not load.\n\n![tracking](https://never-fetch.invalid/markdown.png) <img src="https://never-fetch.invalid/html.png">',metadata:{kind:'message',status:'complete',entryId:++cursor}};history.push(tracking);
live.send(JSON.stringify({type:'messages',conversationId:'1',messages:[tracking],cursor:String(cursor),beforeCursor:String(cursor),hasMore:false}));
await page.getByText('Remote images must not load.',{exact:true}).waitFor();
assert.equal(await page.locator('[data-agent-message-id="outbound:901"] .md-content img').count(),0);
// An output-only capability must preserve history without enabling input controls.
inputAttachments=false;await page.reload({waitUntil:'networkidle'});await page.locator('[data-testid="agent-sidebar-item"]').click();
await inlineImage.waitFor();await inlineImage.evaluate(img=>img.decode());
assert.equal(await assistantBubble.count(),1);assert.equal(await page.locator('.agent-attachment__preview').count(),1);
assert.ok((await inlineImage.getAttribute('src')).startsWith('blob:'));
assert.equal(await page.getByRole('button',{name:'Attach files',exact:true}).count(),0);
assert.equal(await page.getByLabel('Choose attachments').count(),0);
assert.deepEqual(trackingRequests,[],'Neither live nor historical Markdown/HTML image markup fetches remote bytes');
await page.setViewportSize({width:390,height:844});if(process.env.ATTACHMENT_SCREENSHOT)await page.screenshot({path:process.env.ATTACHMENT_SCREENSHOT}); await page.setViewportSize({width:390,height:844});if(process.env.ATTACHMENT_SCREENSHOT)await page.screenshot({path:process.env.ATTACHMENT_SCREENSHOT});
assert.equal(await page.evaluate(()=>document.documentElement.scrollWidth>window.innerWidth),false); assert.equal(await page.evaluate(()=>document.documentElement.scrollWidth>window.innerWidth),false);
assert.deepEqual(errors,[]);console.log('PASS browser picker/retry/quota/remove, file-only send/reload, authenticated preview/download, copy, paste/drop, mobile layout'); assert.deepEqual(errors,[]);console.log('PASS browser picker/retry/quota/remove, file-only send/reload, authenticated preview/download, copy, paste/drop, assistant image-only live/history, output-only capability, blocked Markdown/HTML images, mobile layout');
}finally{await context.close();await browser.close();} }finally{await context.close();await browser.close();}
+37
View File
@@ -5,11 +5,48 @@ import { createAgentUploadQueue, agentAttachments, parseAgentAttachment } from '
import { AgentRoomClient } from '../src/lib/agent-room.ts'; import { AgentRoomClient } from '../src/lib/agent-room.ts';
import { createAgentOptimisticOutbox } from '../src/lib/agent-optimistic-outbox.ts'; import { createAgentOptimisticOutbox } from '../src/lib/agent-optimistic-outbox.ts';
import { agentTranscriptMessages } from '../src/lib/agent-room-presentation.ts'; import { agentTranscriptMessages } from '../src/lib/agent-room-presentation.ts';
import { createAgentRoomState, applyAgentEvent, mergeAgentMessages } from '../src/lib/agent-room.ts';
const file = { id:'att_12345678-1234-4234-8234-123456789abc',kind:'text',filename:'notes.txt',mimeType:'text/plain',sizeBytes:4,sha256:createHash('sha256').update('data').digest('hex'),workspacePath:'/data/workspace/uploads/att_12345678-1234-4234-8234-123456789abc/notes.txt' }; const file = { id:'att_12345678-1234-4234-8234-123456789abc',kind:'text',filename:'notes.txt',mimeType:'text/plain',sizeBytes:4,sha256:createHash('sha256').update('data').digest('hex'),workspacePath:'/data/workspace/uploads/att_12345678-1234-4234-8234-123456789abc/notes.txt' };
const cap = {version:1,maxCount:8,maxTotalBytes:32*1024*1024,maxImageBytes:6*1024*1024,maxFileBytes:8*1024*1024,imageMimeTypes:['image/png','image/jpeg','image/gif','image/webp']}; const cap = {version:1,maxCount:8,maxTotalBytes:32*1024*1024,maxImageBytes:6*1024*1024,maxFileBytes:8*1024*1024,imageMimeTypes:['image/png','image/jpeg','image/gif','image/webp']};
const tick=()=>new Promise(r=>setTimeout(r,0)); const tick=()=>new Promise(r=>setTimeout(r,0));
const waitFor=async check=>{const until=Date.now()+5000;while(!check()){assert.ok(Date.now()<until,'async condition timed out');await tick();}}; const waitFor=async check=>{const until=Date.now()+5000;while(!check()){assert.ok(Date.now()<until,'async condition timed out');await tick();}};
test('assistant images retain stable live/history identity, omit paths, and never enable input capability',async()=>{
const image={...file,kind:'image',mimeType:'image/png',filename:'image.png',url:'https://never-fetch.invalid/image'};
const safe={...image};delete safe.workspacePath;delete safe.url;
const message={id:'outbound:42',role:'assistant',content:'',createdAt:new Date().toISOString(),metadata:{kind:'message',status:'complete',entryId:42,agentAttachments:[image]}};
const page={type:'messages',conversationId:'1',messages:[message],cursor:'42',beforeCursor:'42',hasMore:false};
const live=applyAgentEvent(createAgentRoomState(),page);
assert.deepEqual(agentAttachments(live.messages[0].metadata,'assistant'),[safe]);
const history=mergeAgentMessages(live.messages,[structuredClone(message)]);
assert.equal(history.length,1);assert.equal(history[0].id,'outbound:42');
assert.deepEqual(agentAttachments(history[0].metadata,'assistant'),[safe]);
assert.equal(agentTranscriptMessages(history).length,1,'image-only assistant bubble remains visible');
for(const invalid of [{...image,kind:'file'},{...image,mimeType:'image/svg+xml'},{...image,sizeBytes:6*1024*1024+1},{...image,filename:'/private/source.png'}])
assert.deepEqual(agentAttachments({agentAttachments:[invalid]},'assistant'),[]);
assert.deepEqual(agentAttachments(message.metadata,'tool'),[]);
assert.deepEqual(agentAttachments({agentAttachments:[image,image]},'assistant'),[]);
const descriptor={capabilities:{assistantImages:{version:1}}};
const queue=createAgentUploadQueue({capability:()=>descriptor.capabilities.attachments,changed:()=>{},upload:async()=>{throw new Error('Must not upload');}});
assert.throws(()=>queue.add([new File(['data'],'notes.txt')]),/not available/);queue.dispose();
});
test('downloads bound concurrent bytes, validate hashes/types, cancel queued work, and reject changed credentials',async()=>{
const complete=[];let calls=0,token='one';
const client=new AgentRoomClient({token:()=>token,fetch:async()=>{calls++;return new Promise(resolve=>complete.push(resolve));}});
const a=client.download(file), b=client.download(file), cancel=new AbortController();
const canceled=client.download(file,cancel.signal);const canceledAssertion=assert.rejects(canceled);cancel.abort();
await canceledAssertion;assert.equal(calls,2);
complete.shift()(new Response('data'));await a;
const c=client.download(file);assert.equal(calls,3);
complete.shift()(new Response('evil'));await assert.rejects(b,/checksum/);
token='two';complete.shift()(new Response('data'));await assert.rejects(c,/Authentication changed/);
const huge=new AgentRoomClient({fetch:async()=>new Response('data-exceeds-receipt')});
await assert.rejects(huge.download(file),/size limit/);
const wrongType=new AgentRoomClient({fetch:async()=>new Response('data',{headers:{'content-type':'text/html'}})});
await assert.rejects(wrongType.download({...file,kind:'image',mimeType:'image/png'}),/type did not match/);
});
test('composer gates unsupported servers, count/type/size and batches before upload; preserves retry ID and remove races',async()=>{ test('composer gates unsupported servers, count/type/size and batches before upload; preserves retry ID and remove races',async()=>{
let capability, attempts=[], next=0, changed=0, finish; let capability, attempts=[], next=0, changed=0, finish;
const queue=createAgentUploadQueue({capability:()=>capability,newId:()=>`upload-${++next}`,changed:()=>changed++,upload:async(_file,id,signal)=>{attempts.push({id,signal});if(attempts.length===1)throw new Error('Quota full');return new Promise(resolve=>finish=resolve);}}); const queue=createAgentUploadQueue({capability:()=>capability,newId:()=>`upload-${++next}`,changed:()=>changed++,upload:async(_file,id,signal)=>{attempts.push({id,signal});if(attempts.length===1)throw new Error('Quota full');return new Promise(resolve=>finish=resolve);}});
+74 -1
View File
@@ -4,6 +4,7 @@ import {
AgentRoomClient, AgentReadSourceChangedError, advanceAgentCursor, agentAuthFrame, agentDisplayMessages, agentUnreadCount, AgentRoomClient, AgentReadSourceChangedError, advanceAgentCursor, agentAuthFrame, agentDisplayMessages, agentUnreadCount,
applyAgentEvent, applyAgentPage, applyAgentReadState, createAgentReadSession, createAgentReadWriter, applyAgentEvent, applyAgentPage, applyAgentReadState, createAgentReadSession, createAgentReadWriter,
createAgentRoomState, mergeAgentMessages, normalizeAgentCursor, parseAgentRoomEvent, syncAgentHistory, createAgentRoomState, mergeAgentMessages, normalizeAgentCursor, parseAgentRoomEvent, syncAgentHistory,
applyAgentHistoryPage, agentUnreadCountIsPartial,
} from "../src/lib/agent-room.ts"; } from "../src/lib/agent-room.ts";
import { createAgentOptimisticOutbox } from "../src/lib/agent-optimistic-outbox.ts"; import { createAgentOptimisticOutbox } from "../src/lib/agent-optimistic-outbox.ts";
@@ -32,7 +33,7 @@ test("tool starts and results merge across replay without downgrading completed
assert.equal(messages[0].createdAt, start.createdAt); assert.equal(messages[0].createdAt, start.createdAt);
}); });
test("initial synchronization reads every page including invisible entries and counts only unread assistant messages", async () => { test("forward recovery reads every missing page including invisible entries and counts only unread assistant messages", async () => {
const requests = []; const requests = [];
const responses = [page("2", [message(1), message(2, "user")], true), page("4", [], true), page("7", [message(5), message(6, "tool"), message(7)], false)]; const responses = [page("2", [message(1), message(2, "user")], true), page("4", [], true), page("7", [message(5), message(6, "tool"), message(7)], false)];
let state = createAgentRoomState(); let state = createAgentRoomState();
@@ -47,6 +48,78 @@ test("initial synchronization reads every page including invisible entries and c
assert.equal(agentUnreadCount(state), 1); assert.equal(agentUnreadCount(state), 1);
}); });
test("recent history is one cursorless window and older history uses the returned source boundary", async () => {
const requests = [];
const client = new AgentRoomClient({ fetch: async (url, options) => {
requests.push({ url, signal: options.signal });
return new Response(JSON.stringify({ ...page("1200", [message(1200)], true), beforeCursor: "1101" }));
} });
const controller = new AbortController();
let state = applyAgentHistoryPage(createAgentRoomState(), await client.history(undefined, controller.signal));
assert.equal(requests.length, 1, "hasMore must not drain old history");
assert.equal(requests[0].url, "/api/v1/agent/messages?limit=100");
assert.equal(requests[0].signal, controller.signal);
assert.equal(state.historyLoaded, true);
assert.equal(state.cursor, "1200");
assert.equal(state.beforeCursor, "1101", "boundary is not derived from visible message 1200");
assert.equal(state.hasOlder, true);
await client.history(state.beforeCursor, controller.signal);
assert.equal(requests[1].url, "/api/v1/agent/messages?before=1101&limit=100");
});
test("delayed older pages preserve newer edits, live cursor, source and read state", () => {
let state = applyAgentHistoryPage(sourceRoom(), { ...page("100", [message(100)], true), beforeCursor: "90" });
state = applyAgentPage(state, page("110", [message(100, "assistant", { content: "New streamed text" }), message(110)]));
const next = applyAgentHistoryPage(state, { ...page("89", [message(80), message(100, "assistant", { content: "Old snapshot" })], true), beforeCursor: "70" }, "90");
assert.equal(next.messages.find(m => m.id === "entry:100").content, "New streamed text");
assert.equal(next.cursor, "110");
assert.equal(next.readCursor, "90");
assert.equal(next.sourceId, sourceA);
assert.equal(next.beforeCursor, "70");
assert.equal(next.hasOlder, true);
assert.equal(applyAgentHistoryPage(next, page("89", [message(80)], false), "90"), next, "retired boundary cannot replace the next window");
});
test("empty internal pages advance older boundaries without chasing another page or moving read state", () => {
const initial = applyAgentHistoryPage(createAgentRoomState(), { ...page("1200", [], true), beforeCursor: "1101" });
assert.equal(initial.historyLoaded, true);
assert.equal(initial.messages.length, 0);
const next = applyAgentHistoryPage(initial, { ...page("1100", [], true), beforeCursor: "1001" }, "1101");
assert.equal(next.beforeCursor, "1001");
assert.equal(next.cursor, "1200");
assert.equal(next.readCursor, "0");
assert.equal(next.hasOlder, true);
const end = applyAgentHistoryPage(next, { ...page("0"), beforeCursor: "1001" }, "1001");
assert.equal(end.hasOlder, false);
assert.equal(end.cursor, "1200");
});
test("malformed or stalled older pages cannot change history or delivery boundaries", () => {
const state = applyAgentHistoryPage(createAgentRoomState(), { ...page("100", [message(100)], true), beforeCursor: "90" });
for (const bad of [
{ ...page("89", [], true), beforeCursor: "90" },
{ ...page("89", [], true), beforeCursor: null },
{ ...page("91", [], false), beforeCursor: "80" },
{ ...page("89", [], false), beforeCursor: "91" },
{ ...page("bad", [], false), beforeCursor: "80" },
]) assert.throws(() => applyAgentHistoryPage(state, bad, "90"), /cursor|boundary/);
assert.throws(() => applyAgentHistoryPage(state, { ...page("89"), conversationId: "different" }, "90"), /conversation changed/);
assert.equal(state.beforeCursor, "90");
assert.equal(state.cursor, "100");
});
test("unread counts disclose unknown older replies until reading the latest rendered window", () => {
let state = applyAgentHistoryPage(createAgentRoomState(), { ...page("1200", [message(1199), message(1200, "user")], true), beforeCursor: "1101" });
assert.equal(agentUnreadCount(state), 1);
assert.equal(agentUnreadCountIsPartial(state), true);
state = applyAgentReadState(state, { cursor: "1100", updatedAt: null });
assert.equal(agentUnreadCountIsPartial(state), false);
assert.equal(agentUnreadCount(state), 1);
state = applyAgentReadState(state, { cursor: "1200", updatedAt: null });
assert.equal(agentUnreadCount(state), 0);
assert.equal(state.hasOlder, true, "reading latest does not fetch or discard older pages");
});
test("queued user messages keep stable IDs and disappear if withdrawn or promoted", () => { test("queued user messages keep stable IDs and disappear if withdrawn or promoted", () => {
const pending = message(1, "user", { id: "inbound:sybil:request1", metadata: { kind: "message", status: "queued" } }); const pending = message(1, "user", { id: "inbound:sybil:request1", metadata: { kind: "message", status: "queued" } });
let state = applyAgentEvent(createAgentRoomState(), { type: "activity", conversationId: "1", activity: activity([pending]) }); let state = applyAgentEvent(createAgentRoomState(), { type: "activity", conversationId: "1", activity: activity([pending]) });
+250
View File
@@ -0,0 +1,250 @@
import assert from 'node:assert/strict';
import { mkdir, writeFile } from 'node:fs/promises';
import { createHash } from 'node:crypto';
// Local production preview; all APIs/sockets are synthetic and external requests are blocked.
// PLAYWRIGHT_MODULE=/path/to/playwright/index.mjs WEB_TEST_URL=http://127.0.0.1:5196
// TRANSCRIPT_EVIDENCE=/tmp/evidence TRANSCRIPT_BASELINE=1 node tests/agent-transcript.browser.mjs
const { chromium } = await import(process.env.PLAYWRIGHT_MODULE || 'playwright');
const target = new URL(process.env.WEB_TEST_URL || 'http://127.0.0.1:5196/');
assert.ok(target.protocol === 'http:' && ['localhost', '127.0.0.1', '[::1]'].includes(target.hostname));
const baseline = process.env.TRANSCRIPT_BASELINE === '1';
const evidence = process.env.TRANSCRIPT_EVIDENCE;
if (evidence) await mkdir(evidence, { recursive: true });
const browser = await chromium.launch({ headless: true, executablePath: process.env.CHROME_EXECUTABLE || undefined });
const context = await browser.newContext({ viewport: { width: 1280, height: 900 }, serviceWorkers: 'block' });
const page = await context.newPage();
const errors = [], requests = [], reads = [], socketCursors = [];
page.on('pageerror', error => errors.push(error.message));
const idle = { state: 'idle', isThinking: false, queued: 0, backgroundWorkers: 0, tools: [], queuedMessages: [] };
const sourceId = '00000000-0000-4000-8000-000000000001';
const row = (id, content) => ({ id: `entry:${id}`, role: id % 3 ? 'assistant' : 'user',
createdAt: new Date(Date.UTC(2026, 9, 16, 0, 0, id)).toISOString(),
content: content || `Fixture message ${id}\n\n${'Synthetic transcript text for delayed loading. '.repeat(id % 11 === 0 ? 30 : 3)}`,
metadata: { kind: 'message', entryId: id, status: 'complete' } });
const history = Array.from({ length: 1200 }, (_, i) => row(i + 1));
let live, readCursor = '0', failOlder = false, imageBytes, emptyRecent = false;
const start = performance.now();
const elapsed = () => Math.round(performance.now() - start);
await context.addInitScript(() => {
localStorage.clear(); sessionStorage.clear();
window.fixtureTiming = {};
const sample = () => {
const scroller = document.querySelector('.agent-room__scroll');
if (scroller) {
const box = scroller.getBoundingClientRect();
const visible = [...scroller.querySelectorAll('[data-agent-message-id]')].filter(e => {
const r = e.getBoundingClientRect(); return r.bottom > box.top && r.top < box.bottom;
});
if (visible.length && !window.fixtureTiming.first) window.fixtureTiming.first = { ms: performance.now(), ids: visible.map(e => e.dataset.agentMessageId) };
if (visible.some(e => e.dataset.agentMessageId === 'entry:1200') && !window.fixtureTiming.latestMs) window.fixtureTiming.latestMs = performance.now();
}
if (!window.fixtureTiming.latestMs) requestAnimationFrame(sample);
};
requestAnimationFrame(sample);
});
await page.route('**/*', async route => {
const req = route.request(), url = new URL(req.url());
if (url.origin !== target.origin) return route.abort();
if (!url.pathname.startsWith('/api/')) return route.continue();
const path = url.pathname, method = req.method();
assert.equal(req.headers().authorization, undefined, 'Fixture must never use a real credential');
let value;
if (path.endsWith('/content')) {
await new Promise(resolve => setTimeout(resolve, 600));
return route.fulfill({ contentType: 'image/png', body: imageBytes });
} else if (path.endsWith('/agent/messages')) {
assert.equal(method, 'GET', 'Fixture must not send messages');
const after = url.searchParams.get('after'), before = url.searchParams.get('before'), limit = Number(url.searchParams.get('limit') || 200);
const record = { query: url.search, requestedMs: elapsed() }; requests.push(record);
await new Promise(resolve => setTimeout(resolve, before ? 1600 : 800));
if (before && failOlder) { failOlder = false; return route.fulfill({ status: 503, json: { message: 'Fixture older page unavailable' } }); }
const eligible = history.filter(m => after !== null ? m.metadata.entryId > Number(after) : before !== null ? m.metadata.entryId < Number(before) : true);
const messages = after !== null ? eligible.slice(0, limit) : eligible.slice(-limit);
value = { conversationId: 'fixture-room', messages, cursor: String(messages.at(-1)?.metadata.entryId ?? after ?? 0),
beforeCursor: messages.length ? String(messages[0].metadata.entryId) : null, hasMore: eligible.length > messages.length };
if (emptyRecent && before === null && after === null) value.messages = [];
Object.assign(record, { responseMs: elapsed(), rows: messages.length, bytes: Buffer.byteLength(JSON.stringify(value)) });
} else if (path.endsWith('/read-state')) {
if (method === 'PUT') { readCursor = req.postDataJSON().cursor; reads.push({ cursor: readCursor, ms: elapsed() }); }
value = { cursor: readCursor, sourceId, updatedAt: null };
} else if (path.endsWith('/agent')) value = { id: 'fixture-room', title: 'Sybil Agent', activity: idle, protocol: 'pi-room.v1', websocket: '/v1/agent/ws' };
else if (path.endsWith('/auth/session')) value = { authenticated: true, mode: 'open' };
else if (path.endsWith('/models')) value = { models: [], providers: {} };
else if (path.endsWith('/tools') || path.endsWith('/chat-tools')) value = { tools: [] };
else value = [];
await route.fulfill({ json: value });
});
await page.routeWebSocket('**/api/v1/agent/ws', socket => {
live = socket;
socket.onMessage(raw => { const frame = JSON.parse(String(raw)); if (frame.type === 'auth') {
assert.equal(frame.token, null); socketCursors.push(frame.after);
socket.send(JSON.stringify({ type: 'ready', conversationId: 'fixture-room' }));
} });
});
const send = messages => {
for (const message of messages) { const index = history.findIndex(m => m.id === message.id); if (index < 0) history.push(message); else history[index] = message; }
live.send(JSON.stringify({ type: 'messages', conversationId: 'fixture-room', messages,
cursor: String(messages.at(-1).metadata.entryId), beforeCursor: String(messages[0].metadata.entryId), hasMore: false }));
};
const position = () => page.locator('.agent-room__scroll').evaluate(el => ({ top: el.scrollTop, distance: el.scrollHeight - el.scrollTop - el.clientHeight,
rows: el.querySelectorAll('[data-agent-entry-id]').length }));
const anchor = () => page.locator('.agent-room__scroll').evaluate(el => {
const top = el.getBoundingClientRect().top;
const entry = [...el.querySelectorAll('[data-agent-entry-id]')].find(e => e.getBoundingClientRect().bottom > top + 8);
return { id: entry.dataset.agentEntryId, offset: entry.querySelector('[data-agent-message-id]').getBoundingClientRect().top - top };
});
const anchorDrift = saved => page.locator('.agent-room__scroll').evaluate((el, saved) => {
const entry = [...el.querySelectorAll('[data-agent-entry-id]')].find(e => e.dataset.agentEntryId === saved.id);
return entry.querySelector('[data-agent-message-id]').getBoundingClientRect().top - el.getBoundingClientRect().top - saved.offset;
}, saved);
const metrics = { mode: baseline ? 'before' : 'after', fixtureRows: 1200, pageDelayMs: 800, olderDelayMs: 1600 };
try {
await page.goto(new URL('/?agent=root', target).href, { waitUntil: 'domcontentloaded' });
await page.locator('[data-testid="agent-sidebar-item"]').click();
await page.waitForFunction(() => window.fixtureTiming?.first);
if (evidence) await page.screenshot({ path: `${evidence}/${metrics.mode}-first.png` });
await page.waitForFunction(() => window.fixtureTiming?.latestMs, { timeout: 20000 });
await page.getByRole('status').filter({ hasText: /^Connected$/ }).waitFor();
await page.waitForTimeout(400);
Object.assign(metrics, await page.evaluate(() => window.fixtureTiming), { initial: await position(), initialRequests: requests.map(r => ({ ...r })), socketCursors: [...socketCursors] });
if (evidence) await page.screenshot({ path: `${evidence}/${metrics.mode}-latest.png` });
if (!baseline) {
assert.equal(requests.length, 1, 'No eager older/forward chain on initial load');
assert.equal(new URLSearchParams(requests[0].query).has('after'), false);
assert.equal(socketCursors[0], '1200');
assert.ok(metrics.initial.rows <= 100);
assert.ok(metrics.initial.distance < 2);
await page.waitForTimeout(1800);
assert.equal(requests.length, 1, 'Idle initial view must not drain history');
await page.locator('.agent-room__scroll').evaluate(el => { el.scrollTop = 0; });
await page.waitForTimeout(150);
const saved = await anchor(), readBefore = readCursor;
await page.getByRole('button', { name: 'Load older messages', exact: true }).click();
send([row(1201)]);
await page.waitForTimeout(500);
assert.equal(readCursor, readBefore, 'Incoming while reading must stay unread');
metrics.incomingDriftPx = await anchorDrift(saved);
await page.getByRole('button', { name: 'Load older messages', exact: true }).waitFor();
await page.waitForTimeout(400);
metrics.prependDriftPx = await anchorDrift(saved);
metrics.afterPrepend = await position();
assert.ok(Math.abs(metrics.incomingDriftPx) < 2);
assert.ok(Math.abs(metrics.prependDriftPx) < 2);
assert.equal(readCursor, readBefore);
assert.equal(requests.length, 2, 'One user action loads one older page');
assert.equal(await page.locator('[data-agent-entry-id="entry:1001"] [data-animation="enter"]').count(), 0);
const beforeGrowth = await anchor();
send([row(1201, 'Dynamic streamed content.\n\n'.repeat(80))]);
await page.waitForTimeout(350);
metrics.streamingReadingDriftPx = await anchorDrift(beforeGrowth);
assert.ok(Math.abs(metrics.streamingReadingDriftPx) < 2);
await page.locator('.agent-room__scroll').evaluate(el => { el.scrollTop = el.scrollHeight; });
await page.waitForTimeout(250);
send([row(1201, 'Dynamic streamed content continues.\n\n'.repeat(100))]);
await page.waitForTimeout(400);
metrics.streamingPinnedDistancePx = (await position()).distance;
assert.ok(metrics.streamingPinnedDistancePx < 2);
const order = await page.locator('.agent-room__timeline').evaluate(el => {
const ids = [...el.querySelectorAll('[data-agent-entry-id]')].map(e => Number(e.dataset.agentEntryId.split(':')[1]));
const range = document.createRange(); range.selectNodeContents(el);
return { ids, text: range.toString(), direction: getComputedStyle(el).flexDirection,
skipped: [...el.querySelectorAll('[data-agent-message-id]')].filter(e => !e.checkVisibility({ contentVisibilityAuto: true })).length };
});
assert.deepEqual(order.ids, [...order.ids].sort((a, b) => a - b));
assert.ok(order.text.indexOf('Fixture message 1001') < order.text.indexOf('Fixture message 1200'));
assert.equal(order.direction, 'column');
assert.equal(await page.getByRole('button', { name: /latest/i }).count(), 0);
metrics.skippedOffscreenRows = order.skipped;
assert.ok(order.skipped > 0, 'Offscreen rows should skip layout/paint');
const ax = await page.locator('.agent-room__scroll').ariaSnapshot();
assert.ok(ax.includes('Fixture message 1001') && ax.includes('Fixture message 1200'));
assert.ok(ax.indexOf('Fixture message 1001') < ax.indexOf('Fixture message 1200'));
// Authenticated attachment bytes are synthetic, delayed, and never fetched off origin.
imageBytes = Buffer.from(await page.evaluate(() => {
const canvas = document.createElement('canvas'); canvas.width = 240; canvas.height = 180;
const ctx = canvas.getContext('2d'); ctx.fillStyle = '#8974da'; ctx.fillRect(0, 0, 240, 180);
return canvas.toDataURL('image/png').split(',')[1];
}), 'base64');
const file = { id: 'att_12345678-1234-4234-8234-123456789abc', filename: 'fixture.png', mimeType: 'image/png', kind: 'image',
sizeBytes: imageBytes.length, sha256: createHash('sha256').update(imageBytes).digest('hex') };
send([{ ...row(1202), metadata: { ...row(1202).metadata, agentAttachments: [file] } }]);
await page.waitForTimeout(500);
await page.getByRole('button', { name: 'Preview', exact: true }).click();
await page.locator('.agent-attachment__preview').waitFor();
await page.waitForTimeout(250);
metrics.attachmentPinnedDistancePx = (await position()).distance;
assert.ok(metrics.attachmentPinnedDistancePx < 2);
send([row(1203, 'Following the delayed attachment.\n\n'.repeat(60))]);
await page.waitForTimeout(500);
await page.locator('.agent-room__scroll').evaluate(el => {
const entry = el.querySelector('[data-agent-entry-id="entry:1203"]');
el.scrollTop += entry.getBoundingClientRect().top - el.getBoundingClientRect().top + 80;
});
await page.waitForTimeout(200);
const attachmentAnchor = await anchor();
// Collapse an earlier preview while reading the following message.
await page.getByRole('button', { name: 'Hide preview', exact: true }).evaluate(el => el.click());
await page.waitForTimeout(300);
metrics.attachmentReadingDriftPx = await anchorDrift(attachmentAnchor);
assert.ok(Math.abs(metrics.attachmentReadingDriftPx) < 2);
// A failed older page is locally retryable without clearing the recent window.
await page.locator('.agent-room__scroll').evaluate(el => { el.scrollTop = 0; });
await page.waitForTimeout(200);
const rowsBeforeFailure = (await position()).rows;
failOlder = true;
await page.getByRole('button', { name: 'Load older messages', exact: true }).click();
await page.getByRole('alert').filter({ hasText: 'Fixture older page unavailable' }).waitFor();
assert.equal((await position()).rows, rowsBeforeFailure);
const socketsBefore = socketCursors.length;
await page.getByRole('button', { name: 'Load older messages', exact: true }).click();
await page.waitForTimeout(150);
live.close({ code: 1012, reason: 'Fixture reconnect' });
await page.getByRole('button', { name: 'Reconnect to Sybil Agent', exact: true }).click();
await page.waitForTimeout(2200);
assert.ok(socketCursors.length > socketsBefore);
assert.equal(socketCursors.at(-1), '1203');
assert.equal((await position()).rows, rowsBeforeFailure, 'Retired older response must not publish into the new session');
assert.equal(await page.getByRole('alert').count(), 0);
await page.locator('.agent-room__scroll').evaluate(el => { el.scrollTop = 0; });
await page.getByRole('button', { name: 'Load older messages', exact: true }).click();
await page.waitForFunction(() => document.querySelector('[data-agent-entry-id="entry:901"]'));
metrics.retiredOlderIgnored = true;
metrics.olderFailureRetry = true;
assert.equal(requests.filter(r => r.query.includes('after=0')).length, 0);
await page.locator('.agent-room__scroll').evaluate(el => { el.scrollTop = el.scrollHeight; });
await page.waitForTimeout(200);
await page.setViewportSize({ width: 390, height: 844 });
await page.waitForTimeout(400);
metrics.mobilePinnedDistancePx = (await position()).distance;
assert.ok(metrics.mobilePinnedDistancePx < 2);
assert.equal(await page.evaluate(() => document.documentElement.scrollWidth > innerWidth), false);
assert.equal(await page.getByRole('button', { name: /latest/i }).count(), 0);
if (evidence) await page.screenshot({ path: `${evidence}/after-mobile.png` });
// A source window can contain only internal entries. It stays bounded and
// must not mark unloaded public history read or animate an older prepend.
emptyRecent = true; readCursor = '0';
const requestsBeforeEmpty = requests.length;
await page.setViewportSize({ width: 1280, height: 900 });
await page.goto(new URL('/?agent=root', target).href, { waitUntil: 'domcontentloaded' });
await page.locator('[data-testid="agent-sidebar-item"]').click();
await page.getByRole('button', { name: 'Load older messages', exact: true }).waitFor();
await page.waitForTimeout(1000);
assert.equal((await position()).rows, 0);
assert.equal(requests.length, requestsBeforeEmpty + 1);
assert.equal(readCursor, '0');
await page.getByRole('button', { name: 'Load older messages', exact: true }).click();
await page.waitForFunction(() => document.querySelectorAll('[data-agent-entry-id]').length > 0);
assert.equal(await page.locator('[data-animation="enter"]').count(), 0);
metrics.internalOnlyWindowBounded = true;
metrics.internalOnlyWindowUnacknowledged = true;
}
assert.deepEqual(errors, []);
metrics.requests = requests; metrics.reads = reads; metrics.errors = errors;
if (evidence) await writeFile(`${evidence}/${metrics.mode}.json`, JSON.stringify(metrics, null, 2));
console.log(JSON.stringify(metrics, null, 2));
} catch (error) {
console.error(JSON.stringify({ metrics, requests, errors, body: (await page.locator('body').innerText()).slice(0, 1800) }));
throw error;
} finally { await context.close(); await browser.close(); }