224 lines
13 KiB
Markdown
224 lines
13 KiB
Markdown
# Sybil for iOS and Mac Catalyst
|
|
|
|
Run `just run` for the iPhone simulator, or `just run-mac` for Mac Catalyst.
|
|
`just build-mac` builds without launching the Mac app. Both regenerate the Xcode
|
|
project from `project.yml`. The Mac recipes use ad-hoc signing for local development;
|
|
release/archive signing settings are unchanged.
|
|
|
|
The conversation sidebar includes **Search chats and web searches** on iPhone,
|
|
iPad, and Mac. It searches all saved history by title, conversation text, and saved
|
|
web-search query/answer, with single-line excerpts that bold matching words and
|
|
truncate in the middle. Search waits briefly while typing,
|
|
cancels superseded requests, and leaves the current conversation open. Clear the
|
|
field to return to recent items. Searches use word prefixes (for example, `plan`
|
|
finds `planning`); if more than 100 items match, refine the query.
|
|
|
|
## Sybil Agent
|
|
|
|
The permanent **Sybil Agent** sidebar item opens the persistent room on both
|
|
phone and iPad. It uses the configured Sybil API URL (including an explicit
|
|
`/api` path) and token. The room stays subscribed while other chats are selected;
|
|
foreground and network recovery replay from the last merged cursor. Shared unread
|
|
counts include only public assistant replies and are acknowledged after the
|
|
visible transcript reaches the bottom and its entrance animation finishes.
|
|
|
|
Multiple messages can be queued while the Agent works. The local outbox is saved
|
|
atomically under Application Support before submission and is scoped to the API
|
|
base URL. An interrupted send stays on its original bubble with **Retry**, retaining
|
|
its request ID and timestamp across relaunch. A canonical server echo confirms it
|
|
even if the HTTP acknowledgement is lost. Raw tool calls and results are hidden
|
|
in the Agent transcript, including loaded history and live updates. History windows,
|
|
date separators and scroll anchors use visible messages; raw merged history and
|
|
cursors remain intact. Send failures, room errors and activity indicators remain
|
|
visible. Model selection and tool cards continue to belong to ordinary chats.
|
|
|
|
Agent contracts remain in `../docs/api/rest.md` and
|
|
`../docs/api/streaming-chat.md`. ROOT attachments consume the coordinated v1
|
|
contract (2026-10-14), now documented in the integrated API sources above.
|
|
Attachment readiness requires the coordinated backend/runtime and existing public
|
|
API bearer authentication. This iOS change does not deploy or configure them.
|
|
|
|
### ROOT attachments
|
|
|
|
The Agent composer exposes Files and Photo Library only when the authenticated
|
|
descriptor advertises supported `capabilities.attachments.version == 1`. Missing,
|
|
malformed, or unsupported capability keeps the client text-only. Ordinary chat
|
|
attachments retain their existing inline format and behavior.
|
|
Open-proxy HTTP 503 / WebSocket 4503 failures disable attachment selection while
|
|
retaining draft descriptors and outbox retry identities; reconnect checks support.
|
|
|
|
Files preserve their original bytes, including PDF, archives, text and HEIC files.
|
|
Photo Library HEIC/HEIF selections are intentionally converted to bounded JPEG
|
|
visual input with a matching `.jpg` filename. PNG/JPEG/GIF/WebP are validated as
|
|
raster images; model input is static, with no claim of document or animation
|
|
understanding. Safe original names supplied by the picker are retained; PhotosPicker
|
|
may provide an export filename rather than the original Photo Library asset name.
|
|
The production app does not request broad Photo Library access.
|
|
|
|
Selection order is stable. One preparation/upload runs at a time, with per-item
|
|
loading, upload, error, retry and removal. Sending requires every item to be ready;
|
|
an empty caption can send ready attachments. Advertised limits are checked against
|
|
v1 hard ceilings (8 files, 32 MiB total, 6 MiB/image, 8 MiB/file, 50 million pixels,
|
|
16,384 pixels/axis). Quota snapshots are decoded but do not reserve space. Server
|
|
messages/codes, quota details and Retry-After survive failures, including 409, 413,
|
|
429 and 507. Removal cancels local work and removes the reference; it does not
|
|
delete an uploaded server original.
|
|
|
|
The caption, ordered upload UUIDs and ready descriptors are saved in a small atomic
|
|
manifest under Application Support, scoped to the API base URL. Prepared bytes are
|
|
staged in app-owned files (at most 8 MiB each and 32 MiB per draft); no inline bytes
|
|
or tokens enter the outbox. Relaunch restores ready drafts and offers explicit retry
|
|
for interrupted staged uploads with the same upload UUID. A selection interrupted
|
|
before staging must be removed and selected again. Uploads suspend when the room
|
|
stops; failed work does not automatically retry. Uploaded originals have no client
|
|
expiration. After submission, the outbox owns the caption, ordered descriptors and
|
|
stable request ID. A persisted draft identity recovers a crash during this handoff
|
|
without creating a second send. Storage failure blocks submission rather than
|
|
discarding a draft. Attachment receipts must acknowledge exactly the submitted IDs
|
|
in order; matching canonical echoes still win ambiguous HTTP failures.
|
|
|
|
History and queued/live messages decode only `metadata.agentAttachments` for ROOT.
|
|
Cards retain filename, MIME, size and the optional server-provided `workspacePath`,
|
|
which can be copied. The client never constructs a runtime path or sends one as a
|
|
reference; leading-dot display filenames remain unchanged. Captions remain public
|
|
text, including the server's fallback for attachment-only messages.
|
|
|
|
Original downloads use the configured authenticated URLSession and API base
|
|
(including `/api`), reject redirects, and verify exact size and SHA-256. At most two
|
|
downloads run concurrently, with a 16 MiB in-memory cache discarded on configuration
|
|
change. Queued/active cancellation and generation checks prevent stale previews
|
|
from crossing configurations. Raster previews use bounded thumbnails; text previews
|
|
are plain text, including HTML/SVG. Save/share exports exact original bytes to a
|
|
temporary file removed when its card disappears or configuration changes. An app
|
|
crash may leave a temporary export for normal OS temporary-directory cleanup.
|
|
Image preview frames have fixed height to preserve scroll anchors during loading.
|
|
Attachment modals track their own visibility without changing Quick Question state.
|
|
|
|
### Agent regression and Simulator checks
|
|
|
|
Run these commands from this worktree's `ios/` directory:
|
|
|
|
```sh
|
|
just build
|
|
just test
|
|
just run
|
|
just screenshot build/agent-evidence/simulator.png
|
|
```
|
|
|
|
`just test` runs reducer, cursor, outbox, presentation, shared-read and lifecycle
|
|
regressions along with attachment capability/metadata, multipart/auth/bytes/errors,
|
|
HEIC conversion versus original-file preservation, ordered receipts, draft/outbox
|
|
relaunch, cancellation, two-transfer concurrency and configuration isolation tests
|
|
through Xcode on iPhone 16e.
|
|
For interactive UI coverage, start the local fixture (Python with `aiohttp`):
|
|
|
|
```sh
|
|
python3 Tests/Fixtures/agent_server.py
|
|
```
|
|
|
|
In a second terminal, run:
|
|
|
|
```sh
|
|
just generate
|
|
xcodebuild test -scheme SybilAgentUI \
|
|
-destination 'platform=iOS Simulator,name=iPhone 16e,OS=latest' \
|
|
-parallel-testing-enabled NO \
|
|
-resultBundlePath build/agent-evidence/phone-ui.xcresult
|
|
```
|
|
|
|
The fixture binds only to `127.0.0.1:8799`, uses `fixture-token`, and never contacts
|
|
external services. UI tests exercise the real REST/WebSocket client and retain
|
|
screenshots in the result bundle. They cover queued submissions, uncertain
|
|
admission/retry, tools, asynchronous replies, inactive sidebar unread counts,
|
|
shared reads, reconnect/malformed replay, foreground recovery, long-history scroll
|
|
anchors and multiline keyboard layout. The launch arguments select the fixture
|
|
through ordinary API settings; no fixture mode is compiled into the application.
|
|
Attachment cases exercise authenticated originals, safe text/raster display,
|
|
history reload, capability absence and picker cancellation. The real Photo Library
|
|
selection-to-send test is explicitly skipped: this iOS 26.5 XCTest environment has
|
|
unreliable Photos authorization, callback isolation and synthetic-asset cleanup.
|
|
Selection-to-send remains a manual/device pre-release check, not a validated UI
|
|
path. The gate prevents photo mutations during the general UI suite. Synthetic
|
|
HEIC conversion and original HEIC file preservation are covered by package tests.
|
|
The fixture accepts multipart uploads, stable upload/message retries, structured
|
|
errors and attachment receipts; it is not a runtime image-decoder or quota test.
|
|
Use a fresh result-bundle path for each run. Check the same UI on iPad and with
|
|
Simulator Reduce Motion enabled before release.
|
|
|
|
## Quick Question on macOS
|
|
|
|
While the Catalyst app is running, **Option+Space** toggles a floating Quick
|
|
Question panel, including over other applications. The prompt is focused as soon
|
|
as it opens, with any existing prompt selected so typing replaces it. The same
|
|
select-on-open behavior applies to Quick Question on iPhone and iPad.
|
|
**Return** submits; **Shift+Return** inserts a new line. **Escape**, a
|
|
second **Option+Space**, the close button, or clicking outside dismisses it.
|
|
Hiding the panel preserves the draft and answer and lets an in-flight answer finish.
|
|
**Open in Chat** saves the question and answer in the regular workspace.
|
|
|
|
Quick Question defaults to OpenAI `chat-latest` on iPhone, iPad, and Mac. Its saved
|
|
provider/model selection takes precedence and is separate from regular chat preferences.
|
|
|
|
The regular Mac chat/search composer uses the same Return/Shift+Return rules.
|
|
Holding Return does not repeatedly submit. On iOS, Return submits from Quick
|
|
Question, while the normal chat/search composer continues to use Return for a
|
|
new line.
|
|
|
|
Closing the last Mac window leaves Sybil running, so Option+Space still works.
|
|
Use **Sybil → Quit Sybil** or **Command+Q** to quit the application. The Catalyst
|
|
build opts out of automatic termination, and the native shortcut service holds a
|
|
matching automatic-termination activity until it stops. No hidden keep-alive
|
|
window or sleep-preventing activity is used. Mac scene support also allows the
|
|
workspace to reopen after its last window is closed.
|
|
|
|
When Catalyst is backgrounded, opening Quick Question first requests app
|
|
activation and waits for UIKit to enter the foreground before creating or ordering
|
|
the native panel. Presentation is deferred out of the hotkey event callback.
|
|
Pressing Option+Space again cancels a pending presentation. Dismissing a panel
|
|
that required activation returns focus to the previous app; explicitly opening
|
|
the workspace keeps focus in Sybil.
|
|
|
|
On Mac, **Command+N** starts a new chat in the focused workspace, or opens a new
|
|
workspace window when none are open. **Command+Shift+N** always opens a new window,
|
|
including while signed out. **Command+Option+N** starts a new search; iOS retains
|
|
its existing Command+Shift+N search shortcut.
|
|
|
|
The shortcut is also shown under **File → Quick Question**. If another application
|
|
already owns Option+Space, the panel reports that the shortcut is unavailable;
|
|
the menu item can still be clicked. No Accessibility or Input Monitoring grant is
|
|
needed for the registered hotkey.
|
|
|
|
The native AppKit panel and Carbon hotkey live in `Apps/SybilMacQuickQuestion`,
|
|
a macOS bundle embedded only for the Mac Catalyst destination. Catalyst loads it
|
|
through the Foundation-only `SybilMacQuickQuestionBridge` contract. Requests,
|
|
provider/model preferences, streaming, and conversion to chat remain in the
|
|
existing `SybilViewModel`; there is no separate backend or API contract. The panel
|
|
uses AppKit controls rather than trying to load the macOS SwiftUI runtime into a
|
|
Catalyst process.
|
|
|
|
The iPhone/iPad app keeps its existing Quick Question sheet and Home Screen quick
|
|
action, and does not build or embed the macOS helper.
|
|
|
|
## Verification
|
|
|
|
- `just test`: existing Swift package tests on the iPhone simulator.
|
|
- From `Packages/Sybil`, run `xcodebuild test -scheme Sybil -destination 'platform=macOS,variant=Mac Catalyst' -parallel-testing-enabled NO` for package tests including the Catalyst panel controller.
|
|
- For a manual Mac check, open the panel from another app, type immediately,
|
|
submit, hide/reopen during streaming, and verify Escape/outside-click dismissal
|
|
returns input to the previous app. Check a second display and a full-screen Space
|
|
when available.
|
|
- Close every Mac window, invoke Option+Space, and verify the saved question and
|
|
answer remain available. Reopen the workspace from the panel, then explicitly
|
|
quit with Command+Q. Rebuild and relaunch after changing lifecycle plist keys.
|
|
- In both Mac and iOS Quick Question, reopen an existing prompt and type to replace
|
|
it. On iOS, verify Return submits Quick Question while Return in the normal
|
|
prompt adds a newline. In the Mac workspace, verify Return submits and
|
|
Shift+Return adds a newline.
|
|
- Check Command+N both with a workspace open and after closing every workspace.
|
|
Command+Shift+N must open another window even when one already exists, and
|
|
Command+Option+N must start a search without opening a window.
|
|
- With all workspace windows closed, switch to another app and repeatedly open
|
|
and dismiss Quick Question, including two quick Option+Space presses while it
|
|
is opening. Check that no late panel appears after cancellation and that no
|
|
`UINSAppLifecycleStateRunningNoOpenWindows` assertion occurs.
|