Sybil-2/ios/Tests/AgentPerformance.md

Agent interaction benchmarks

Run the production Agent view in an app-hosted XCTest on the iPhone 16e simulator:

just generate
xcodebuild test -scheme SybilPresentation \
  -destination 'platform=iOS Simulator,name=iPhone 16e,OS=latest' \
  -derivedDataPath build/DerivedData -parallel-testing-enabled NO \
  -only-testing:SybilPresentationTests/AgentPerformanceTests/testInputAndEntranceResponsiveness \
  -resultBundlePath build/agent-input-performance.xcresult

testInputAndEntranceResponsiveness seeds 160 local mixed literal/Markdown messages through the normal room reducer (100 initially displayed), then:

• Receives 12 more messages at 550 ms intervals through the production 100 ms publication batch and 240 ms entrance animation, checking bottom following.
• Presents and dismisses the actual software keyboard six times, 600 ms per transition. It asserts that the transcript viewport shrinks by over 100 points and remains at the bottom; a hardware-keyboard-only run is invalid.
• Calls UIKit insertText on the actual focused composer every 30 ms, including enough text to wrap and grow to its six-line limit. It verifies the draft, bottom following, and that keyboard/draft changes don't reproject the history.

The real composer, Markdown rows, avatars and entrance animations remain enabled. No server, production credentials, notification delivery or network images are needed. The log prints AGENT_INPUT_PERFORMANCE_JSON; the result bundle retains JSON and screenshots of the actual hosted view before and after typing. Screenshots capture the hosting view, not the separate system keyboard window.

Use the same test source, simulator, build configuration and host load for before/after measurements. Don't run other builds or profilers during sampling. Functional assertions about observation boundaries should fail against the old implementation; use the same sampling procedure when collecting a baseline.

Measurement limits

Delivery-gap p50/p95/p99/max and counts over 25/50 ms measure CADisplayLink main-run-loop callback delivery, not compositor FPS, GPU hitches, touch-to-key latency or physical-device performance. The hosted view excludes the surrounding app shell. UIKit input exercises editing/layout without synthesizing key taps. The first keyboard presentation includes system warmup; residual system and simulator scheduling gaps remain. Native gestures, read settling, header fade, attachments and reading anchors are covered separately by the loopback UI tests.

AgentRenderDiagnostics counters are Debug-only and opt-in. They explain work; a drop in counts alone does not establish improved responsiveness.

The shipped transcript retains its eager stack and exact row geometry. A lazy experiment reduced measured work but failed the existing 2–3 point prepend anchor checks. Retaining a pending anchor across geometry updates also failed; an indefinite correction loop is not acceptable. The eager fallback keeps the existing one-shot correction, isolates composer/read observation, and removes row frame observers while following the bottom.

The experimental lazy content height changed from 43869 to 35870 points because unmaterialized rows use estimates, not because content was shortened. Its after-arrival screenshot was byte-identical to the eager baseline. Row text, spacing, attachments and entrance growth were unchanged; the anchor failure was sufficient to reject the lazy variant.

Scrolling harness

testThinkingScrollResponsiveness retains the earlier three approximately 12-second phases: idle scroll, stationary thinking, and scrolling with changing queue/worker/tool activity every 500 ms. Its display link sweeps 2400 points at 600 points/second. This programmatic sweep does not reproduce native drag phases; use the UI tests for reading-anchor geometry and interactive keyboard dismissal.

The eager-stack reference in b9a6f5d has exact heights of 39181.333 points idle and 39223.333 while thinking, retained as regression assertions. That earlier scrolling fix reduced busy-scroll p95 from 88.8 to 19.4 ms; it did not address the input/entrance paths measured here. Physical phone validation remains necessary for hardware performance claims.

2026-10-16 focused comparison

Single Debug simulator runs on the same iPhone 16e / iOS 26.5, starting at 7510603 (which includes b9a6f5d). Values are callback-gap p95 milliseconds:

Path	Before	Shipped eager isolation	Rejected lazy isolation
12 message entrances	40.8	33.2	16.8
Six keyboard cycles	33.2	33.1	29.2
Multiline composer typing	24.8	25.2	21.3

Final transcript projections were 12/0/0 versus 220/24/10, with no row-frame preference callbacks in these pinned-bottom phases. Eager content height stayed 43869.333 points. Entrance p99 improved 83.4→57.1 ms; keyboard p99 55.9→44.5; typing p99 33.2→26.7. Residual gaps remain: entrance/keyboard had 10/3 gaps over 50 ms; typing had 30 over 25 ms (baseline 19). Keyboard p95 was essentially flat and typing p95 did not improve. These single runs establish removed transcript work and a safer entrance improvement, not universally smooth input or device FPS. The faster lazy option was rejected because strict reading/prepend anchors failed.