Sybil-2/ios/Tests/AgentPerformance.md

Agent interaction benchmarks

Run the production Agent view in an app-hosted XCTest on the iPhone 16e simulator:

just generate
xcodebuild test -scheme SybilPresentation \
  -destination 'platform=iOS Simulator,name=iPhone 16e,OS=latest' \
  -derivedDataPath build/DerivedData -parallel-testing-enabled NO \
  -only-testing:SybilPresentationTests/AgentPerformanceTests/testInputAndEntranceResponsiveness \
  -resultBundlePath build/agent-input-performance.xcresult

testInputAndEntranceResponsiveness seeds 160 local mixed literal/Markdown messages through the normal room reducer (100 initially displayed), then:

• Receives 12 more messages at 550 ms intervals through the production 100 ms publication batch and 240 ms entrance animation, checking bottom following.
• Presents and dismisses the actual software keyboard six times, 600 ms per transition. It asserts that the transcript viewport shrinks by over 100 points and remains at the bottom; a hardware-keyboard-only run is invalid.
• Calls UIKit insertText on the actual focused composer every 30 ms, including enough text to wrap and grow to its six-line limit. It verifies the draft, bottom following, and that keyboard/draft changes don't reproject the history.

The real composer, Markdown rows, avatars and entrance animations remain enabled. No server, production credentials, notification delivery or network images are needed. The log prints AGENT_INPUT_PERFORMANCE_JSON; the result bundle retains JSON and screenshots of the actual hosted view before and after typing. Screenshots capture the hosting view, not the separate system keyboard window.

Use the same test source, simulator, build configuration and host load for before/after measurements. Don't run other builds or profilers during sampling. Functional assertions about observation boundaries should fail against the old implementation; use the same sampling procedure when collecting a baseline.

Measurement limits

Delivery-gap p50/p95/p99/max and counts over 25/50 ms measure CADisplayLink main-run-loop callback delivery, not compositor FPS, GPU hitches, touch-to-key latency or physical-device performance. The hosted view excludes the surrounding app shell. UIKit input exercises editing/layout without synthesizing key taps. The first keyboard presentation includes system warmup; residual system and simulator scheduling gaps remain. Native gestures, read settling, header fade, attachments and reading anchors are covered separately by the loopback UI tests.

AgentRenderDiagnostics counters are Debug-only and opt-in. They explain work; a drop in counts alone does not establish improved responsiveness.

The transcript now uses LazyVStack with the observation boundaries introduced in bbaa5e8. Loading older history may move the viewport; this is the only waived position guarantee. Native regressions still assert reading anchors for incoming messages, bottom following, keyboard behavior, read settling, fade, and attachments. There is no saved prepend anchor or convergence loop.

Lazy content height estimates change as offscreen rows materialize. The earlier 43869→35870 point difference was not shortened content: after-arrival screenshots were byte-identical. Compare visible geometry and native behavior, not estimated whole-history height, when assessing lazy performance.

Scrolling harness

testThinkingScrollResponsiveness retains the earlier three approximately 12-second phases: idle scroll, stationary thinking, and scrolling with changing queue/worker/tool activity every 500 ms. Its display link sweeps 2400 points at 600 points/second. This programmatic sweep does not reproduce native drag phases; use the UI tests for reading-anchor geometry and interactive keyboard dismissal.

The eager-stack reference in b9a6f5d has exact heights of 39181.333 points idle and 39223.333 while thinking. That reference harness retains its eager-only total-height assertions; run it at bbaa5e8 for the eager comparison. Those totals are not valid assertions about lazy estimated content size. Use the input harness and native gesture tests above for the current lazy implementation. That earlier scrolling fix reduced busy-scroll p95 from 88.8 to 19.4 ms; it did not address the input/entrance paths measured here. Physical phone validation remains necessary for hardware performance claims.

2026-10-16 comparison before the prepend waiver

Single Debug simulator runs on the same iPhone 16e / iOS 26.5, starting at 7510603 (which includes b9a6f5d). Values are callback-gap p95 milliseconds:

Path	Before	Eager isolation (bbaa5e8)	Earlier lazy experiment
12 message entrances	40.8	33.2	16.8
Six keyboard cycles	33.2	33.1	29.2
Multiline composer typing	24.8	25.2	21.3

Eager transcript projections were 12/0/0 versus 220/24/10, with no row-frame preference callbacks in these pinned-bottom phases. Eager content height stayed 43869.333 points. Entrance p99 improved 83.4→57.1 ms; keyboard p99 55.9→44.5; typing p99 33.2→26.7. Residual gaps remain: entrance/keyboard had 10/3 gaps over 50 ms; typing had 30 over 25 ms (baseline 19). Keyboard p95 was essentially flat and typing p95 did not improve. These single runs establish removed transcript work and a safer entrance improvement, not universally smooth input or device FPS. The lazy option was initially rejected because strict prepend anchors failed. The subsequent release accepts that prepend tradeoff; all other guarantees remain.

Lazy release after the prepend waiver

The same input harness passes unchanged: p95 entrance/keyboard/typing gaps are 16.9/28.1/20.9 ms, versus the eager run's 33.2/33.1/25.2 ms. Projections stay 12/0/0; pinned row-frame preference callbacks stay zero. The after-arrival PNG is byte-identical to the eager baseline. Residual gaps remain (one keyboard gap over 50 ms and one typing gap over 25 ms); these are single simulator samples. Five focused native regressions pass with only the two prepend-offset assertions replaced by checks that the remaining history loaded.