69 lines
3.7 KiB
Markdown
69 lines
3.7 KiB
Markdown
|
# Agent scrolling benchmark
|
||
|
|
|
||
|
|
Run the production Agent view in an app-hosted XCTest on the iPhone 16e simulator:
|
||
|
|
|
||
|
|
```sh
|
||
|
|
just generate
|
||
|
|
xcodebuild test -scheme SybilPresentation \
|
||
|
|
-destination 'platform=iOS Simulator,name=iPhone 16e,OS=latest' \
|
||
|
|
-derivedDataPath build/DerivedData -parallel-testing-enabled NO \
|
||
|
|
-only-testing:SybilPresentationTests/AgentPerformanceTests \
|
||
|
|
-resultBundlePath build/agent-performance.xcresult
|
||
|
|
```
|
||
|
|
|
||
|
|
`AgentPerformanceTests` seeds a room through its normal event reducer with 160
|
||
|
|
local messages (100 initially displayed). User text is literal; assistant text
|
||
|
|
contains paragraphs, lists, links, emphasis and code. The real Agent view,
|
||
|
|
composer, avatar, entrance layout and thinking spinner remain enabled. No server,
|
||
|
|
production credentials, notification delivery or network images are needed.
|
||
|
|
|
||
|
|
Three approximately 12-second phases run in order: idle scrolling, thinking at
|
||
|
|
the bottom, and thinking while scrolling. The test explicitly resets the offset
|
||
|
|
to the bottom before the stationary thinking phase (including in the baseline).
|
||
|
|
Thinking activity changes queue depth, worker count and tool progress every
|
||
|
|
500 ms through the normal 100 ms publication
|
||
|
|
batch. The display link sweeps a 2400-point range at 600 points/second, reversing
|
||
|
|
every four seconds. Use the **same test source, simulator, build configuration
|
||
|
|
and host load** for before/after runs. Don't run other builds or profilers during
|
||
|
|
a measured run. Warm builds and repeat runs help distinguish stable effects from
|
||
|
|
host noise.
|
||
|
|
|
||
|
|
The geometry assertions use iPhone 16e, iOS 26.5, and the default text size. They
|
||
|
|
retain the unoptimized transcript heights (39181.333 points idle, 39223.333 while
|
||
|
|
thinking), so an isolation wrapper cannot silently change row spacing. Rebaseline
|
||
|
|
these explicitly when changing simulator/font configuration or the fixture.
|
||
|
|
|
||
|
|
The log prints `AGENT_PERFORMANCE_JSON`; the result bundle retains the JSON as an
|
||
|
|
attachment. Delivery-gap p50/p95/p99/max and counts over 25/50 ms measure actual
|
||
|
|
main-run-loop responsiveness. `missedBudgetIntervals` estimates gaps relative to
|
||
|
|
the reported display-link budget. These are **not compositor FPS, GPU hitches,
|
||
|
|
touch latency or physical-device results**. Scrolling is programmatic; native
|
||
|
|
swipe/anchor/read behavior needs the local-fixture UI regression tests as well.
|
||
|
|
The hosted view excludes the surrounding app shell. Physical iPhone scrolling,
|
||
|
|
streaming and thinking animation remain a required follow-up.
|
||
|
|
|
||
|
|
Opt-in `AgentRenderDiagnostics` counters are compiled only in Debug and are off
|
||
|
|
outside this test. They explain work (Agent body evaluations, message projection,
|
||
|
|
message text construction, isolated row evaluations, geometry callbacks and
|
||
|
|
snapshot publications). A drop in these counts alone does not establish improved
|
||
|
|
frame responsiveness.
|
||
|
|
|
||
|
|
## Reference result
|
||
|
|
|
||
|
|
On monsoon, Xcode 26.5, Debug, iPhone 16e / iOS 26.5, clean repeat runs against
|
||
|
|
`f534f76` plus counters and against the geometry/row isolation fix gave:
|
||
|
|
|
||
|
|
| Phase | Before p95 / p99 (ms) | After p95 / p99 (ms) | Gaps >50 ms, before / after |
|
||
|
|
| --- | ---: | ---: | ---: |
|
||
|
|
| Idle scroll | 50.01 / 50.02 | 16.68 / 16.71 | 128 / 0 |
|
||
|
|
| Thinking stationary | 33.33 / 81.92 | 28.91 / 38.35 | 22 / 2 |
|
||
|
|
| Thinking scroll | 88.78 / 90.89 | 19.41 / 30.42 | 123 / 1 |
|
||
|
|
|
||
|
|
Each phase lasted approximately 12 seconds. Content heights matched exactly.
|
||
|
|
Busy-scroll Agent body/projection evaluations fell from 261 to 27; unchanged
|
||
|
|
text constructions were zero in both versions. Geometry isolation alone removed
|
||
|
|
the idle-scroll stalls; individual row isolation reduced activity-related gaps.
|
||
|
|
These results support those two boundaries, without a Markdown cache or lazy
|
||
|
|
layout change. Occasional simulator gaps remain; this does not replace device
|
||
|
|
validation.
|