[earakely-scale]

M7.1 of the CUA env epic (FD-4345): computer-recording/v1, FD-4426. It follows the reviewed design, CUA Env M7.1: Computer Recording, with option C: the driver streams and the SDK does the rest.

What's in it

• The SDK (agentenv_protocol.computer.recording). A driver declares recording = Recording() to serve computer-recording/v1. The base class owns each recording's life:

  • start and end, with their retries;
  • max_concurrent, max_duration_s and a free-disk floor;
  • segments rolled over at max_segment_s, on a size change, or after a gap;
  • uploads through start's write_namespace grant as each segment closes, with end's fresh grant retrying any that missed;
  • the manifest.

  The pixels come from a new @record operation, an async iterator of either:

  • RecordedFrame: frames the OS streams, which the base class encodes as H.264, each timed by its captured_at;
  • RecordedSegment: files a native recorder finished.

  A driver without @record falls back to the base class polling @capture, with a pointer drawn at @pointer_position, capped at 15 fps. PyAV comes from a new recording extra (av>=17, which keeps Python 3.10).

• The probe (full). It records for 2 s without a grant and reads every segment back through computer-exec. It checks each segment's size, SHA-256 and MP4 header (size, frame count, duration) against the manifest, and that there's no gap between segments. It also checks that a repeated request_id joins the recording, and that max_concurrent holds. A small MP4 header reader reads the files, with no decoder.

• Core:

  • start_computer_recording {env_id, name, fps?, max_edge?, cursor?} issues the grant, under agentenv-recordings/<uuid>/.
  • end_computer_recording {name} ends the recording and registers the uploaded segments and manifest.json as one FileArtifactUniverse, <instance>__recording-<name>. context.metadata["computer_recordings"][name] says how it went.
  • teardown_run, teardown_sandboxes and reset_env end open recordings first, with reason teardown or reset.
  • A recording whose driver forgot it, as after a restart, is registered from what its namespace holds, with ended_by: error.

• The reference driver. Its @record is PyAV's x11grab on the display, read in a thread, with the real cursor and wall-clock frame times, at up to 30 fps.

• The scaffold:

  • the Dockerfile installs the recording extra;
  • a new recording task records a scripted drive of the Fixture Desk;
  • the agent task records Claude Code's run.

Where this differs from the design

1. Five feature commits, not six. The SDK's declaration and its recorder share a module, so they're one commit. The e2e and review fixes follow as their own commits.
2. start returns right away, with started_at set to the time it was asked. The manifest carries each segment's first- and last-frame times.
3. start_computer_recording takes no scope or window yet. The driver records the whole display until M7.2.
4. The namespace is agentenv-recordings/<uuid>/, as snapshot_env names its own, rather than <instance>/<name>/. The instance and name are in the metadata and the universe id.
5. The probe's recording checks run at full. Reading a segment back needs computer-exec and python3, which full already requires.

Testing

• Unit: tst/unit 10,094 passed, and packages/agentenv-protocol/tests 844 passed. Recording coverage is 99%, and the protocol tests also pass on Python 3.10 with av 17.

  • The SDK has 32 tests on the recorder:
    • stream and capture sources;
    • rollover on time, size and gap;
    • the limits, the free-disk floor and expiry;
    • uploads and their retries;
    • sealing on end, reset and teardown.
  • The probe has 11 tests, covering each way a recording can fall short.
  • Core has 15 tests on the two steps and teardown.

• Integration (tst/integration/computer/scaffold_test.py, laptop): 3 passed.

  • A scripted drive of the Fixture Desk is recorded. Every screenshot the drive took matches the recording's frame at its captured_at.
  • Claude Code drives the desk under a recording.

• EC2 c6i.4xlarge (agent-environments-admin) at 57792e9:

  • the scaffold tests: 2 passed (the agent test skips without a model key);
  • the recording task:
    • 55 frames at 10.0 fps, none more than 100 ms apart;
    • each of the drive's 6 screenshots matches the frame nearest its captured_at, within 45 ms (thumbnail difference ≤ 1.21 of 255);
    • each capture lands about 33 ms after its action is sent;
  • a driver killed mid-recording (restart): registered from its namespace, with 3 segments plus the manifest, ended_by: error, and none missing;
  • the probe at full, 3 times: 51 passed and 1 not applicable each time, with computer-recording/v1 verified, and no recordings folder left behind.

• Chaos (16 checks on each machine), with every recording read back and decoded:

  	Laptop	EC2
  X server killed mid-recording	keeps recording: a 1.2 s gap, then a new segment	same, 1.2 s
  5 s segments	4 segments, gaps of 99–103 ms, every frame decodes	4 segments, gaps of 100 ms, every frame decodes
  Disk filled	ends as limit; what was recorded plays	same
  Limits	a repeated request_id joins the first recording; a third is refused with limit_exceeded; end again returns the same manifest	same
  64 clicks while recording at 30 fps	all land; p95 latency 8.9 ms with and without recording	all land; p95 14.2 → 15.0 ms
  Driver restarted	end answers invalid_argument; the closed segments stay	same

  CPU as a share of one core, on a still 1280×800 screen:

  fps	Laptop, custom @record	Laptop, default	EC2, custom @record	EC2, default
  10	11.9%	15.0%	12.1%	23.6%
  15	17.0%	22.2%	18.1%	35.5%
  30	34.7%	not possible (capped at 15)	35.7%	not possible
  Driver memory	110–119 MB	132–134 MB	121–140 MB	141–148 MB

• One hour at 10 fps (EC2, at 8eab232, the commit before the review fixes) with a click every 10 s:

  • 36,000 frames in 12 segments of 5 minutes; every frame decodes, and the largest gap between segments is 100 ms;

  • ended as limit at the default max_duration_s of 3,600 s;

  • driver memory: 100 MB at the start, rising 2–4 MB at each of the first six segment rollovers, then flat at 148 MB for the last 27 minutes.

  • At 57792e9, a 15-minute run of the same: 9,027 frames in 4 segments, every frame decodes, gaps ≤ 100 ms, and memory went from 100 to 141 MB in the same steps.

• An agent at work, recorded both ways (laptop; Claude Code on Sonnet 5.5, the Fixture Desk task, both runs passed):

  	Custom @record (x11grab)	Default (polling @capture)
  fps asked → achieved	30 → 30.0	15 (the fallback's cap) → 15.0
  Frames	613 over 20.4 s	273 over 18.1 s
  Frame gap, p95 / max	40 / 44 ms	76 / 88 ms
  Size	457 KB, 22 KB/s	432 KB, 24 KB/s
  Pointer	the real X cursor	an arrow drawn at @pointer_position
  Agent screenshots matching the frame at their captured_at	8/8, within 16 ms	7/7, within 26 ms
  Desktop CPU / memory during the run	31% / 116 MiB	25% / 139 MiB

  Each recording was registered as a FileArtifactUniverse. Each segment's SHA-256 and size match the manifest, and every frame decodes.

Found by the e2e

x11grab marks every frame a keyframe, and libx264 honoured that, so every encoded frame was a keyframe: 11.7 MB for 20 s. The encoder now clears pict_type before encoding, which brings the same recording to 423 KB. A unit test confirms it: 16 of 16 frames were keyframes without the fix.

Review

Greptile's 12 findings are fixed, and each thread has a reply:

• Ending and expiry:
  • sealing on end is shielded;
  • expiry runs on the wall clock;
  • the failing clock resets only after a save;
  • a recording's executor is released, and stopped recordings are bounded.
• Core steps:
  • a retried start reuses its namespace (uuid5 of the request);
  • fallback failures are caught;
  • the grant is capped at MAX_SEGMENTS;
  • a reused name is refused.
• Probe:
  • it reads segments in chunks, and a read that fails or comes back empty now fails the check;
  • it checks the recording's span and a minimum frame count.
• Constants: two magic numbers are now named constants.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; no blocking findings remain.

Summary

Adds display recording through computer-recording/v1, with segment uploads, saved artifacts, and cleanup before reset or teardown.

• Computer drivers can record display video as timed MP4 segments.
• Task runs can save computer recordings as file artifacts.
• The reference X11 driver can record its display with PyAV.
• The full computer probe checks recordings and their uploaded segments.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Start["Start recording"] --> Driver["Driver stream or capture"]
  Driver --> Segments["Close and upload segments"]
  Segments --> End["End, reset, or teardown"]
  End --> Retry["Retry missed uploads"]
  Retry --> Save["Save segments and manifest as artifacts"]

Loading

_{Reviews (4) · Last reviewed commit: "fix(computer): fail the probe on a segme..." · Reviewed by Greptile}

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	pypi/​av@​19.0.2
	pypi/​av@​18.1.0

View full report

[earakely-scale]

End-to-end results at 57792e9

The full numbers are in the Testing section above.

• CI: 13/13 checks pass. Greptile is at 5/5, and all 12 of its findings are fixed.

• EC2 c6i.4xlarge:

  • the scaffold tests pass;
  • the recorded drive lines up with its screenshots: each one matches the frame at its captured_at, within 45 ms;
  • a driver killed mid-recording is registered from its namespace;
  • the probe at full passes 51 checks, three times running;
  • chaos: 16/16;
  • a 15-minute soak passes, as did the earlier one-hour soak: 36,000 frames, every one decodes, and memory levels off at 148 MB.

• Laptop:

  • chaos: 16/16;
  • Claude Code (Sonnet 5.5) completed the Fixture Desk task twice under a recording, once with the reference driver's @record and once with the @capture fallback.

• @record against the fallback:

  	@record (x11grab)	Fallback (@capture)
  Frame rate	30.0 fps	15.0 fps (its cap)
  Frame gap, p95	40 ms	76 ms
  CPU at 15 fps (laptop)	17%	22%
  CPU at 15 fps (EC2)	18%	36%
  Pointer	the real cursor, an I-beam over text fields	an arrow the SDK draws

  Every screenshot in the agent's trajectory matches the video frame at its captured_at.