Skip to content

perf(selenium-devtools-py): push DOM mutations over BiDi instead of draining per command - #321

Draft
vishnuv688 wants to merge 8 commits into
mainfrom
worktree-py-drain-gating
Draft

perf(selenium-devtools-py): push DOM mutations over BiDi instead of draining per command#321
vishnuv688 wants to merge 8 commits into
mainfrom
worktree-py-drain-gating

Conversation

@vishnuv688

@vishnuv688 vishnuv688 commented Aug 20, 2026

Copy link
Copy Markdown
Member

What & why

DOM replay in the Python adapter was pulled: after every command that could have moved the page, the adapter ran an execute_script to drain the page-side mutation buffer. That drain is synchronous on the user's own command return path — every captured command paid a WebDriver round trip for it.

This branch removes it in two steps.

  1. f391dda — locator resolution stops triggering a drain. Resolving a locator reads the DOM and cannot change it, so the drain it triggered could only ever return an empty buffer: 4 of the 9 commands in a login flow. Same commit lowers selenium's BiDi reply poll interval (a sleep(interval) loop whose 0.1 default costs up to 100 ms per BiDi command however fast the browser answers).
  2. 835568f + 278c55a — the collector learned to push. core/bidi-preload.ts registers the document-start preload with a BiDi channel argument and parks the emit function on a window global before the collector's own module body runs, so the collector claims it and pushes every batch — including the document anchor, the first and largest payload of a document's life. With a channel open, the Python adapter drains nothing per command.

Mutations now also arrive as the page makes them, rather than batched at the next command boundary.

Type of change

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Polish (an improvement to an existing feature)
  • Breaking change (existing behavior changes for users)
  • Documentation
  • Internal (build, CI, dependencies, tooling)

Packages touched

  • shared (types and contracts) — the two channel names, single-sourced
  • core (framework-agnostic capture/reporting) — channel subscription, payload narrowing, ingestPushedMutations
  • elements
  • service
  • nightwatch-devtools
  • selenium-devtools (Selenium adapter) — 9 lines passing onMutations
  • backend
  • app
  • script (page-injected runtime) — the collector claims the sink and emits instead of buffering
  • selenium-devtools-py

Notes for reviewers

Why isn't this in core? It is. bidi-preload.ts owns the channel subscription and the payload narrowing, and the JS adapter change is pure wiring. The Python side restates the shape because Python cannot import TS — inherent to a second-language adapter. What matters is that the contract (COLLECTOR_SINK_GLOBAL, COLLECTOR_MUTATION_CHANNEL) is single-sourced in shared and generated into _contract.py, with gen_contract.py failing loudly if either name disappears.

The channel is opened only when the collector about to be installed names the sink global (bidi_preload.can_push). The wheel and the backend that serves the collector are versioned independently, so an older pairing is a real configuration rather than a hypothetical one — and a collector that never claims the sink would buffer everything while nothing drained it. Silent, total loss of DOM replay is the one failure this path must not have, so it is gated on the source text about to be installed, not on the subscription succeeding.

Python gates the per-command drain; JS Selenium does not (yet). Python's drain blocks the command; JS's drainAfterLiveCommand() is fire-and-forget into snapshotCaptures, so it costs driver traffic rather than test latency. JS Selenium currently pushes and drains — worth a follow-up, and simpler there, since its collector is inlined at build time and cannot skew. Nightwatch and the WDIO service do not push at all yet (Nightwatch passes no onMutations; the service uses browser.scriptAddPreloadScript directly rather than core's helper).

The session-end drain stays in every case. It is what collects a document the preload missed, and the batch the page puts back when its channel dies during teardown — the collector re-buffers on a failed emit, and teardown is exactly when the last mutations arrive.

Smaller things. add_preload_script replaces pin(), which cannot carry the channel argument; the selenium surface test now pins that it forwards no contexts (so registration stays global, covering documents created later) and that it passes arguments through. bidi.py's private _attr moved to utils.attr_or, now that the console mapping and the channel both read selenium payloads that arrive as either a generated dataclass or a raw params dict.

Verification

  • Unit: Python 373 tests; JS 2170 across 162 files; contract drift check in sync with shared.
  • Real headless Chrome, examples/selenium/python-test/login.py: the full login → logout flow captured, all three documents (/login/secure/login) anchored, and per-row replay correct — the get row renders /login with empty fields rather than the later filled state.
  • Outstanding: the browser run above confirms DOM replay is healthy, but does not by itself distinguish the push path from the drain fallback — both produce the same three anchors and the same per-row replay. Confirming which path ran means reading the adapter's own log line in the dashboard Console tab (collector registered at document-start, pushing mutations (BiDi preload)). To be pinned down before this leaves draft, along with a measured before/after against DEVTOOLS_BIDI=0.

Screenshots / recordings

No UI change.

…mands; lower selenium's BiDi reply poll interval; fix(selenium-devtools-py): stub bidi.attach with its real signature
await manager.addPreloadScript(
`async () => { ${await loadCollectorSource()} }`
channel
? `async (emit) => { window[${JSON.stringify(COLLECTOR_SINK_GLOBAL)}] = emit; ${source} }`
@vishnuv688 vishnuv688 changed the title perf(selenium-devtools-py): Devtools as a middleware perf(selenium-devtools-py): push DOM mutations over BiDi instead of draining per command Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants