perf(selenium-devtools-py): push DOM mutations over BiDi instead of draining per command - #321
Draft
vishnuv688 wants to merge 8 commits into
Draft
perf(selenium-devtools-py): push DOM mutations over BiDi instead of draining per command#321vishnuv688 wants to merge 8 commits into
vishnuv688 wants to merge 8 commits into
Conversation
…mands; lower selenium's BiDi reply poll interval; fix(selenium-devtools-py): stub bidi.attach with its real signature
| await manager.addPreloadScript( | ||
| `async () => { ${await loadCollectorSource()} }` | ||
| channel | ||
| ? `async (emit) => { window[${JSON.stringify(COLLECTOR_SINK_GLOBAL)}] = emit; ${source} }` |
# Conflicts: # packages/selenium-devtools-py/src/selenium_devtools/constants.py
…raining per command
…orktree-py-drain-gating # Conflicts: # packages/selenium-devtools-py/src/selenium_devtools/constants.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
DOM replay in the Python adapter was pulled: after every command that could have moved the page, the adapter ran an
execute_scriptto drain the page-side mutation buffer. That drain is synchronous on the user's own command return path — every captured command paid a WebDriver round trip for it.This branch removes it in two steps.
f391dda— locator resolution stops triggering a drain. Resolving a locator reads the DOM and cannot change it, so the drain it triggered could only ever return an empty buffer: 4 of the 9 commands in a login flow. Same commit lowers selenium's BiDi reply poll interval (asleep(interval)loop whose 0.1 default costs up to 100 ms per BiDi command however fast the browser answers).835568f+278c55a— the collector learned to push.core/bidi-preload.tsregisters the document-start preload with a BiDi channel argument and parks the emit function on a window global before the collector's own module body runs, so the collector claims it and pushes every batch — including the document anchor, the first and largest payload of a document's life. With a channel open, the Python adapter drains nothing per command.Mutations now also arrive as the page makes them, rather than batched at the next command boundary.
Type of change
Packages touched
shared(types and contracts) — the two channel names, single-sourcedcore(framework-agnostic capture/reporting) — channel subscription, payload narrowing,ingestPushedMutationselementsservicenightwatch-devtoolsselenium-devtools(Selenium adapter) — 9 lines passingonMutationsbackendappscript(page-injected runtime) — the collector claims the sink and emits instead of bufferingselenium-devtools-pyNotes for reviewers
Why isn't this in
core? It is.bidi-preload.tsowns the channel subscription and the payload narrowing, and the JS adapter change is pure wiring. The Python side restates the shape because Python cannot import TS — inherent to a second-language adapter. What matters is that the contract (COLLECTOR_SINK_GLOBAL,COLLECTOR_MUTATION_CHANNEL) is single-sourced insharedand generated into_contract.py, withgen_contract.pyfailing loudly if either name disappears.The channel is opened only when the collector about to be installed names the sink global (
bidi_preload.can_push). The wheel and the backend that serves the collector are versioned independently, so an older pairing is a real configuration rather than a hypothetical one — and a collector that never claims the sink would buffer everything while nothing drained it. Silent, total loss of DOM replay is the one failure this path must not have, so it is gated on the source text about to be installed, not on the subscription succeeding.Python gates the per-command drain; JS Selenium does not (yet). Python's drain blocks the command; JS's
drainAfterLiveCommand()is fire-and-forget intosnapshotCaptures, so it costs driver traffic rather than test latency. JS Selenium currently pushes and drains — worth a follow-up, and simpler there, since its collector is inlined at build time and cannot skew. Nightwatch and the WDIO service do not push at all yet (Nightwatch passes noonMutations; the service usesbrowser.scriptAddPreloadScriptdirectly rather than core's helper).The session-end drain stays in every case. It is what collects a document the preload missed, and the batch the page puts back when its channel dies during teardown — the collector re-buffers on a failed emit, and teardown is exactly when the last mutations arrive.
Smaller things.
add_preload_scriptreplacespin(), which cannot carry the channel argument; the selenium surface test now pins that it forwards nocontexts(so registration stays global, covering documents created later) and that it passesargumentsthrough.bidi.py's private_attrmoved toutils.attr_or, now that the console mapping and the channel both read selenium payloads that arrive as either a generated dataclass or a raw params dict.Verification
shared.examples/selenium/python-test/login.py: the full login → logout flow captured, all three documents (/login→/secure→/login) anchored, and per-row replay correct — thegetrow renders/loginwith empty fields rather than the later filled state.collector registered at document-start, pushing mutations (BiDi preload)). To be pinned down before this leaves draft, along with a measured before/after againstDEVTOOLS_BIDI=0.Screenshots / recordings
No UI change.