From 5f8b57f09366aeaaa9a5f95c017563c42ba39bae Mon Sep 17 00:00:00 2001 From: MendixMau Date: Fri, 16 Oct 2026 00:28:50 +0800 Subject: [PATCH 1/2] feat(demo-recording-polish): 60 fps capture, drawn cursor + ripple, eased scroll, zoom rendered after the take Playwright recordVideo encodes 25 fps vp8 at 1 Mbit/s realtime; demo films look stepped and the viewer cannot see clicks. New generic helpers (project-tests/e2e/demo-polish.js + selftest) capture with CDP screencast at device pixels, assemble a constant 60 fps, draw a Bezier cursor with a click ripple, scroll the real scroll container with easing, log clicks/focus/scrolls and plan a Screen Studio-style zoom from that log in a separate render pass. New skill skills/demo-recording-polish.md (rules, knobs, why zoom afterwards, measured results, failure modes), linked from record-demo-video.md; skill-routing row; install manifest. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 1 + README.md | 1 + ROUTING.md | 1 + agents/test-agent.md | 1 + bin/gate-check.sh | 2 +- bin/lib/install-manifest.sh | 2 +- bin/lib/skill-routing.tsv | 1 + project-tests/e2e/demo-polish.js | 315 ++++++++++++++++++++++ project-tests/e2e/demo-polish.selftest.js | 75 ++++++ skills/demo-recording-polish.md | 190 +++++++++++++ skills/record-demo-video.md | 9 +- 11 files changed, 595 insertions(+), 3 deletions(-) create mode 100644 project-tests/e2e/demo-polish.js create mode 100644 project-tests/e2e/demo-polish.selftest.js create mode 100644 skills/demo-recording-polish.md diff --git a/CHANGELOG.md b/CHANGELOG.md index b7202542..a19aa732 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -36,6 +36,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m - fix(verify-module): the graph, coverage and journeys rung budgets are variables (`VM_TMO_GRAPH`, `VM_TMO_COVERAGE`, `VM_TMO_JOURNEYS`; defaults unchanged at 300/300/900, like the existing `VM_TMO_CONFORMANCE`), so a large build plan or a loaded machine can raise the 300 s that reported a clean coverage run as a timeout (TK-008) — five-app lab project (Mendix 11.15, mxcli v0.26.0) - fix(app-restart): mask the value of any `Name=value` whose name holds password/secret/token/key (e.g. `--constant Module.XPassword=v`), not only values after secret-named flags; the printed `start:` line leaked them (TK-003) — five-app lab project (Mendix 11.15, mxcli v0.26.0) - learn(bug-logs): **six BUG-DRAFT entries from a five-app lab project (mxcli v0.26.0, Mendix 11.15)**: `check` expression gaps (formatDecimal, Decimal toString, raw backslash in replaceAll), declare/retrieve partial checks, microflow splice and alter variable scope, create without commit, Data grid 2 write defects, describe round-trip and DateTime localization. Each gives the workaround; none is filed upstream yet, upstream issue numbers named where they exist (#580, #1489, #1111, #891). Read on demand through `bin/bug-lookup.sh`. — MendixMau, five-app lab report TK-027 to TK-036 +- new(demo-recording-polish): **`project-tests/e2e/demo-polish.js` and `skills/demo-recording-polish.md`: demo films at 60 fps with a visible cursor, a click ripple, eased scrolling and a Screen Studio-style zoom rendered after the take.** Playwright's `recordVideo` encodes 25 fps vp8 at 1 Mbit/s realtime (playwright-core 1.59 `videoRecorder.js`). The helpers capture with CDP `Page.startScreencast` at device pixels (only `--force-device-scale-factor=2` makes the screencast send 3840x2160; `deviceScaleFactor` alone gives 1920x1080, measured) and assemble the timestamped JPEGs at a constant 60 fps (x264 crf 18, bt709). They also draw a Bezier cursor with a 55 px ripple and scroll the real scroll container with easeInOutCubic over `clamp(400 + 0.5 × px, 600, 1400)` ms. Clicks, focus notes and scrolls are logged to `events.json`; `render()` plans one zoom per shot from that log (capped at 1.75x, 400 ms lead, 700 ms push-in, at least a 2 s hold or until the narration line ends, 800 ms out, notes under 2.5 s apart merged, back at 1x before the next scroll or move, every take ends at 1x) and draws each frame from a fractional source rectangle in headless Chromium, so the zoom does not judder the way ffmpeg `zoompan` does. A fixed caption band can be kept unzoomed. `demo-polish.selftest.js` checks 20 camera rules. Both files are in `MXTK_PROJECT_TESTS`; `record-demo-video.md` rules 5 and 6 link to the new skill. Measured on one read-only 55 s Mendix scene recorded both ways: old 30/1 (source vp8 25/1), new 60/1 from 3840x2160 frames, unique frames in a 1 s scroll 15/32 → 36/62 and in a 1.3 s cursor glide 5/41 → 53/80. The app paints at about 50 Hz at 4K (frame gap p50 19.4 ms), so the real motion rate is roughly double, not 60. — MendixMau, three-app Mendix demo films 2026-10-15 - new(mxpatch, exec): **`project-bin/mxpatch.py`, the shared patch core, and a `mxcli fix hashes` check around every `exec.sh --patch` run: the two shipped workflow patchers no longer leave a stale ContentsHash.** mxpatch finds a unit by type + name (+ module) with a byte pre-filter, finds an element by Name, gets/sets a value by key, and writes the unit and its `.mpr` ContentsHash (base64 SHA-256 of the file) in one step; dry run by default, `already applied` on a re-run, and a refusal (exit 3) when the current value is not the expected old one. `wf-set-call-captions.py` and `wf-add-path-terminators.py` now write through it, refuse drift (a caption is replaced only while it is still the stamped microflow name or the `--expect-caption` value; `--expect-paths N` for terminators), and also take the `exec.sh --patch` form (` Module.Workflow`). The index is committed before the file swap, so a kill mid-write leaves a MISMATCH that `fix hashes` reports and a re-run repairs. `mxunit_bson.py` reads BSON 0x09 (UTC datetime). `exec.sh --patch` runs `mxcli fix hashes` (v0.25.0+, probed via `fix --help`) before and after the script and restores the snapshot on any issue the baseline did not have, before mxbuild, which cannot see it. Field evidence: captured mxcli v0.25.0/v0.26.0 runs on a public model (`tests/wave2/fixtures/mxpatch/CAPTURE.txt`): master's patchers leave a MISMATCH, the ported ones verify clean; the hash routine agrees with mxcli on 67 of 67 mismatched and 302 matching units of a second model. Widget-shape copying and pluggable-widget properties wait on a captured page unit. Field run ([#390](https://github.com/MendixMau/mxcli-project-toolkit/issues/390), mxcli v0.26.0, macOS): PASS, and the patch repaired a ContentsHash MISMATCH an old-style `--patch` had left on the unit it rewrote. Fixes from it: the backup of a rewritten unit is kept outside `mprcontents/` (in `$MXPATCH_BACKUP_DIR`, which `exec.sh --patch` points into the snapshot it just took, else a 0700 `/mxpatch-bak-/`; refused when it resolves inside `mprcontents/`, through a symlink or another case spelling; named with pid and counter so two writes in one second keep both), no longer as an untracked `.mxunit.bak` that `git add mprcontents` would commit; the BUILD-LOG row carries the hash bracket (`hashes: baseline 1 mismatch (pre-existing), verify clean (1 pre-existing repaired by this patch's rewrite)`); and the baseline message says that a unit the patch rewrites gets a fresh hash, and to repair the others with `fix hashes --repair`. A baseline MISMATCH that is still reported on a unit the patch rewrote counts as new (restore, exit 1), not as pre-existing. — jopterhorst ([#294](https://github.com/MendixMau/mxcli-project-toolkit/issues/294), [#295](https://github.com/MendixMau/mxcli-project-toolkit/issues/295)) - learn(pipelines/mendix-mpr/samples): **three minimal mxcli repros from the native-module bug hunt, the files the upstream draft cites** (split from [#293](https://github.com/MendixMau/mxcli-project-toolkit/pull/293), 6/6). `fsm-template-9.24.2/native-bughunt/` holds CAPTURE.txt and D1–D3, which are written from scratch in a `Repro` module: `describe entity` printing `index ( desc);`, the native Repeater `content` slot, and no flow writable to a 9.24.2 project. They are the repros for U3, U2 and U14 in `bug-logs/pending-github-issues/2026-10-12-mxcli-0.26-round-trip-report.md`. The other 472 sample files of #293 had no citing reader, because the skills and Rehost splits state their field results inline, so none of them ships. — MendixMau, Decompose/Rehost lab runs 2026-10-12 - learn(inbox, pending-github-issues): **ten field-note drops from the Decompose/Rehost lab runs, and one not-filed draft of 53 mxcli 0.26 defects for mendixlabs/mxcli** (split from [#293](https://github.com/MendixMau/mxcli-project-toolkit/pull/293), 5/6). `contrib/inbox/2026-10-1{2,3}-*.md`: increments 1–8 of the split-a-monolith runs on a public Marketplace template (FSM 9.24.2), the native-module bug hunt, and the GenAI Showcase regeneration (0.26.0 and the 2026-10-13 nightly); each now has the template's front matter and is at most 600 words. `bug-logs/pending-github-issues/2026-10-12-mxcli-0.26-round-trip-report.md`: the U1–U53 table the inbox drops refer to, marked DRAFT — NOT FILED, to be split into one issue per defect and retested before anything is posted. No BUG numbers assigned; triage promotes. — MendixMau, Decompose/Rehost lab runs 2026-10-12/13 diff --git a/README.md b/README.md index f7f6f179..83102b0a 100644 --- a/README.md +++ b/README.md @@ -677,6 +677,7 @@ Every mxcli project has a `.ai-context/skills/` directory (bundled by `mxcli ini | After every module's CONFIRM stage — counts proven modules since the last cluster/full coherence pass and exits DUE once the threshold is reached, so the cadence isn't left to memory | `project-bin/coherence-cadence.sh` | | Turning an already-rigorous run into a narrated proof a stakeholder can trust without running anything | `skills/e2e-evidence-report.md` | | Recording a narrated screen-capture demo of a running app for a human to watch — opening on the app instead of a blank frame, and keeping captions synced to the pixels | `skills/record-demo-video.md` | +| A demo film someone will watch looks captured, not produced — 25 fps smear, scroll jumps, no visible pointer, a dense page nobody can read: 2x CDP screencast at 60 fps, drawn cursor and click ripple, eased scroll, and a zoom rendered afterwards from the click log (never a CSS transform in the page) | `skills/demo-recording-polish.md` | | Sending a demo guide, screenshots or a quickstart doc OUTSIDE the repo — to a customer, prospect or reviewer: relative image paths and "the name shown above" both break the moment the file travels alone, and the usernames a login panel displays are not the ones it accepts | `skills/share-demo-package.md` | | Running lint as a gate rather than a report — per-rule ratchet against a committed baseline, plus the crash and collapse guards that stop a blind rule passing | `project-bin/lint-gate.sh` | | Reading a lint result, or writing/repairing any .star rule — lint's failure mode is a confident clean pass, so 0 findings is a claim needing evidence | `skills/lint-that-actually-runs.md` | diff --git a/ROUTING.md b/ROUTING.md index 22b4b0c3..108dfa86 100644 --- a/ROUTING.md +++ b/ROUTING.md @@ -229,6 +229,7 @@ picks the row up. That is the whole procedure — there is no second list to rem | After every module's CONFIRM stage — counts proven modules since the last cluster/full coherence pass and exits DUE once the threshold is reached, so the cadence isn't left to memory | `project-bin/coherence-cadence.sh` | mdl,gate,review | 5,6 | ondemand | | Turning an already-rigorous run into a narrated proof a stakeholder can trust without running anything | `skills/e2e-evidence-report.md` | test,review | 6 | ondemand | | Recording a narrated screen-capture demo of a running app for a human to watch — opening on the app instead of a blank frame, and keeping captions synced to the pixels | `skills/record-demo-video.md` | test | 6 | ondemand | +| A demo film someone will watch looks captured, not produced — 25 fps smear, scroll jumps, no visible pointer, a dense page nobody can read: 2x CDP screencast at 60 fps, drawn cursor and click ripple, eased scroll, and a zoom rendered afterwards from the click log (never a CSS transform in the page) | `skills/demo-recording-polish.md` | test | 6 | ondemand | | Sending a demo guide, screenshots or a quickstart doc OUTSIDE the repo — to a customer, prospect or reviewer: relative image paths and "the name shown above" both break the moment the file travels alone, and the usernames a login panel displays are not the ones it accepts | `skills/share-demo-package.md` | test,review | 6,7 | ondemand | | Running lint as a gate rather than a report — per-rule ratchet against a committed baseline, plus the crash and collapse guards that stop a blind rule passing | `project-bin/lint-gate.sh` | mdl,gate | 5,6 | ondemand | | Reading a lint result, or writing/repairing any .star rule — lint's failure mode is a confident clean pass, so 0 findings is a claim needing evidence | `skills/lint-that-actually-runs.md` | gate,review | 5,6 | ondemand | diff --git a/agents/test-agent.md b/agents/test-agent.md index 60c6f1df..3d0235ef 100644 --- a/agents/test-agent.md +++ b/agents/test-agent.md @@ -64,6 +64,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/agent-permission-friction.md` | Any refused, denied or blocked command | | `skills/e2e-evidence-report.md` | Turning an already-rigorous run into a narrated proof a stakeholder can trust without running anything | | `skills/record-demo-video.md` | Recording a narrated screen-capture demo of a running app for a human to watch | +| `skills/demo-recording-polish.md` | A demo film someone will watch looks captured, not produced | | `skills/share-demo-package.md` | Sending a demo guide, screenshots or a quickstart doc OUTSIDE the repo | | `skills/improvement-register.md` | Any review pass that runs more than once | | `skills/journey-examples.md` | Writing an actual .journey.json | diff --git a/bin/gate-check.sh b/bin/gate-check.sh index 2a79dbcc..015877d9 100755 --- a/bin/gate-check.sh +++ b/bin/gate-check.sh @@ -1772,7 +1772,7 @@ stage_protocol_paths() { 3) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-design.md skills/dispatching.md skills/small-project-tier.md skills/mendix-best-practices-index.md skills/decompose-app.md skills/mendix-service-seam.md skills/layering-review.md skills/architecture-blueprint.md skills/modularize-domain.md skills/design-artifacts.md skills/interaction-model.md skills/brd-to-build-plan.md skills/workflow-structure-rules.md skills/learned-mdl-cannot-express.md" ;; 4) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-build.md skills/agent-roles.md skills/dispatching.md skills/small-project-tier.md skills/module-brief.md skills/module-folder-convention.md skills/brd-to-build-plan.md skills/coverage-ledger.md skills/workflow-structure-rules.md skills/rest-integration-first-time-right.md skills/learned-constants-and-secrets.md" ;; 5|build-ready) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/agent-roles.md skills/dispatching.md skills/module-brief.md skills/learned-mdl-preflight.md skills/module-folder-convention.md skills/learned-microflow-patterns.md skills/microflow-preflight.md skills/mendix-best-practices-index.md skills/ui-preflight-pages.md skills/design-spacing.md skills/learned-stylegallery.md skills/ui-loop.md skills/module-review.md skills/testing-shape.md skills/decompose-app.md skills/mendix-service-seam.md skills/microflow-loop-antipatterns.md skills/rehost-app.md skills/iterative-build-loop.md skills/mdl-cookbook-microflows.md skills/build/mdl/oneshot-mdl-method.md skills/learned-page-patterns.md skills/oneshot-page-structure-patterns.md skills/mendix-agents.md skills/mendix-agent-ui.md skills/mendix-agent-setup.md skills/fixture-seeding.md skills/journey-proof.md skills/unhappy-path-testing.md skills/monkey-test.md skills/e2e-locators-that-lie.md skills/report-schema.md skills/harness-architecture.md skills/process-coherence-pass.md skills/lint-that-actually-runs.md skills/improvement-register.md skills/journey-examples.md skills/wiring-sweep.md skills/learned-workflow-patterns.md skills/workflow-structure-rules.md skills/rest-integration-first-time-right.md skills/bug-submission-checklist.md skills/empty-widget-triage.md skills/learned-sidebar-collapse-icons.md skills/learned-popup-navigation.md skills/learned-datagrid-customcontent-binding.md skills/learned-popup-feedback-pattern.md skills/learned-mdl-cannot-express.md skills/learned-css-that-never-applied.md skills/learned-detection-gaps.md skills/learned-dg2-patterns.md skills/learned-file-upload-widget.md skills/security-is-not-a-later-script.md skills/learned-local-db-confusion.md skills/full-harness-audit.md skills/test-result-audit.md skills/finding-disposition.md skills/preview-over-hub-tunnel.md skills/walking-skeleton.md skills/platform-link.md skills/teamserver-alignment.md skills/learned-constants-and-secrets.md" ;; - 6) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-cutover.md skills/dispatching.md skills/mendix-best-practices-index.md skills/module-review.md skills/testing-shape.md skills/existing-app-assurance.md skills/decompose-app.md skills/app-analysis.md skills/module-dependency-review.md skills/microflow-loop-antipatterns.md skills/rehost-app.md skills/qa-loop-goal-pattern.md skills/mendix-agent-setup.md skills/e2e-harness-base.md skills/learned-db-assertions.md skills/fixture-seeding.md skills/journey-proof.md skills/unhappy-path-testing.md skills/monkey-test.md skills/e2e-locators-that-lie.md skills/learned-skill-ux-audit.md skills/learned-skill-scope-delta.md skills/report-schema.md skills/harness-architecture.md skills/process-coherence-pass.md skills/e2e-evidence-report.md skills/record-demo-video.md skills/share-demo-package.md skills/lint-that-actually-runs.md skills/improvement-register.md skills/journey-examples.md skills/wiring-sweep.md skills/workflow-structure-rules.md skills/bug-submission-checklist.md skills/empty-widget-triage.md skills/anonymize-client-app-for-demo.md skills/learned-css-that-never-applied.md skills/learned-detection-gaps.md skills/learned-local-db-confusion.md skills/full-harness-audit.md skills/test-result-audit.md skills/finding-disposition.md skills/handoff-to-studio-pro.md skills/preview-over-hub-tunnel.md skills/platform-link.md skills/teamserver-alignment.md skills/learned-constants-and-secrets.md" ;; + 6) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-cutover.md skills/dispatching.md skills/mendix-best-practices-index.md skills/module-review.md skills/testing-shape.md skills/existing-app-assurance.md skills/decompose-app.md skills/app-analysis.md skills/module-dependency-review.md skills/microflow-loop-antipatterns.md skills/rehost-app.md skills/qa-loop-goal-pattern.md skills/mendix-agent-setup.md skills/e2e-harness-base.md skills/learned-db-assertions.md skills/fixture-seeding.md skills/journey-proof.md skills/unhappy-path-testing.md skills/monkey-test.md skills/e2e-locators-that-lie.md skills/learned-skill-ux-audit.md skills/learned-skill-scope-delta.md skills/report-schema.md skills/harness-architecture.md skills/process-coherence-pass.md skills/e2e-evidence-report.md skills/record-demo-video.md skills/demo-recording-polish.md skills/share-demo-package.md skills/lint-that-actually-runs.md skills/improvement-register.md skills/journey-examples.md skills/wiring-sweep.md skills/workflow-structure-rules.md skills/bug-submission-checklist.md skills/empty-widget-triage.md skills/anonymize-client-app-for-demo.md skills/learned-css-that-never-applied.md skills/learned-detection-gaps.md skills/learned-local-db-confusion.md skills/full-harness-audit.md skills/test-result-audit.md skills/finding-disposition.md skills/handoff-to-studio-pro.md skills/preview-over-hub-tunnel.md skills/platform-link.md skills/teamserver-alignment.md skills/learned-constants-and-secrets.md" ;; 7) echo "skills/interview-protocol.md skills/change-intake.md skills/change-intake-detail.md skills/grill-mode.md skills/checkpoints/checkpoint-template.md skills/checkpoints/checkpoint-cutover.md skills/close-the-loop.md skills/share-demo-package.md skills/handoff-to-studio-pro.md skills/platform-link.md skills/teamserver-alignment.md skills/deploy-to-sandbox.md" ;; *) echo "" ;; # diff --git a/bin/lib/install-manifest.sh b/bin/lib/install-manifest.sh index 13a78191..bcfa1511 100644 --- a/bin/lib/install-manifest.sh +++ b/bin/lib/install-manifest.sh @@ -191,7 +191,7 @@ MXTK_LINT_RULES_NOINSTALL="README.md STOCK-HASHES.txt" # # project.config.template.js is listed separately because it is the one file a project is # EXPECTED to edit — installing it over an edited copy would silently revert the port. -MXTK_PROJECT_TESTS="config.js helpers.js settle.js otel.js journey-runner.js named-controls.js journey-runner.selftest.js journey-rung4-scope.test.js monkey.js monkey.selftest.js page-audit.js page-audit-rules.js design-audit.js full-app-walkthrough.js report-normalize.js report-render.js review-report.js example.journey.json" +MXTK_PROJECT_TESTS="config.js helpers.js settle.js otel.js journey-runner.js named-controls.js journey-runner.selftest.js journey-rung4-scope.test.js monkey.js monkey.selftest.js page-audit.js page-audit-rules.js design-audit.js full-app-walkthrough.js report-normalize.js report-render.js review-report.js demo-polish.js demo-polish.selftest.js example.journey.json" MXTK_PROJECT_TESTS_TEMPLATE="project.config.template.js" # Deliberately not installed. Same contract as the lists above. diff --git a/bin/lib/skill-routing.tsv b/bin/lib/skill-routing.tsv index ca7d72ec..cf7795c9 100644 --- a/bin/lib/skill-routing.tsv +++ b/bin/lib/skill-routing.tsv @@ -188,6 +188,7 @@ coherence-cadence project-bin/coherence-cadence.sh After every module's CONFIRM build-plan-status project-bin/build-plan-status.sh After marking a module done, or any time "how much is built vs proven" is asked — renders build-plan.html from done- prefixes and verify-module.sh/improvement-register.md, kept as two honestly separate views; --json writes architecture/build-plan.json parsed from build-plan.md's Phase headings (a plan with no Phase headings gets no file) architect,gate,review 4,5,6 ondemand build e2e-evidence-report skills/e2e-evidence-report.md Turning an already-rigorous run into a narrated proof a stakeholder can trust without running anything test,review 6 ondemand verify record-demo-video skills/record-demo-video.md Recording a narrated screen-capture demo of a running app for a human to watch — opening on the app instead of a blank frame, and keeping captions synced to the pixels test 6 ondemand verify +demo-recording-polish skills/demo-recording-polish.md A demo film someone will watch looks captured, not produced — 25 fps smear, scroll jumps, no visible pointer, a dense page nobody can read: 2x CDP screencast at 60 fps, drawn cursor and click ripple, eased scroll, and a zoom rendered afterwards from the click log (never a CSS transform in the page) test 6 ondemand verify share-demo-package skills/share-demo-package.md Sending a demo guide, screenshots or a quickstart doc OUTSIDE the repo — to a customer, prospect or reviewer: relative image paths and "the name shown above" both break the moment the file travels alone, and the usernames a login panel displays are not the ones it accepts test,review 6,7 ondemand verify mpr-corruption-and-sp-load-errors skills/mpr-corruption-and-sp-load-errors.md Studio Pro will not load the project, or the .mpr looks gutted — recover before relaunching SP, never git checkout mdl,gate - ondemand diagnose lint-gate project-bin/lint-gate.sh Running lint as a gate rather than a report — per-rule ratchet against a committed baseline, plus the crash and collapse guards that stop a blind rule passing mdl,gate 5,6 ondemand verify diff --git a/project-tests/e2e/demo-polish.js b/project-tests/e2e/demo-polish.js new file mode 100644 index 00000000..d322efbc --- /dev/null +++ b/project-tests/e2e/demo-polish.js @@ -0,0 +1,315 @@ +'use strict'; +// demo-polish.js — polished demo capture for Playwright: smooth 60 fps, a visible cursor with a click ripple, +// eased scrolling, and a camera that zooms in on dense moments AFTER the take (Screen Studio style). +// +// const polish = require('./demo-polish'); +// const ctx = await browser.newContext({viewport:{width:1920,height:1080}, deviceScaleFactor:2}); +// await polish.prepare(ctx); // cursor + ripple + scroll helpers, survive navigation +// const page = await ctx.newPage(); ...log in, set up... +// const rec = await polish.startCapture(page, {dir:'take/'}); // CDP screencast, timestamped JPEGs +// await polish.scrollTo(page, locator); // eased, on the real scroll container +// await polish.click(page, locator, {zoom: cardLocator}); // glide, pause, ripple, click; camera note +// await polish.focus(page, denseCard); // camera note without a click +// await rec.stop(); // writes frames.json + events.json +// await polish.render('take/', 'take.mp4', {fps:60}); // composite: eased push-in/hold/zoom-out, sub-pixel +// polish.assemble('take/', 'flat.mp4'); // same frames, no camera (fast, ffmpeg only) +// +// Why zoom afterwards: a CSS transform on the page breaks position:fixed bands, popups and hit-testing, and every +// retake costs a re-shoot. The recorder only logs {t, bbox, kind}; the camera is a pure function of the log, so +// re-framing is a re-render. Plain ffmpeg zoompan rounds the crop to whole pixels and judders; the compositor here +// draws each output frame on a canvas with a fractional source rectangle instead. +// +// Needs: playwright (chromium) and ffmpeg on PATH (or FFMPEG=/path). No other dependencies. +const fs = require('fs'), path = require('path'), {spawn, execFileSync} = require('child_process'); +const FF = process.env.FFMPEG || 'ffmpeg'; +const nap = ms => new Promise(r => setTimeout(r, ms)); +const clamp = (v, a, b) => Math.max(a, Math.min(b, v)); +const ease = t => t < .5 ? 4 * t * t * t : 1 - Math.pow(-2 * t + 2, 3) / 2; // easeInOutCubic +const rnd = (a, b) => a + Math.random() * (b - a); + +// Knobs. Every number is a default a caller can override per call; the skill explains where each comes from. +const KNOBS = { + cursorPx: 30, // arrow height at 1080p + moveMs: [500, 900], // per cursor move, scaled by distance + preClickMs: [150, 250], // settle on the target before pressing + ripplePx: 55, rippleMs: 400, // 0 -> 55 px, ease-out, opacity .6 -> 0 + rippleColor: 'rgba(17,24,39,.95)', rippleFill: 'rgba(17,24,39,.18)', // dark ring, white halo (in the overlay) + scrollMs: px => clamp(400 + 0.5 * px, 600, 1400), + afterScrollMs: [400, 600], + zoomMax: 1.75, zoomMin: 1.25, // never past 1.75; a region that only earns < 1.25 gets no zoom at all + zoomMarginPx: 48, // breathing room around the framed region (CSS px) + leadMs: 400, // push-in starts this long before the click + inMs: 700, holdMs: 2000, outMs: 800, + mergeMs: 2500, // camera notes closer than this share one shot + clickRegion: [960, 540], // zoom:true on a click frames at least this much around it (~1.7x), not the bare button + minHoldMs: 800, // a shot that cannot hold this long before the next scroll/move is dropped, not squeezed +}; + +// ---------------------------------------------------------------- in-page overlay (runs in the browser) +const OVERLAY = `(()=>{if(window.__dp)return;const K=${JSON.stringify({c: KNOBS.cursorPx, r: KNOBS.ripplePx, rm: KNOBS.rippleMs, rc: KNOBS.rippleColor, rf: KNOBS.rippleFill})}; + const ease=t=>t<.5?4*t*t*t:1-Math.pow(-2*t+2,3)/2;let cur=null,pos={x:-100,y:-100}; + const svg=''; + const ensure=()=>{if(cur&&cur.isConnected)return cur;cur=document.createElement('div');cur.id='__dp-cursor';cur.innerHTML=svg; + cur.style.cssText='position:fixed;left:0;top:0;z-index:2147483647;pointer-events:none;filter:drop-shadow(0 2px 3px rgba(0,0,0,.35));will-change:transform'; + (document.body||document.documentElement).append(cur);place(pos.x,pos.y);return cur;}; + const place=(x,y)=>{pos={x,y};if(cur)cur.style.transform='translate('+(x-1.5*K.c/30)+'px,'+(y-1.5*K.c/30)+'px)';}; + const frame=()=>new Promise(r=>requestAnimationFrame(r)); + async function move(x,y,ms){ensure();const x0=pos.x,y0=pos.y,dx=x-x0,dy=y-y0,d=Math.hypot(dx,dy)||1; + // one cubic Bezier with both handles bowed to the same side: a slight arc, never a ruler line + const b=Math.min(80,d*.18)*(Math.random()<.5?-1:1),nx=-dy/d*b,ny=dx/d*b; + const p1={x:x0+dx*.3+nx,y:y0+dy*.3+ny},p2={x:x0+dx*.7+nx*.6,y:y0+dy*.7+ny*.6};const t0=performance.now(); + for(;;){await frame();const u=Math.min(1,(performance.now()-t0)/ms),t=ease(u),m=1-t; + place(m*m*m*x0+3*m*m*t*p1.x+3*m*t*t*p2.x+t*t*t*x,m*m*m*y0+3*m*m*t*p1.y+3*m*t*t*p2.y+t*t*t*y);if(u>=1)break;}} + function ripple(x,y){const r=document.createElement('div'); + // dark ring + white halo: visible on light and dark UI, and never the same colour as a page's own highlight + r.style.cssText='position:fixed;left:'+x+'px;top:'+y+'px;width:'+K.r+'px;height:'+K.r+'px;margin:-'+K.r/2+'px 0 0 -'+K.r/2+'px;border-radius:50%;border:4px solid '+K.rc+';box-shadow:0 0 0 2px rgba(255,255,255,.9),inset 0 0 0 2px rgba(255,255,255,.9);background:'+K.rf+';z-index:2147483646;pointer-events:none'; + document.body.append(r);r.animate([{transform:'scale(0.15)',opacity:.6},{transform:'scale(.75)',opacity:.45,offset:.55},{transform:'scale(1)',opacity:0}],{duration:K.rm,easing:'cubic-bezier(.22,.61,.36,1)',fill:'forwards'}).onfinish=()=>r.remove();} + const scroller=el=>{for(let n=el.parentElement;n;n=n.parentElement){const s=getComputedStyle(n);if(/(auto|scroll)/.test(s.overflowY)&&n.scrollHeight>n.clientHeight+1)return n;}return document.scrollingElement;}; + // Eased scroll of the element's real scroll container; returns how far it went so the caller can time the wait. + async function scroll(el,block,msFor){if(getComputedStyle(el).display==='contents'&&el.firstElementChild)el=el.firstElementChild; + const sc=scroller(el),root=sc===document.scrollingElement,vr=root?{top:0,height:innerHeight}:sc.getBoundingClientRect(); + const r=el.getBoundingClientRect(),pad=Number(document.documentElement.dataset.dpBottomPad||0),vh=vr.height-(root?pad:0); + let off=block==='start'?r.top-vr.top-24:r.top-vr.top-(vh-Math.min(r.height,vh))/2; + const from=sc.scrollTop,to=Math.max(0,Math.min(sc.scrollHeight-sc.clientHeight,from+off)),px=Math.abs(to-from);if(px<2)return 0; + const ms=msFor(px),t0=performance.now();for(;;){await frame();const u=Math.min(1,(performance.now()-t0)/ms);sc.scrollTop=from+(to-from)*ease(u);if(u>=1)break;}return px;} + window.__dp={ensure,place,move,ripple,scroll,pos:()=>pos}; + if(document.readyState==='loading')document.addEventListener('DOMContentLoaded',ensure);else ensure();})()`; + +async function prepare(ctx) { await ctx.addInitScript(OVERLAY); } +// Page.startScreencast sends frames at the CSS viewport size, whatever the context's deviceScaleFactor. Only this +// launch flag makes it send device pixels (3840x2160 for a 1920x1080 viewport) — the headroom a zoom needs to stay sharp. +const launchArgs = (dsf = 2) => [`--force-device-scale-factor=${dsf}`]; +const st = page => page.__dp || (page.__dp = {x: null, y: null, rec: null}); +async function ready(page) { + await page.evaluate(OVERLAY); + const s = st(page); + if (s.x == null) { const v = page.viewportSize(); s.x = v.width * .62; s.y = v.height * .58; } + await page.evaluate(([x, y]) => { window.__dp.ensure(); window.__dp.place(x, y); }, [s.x, s.y]); +} +// Tell the scroller about a fixed band at the bottom (a caption strip) so "center" means the visible area. +async function bottomPad(page, px) { await page.evaluate(px => { document.documentElement.dataset.dpBottomPad = px; }, px); } + +const boxOf = async loc => { const b = await loc.boundingBox(); return b && {x: b.x, y: b.y, width: b.width, height: b.height}; }; +function note(page, kind, bbox, extra) { + const r = st(page).rec; if (r && bbox) r.events.push({t: Date.now(), kind, bbox, ...extra}); +} + +async function moveTo(page, x, y, o = {}) { + await ready(page); const s = st(page), d = Math.hypot(x - s.x, y - s.y); + const [a, b] = o.moveMs || KNOBS.moveMs, ms = o.ms || clamp(a + d * .45, a, b); + await page.evaluate(([x, y, ms]) => window.__dp.move(x, y, ms), [x, y, ms]); + s.x = x; s.y = y; await page.mouse.move(x, y); // the real pointer lands too, so hover states are true +} + +async function scrollTo(page, loc, o = {}) { + await ready(page); const t0 = Date.now(); + const px = await loc.evaluate((el, [b]) => window.__dp.scroll(el, b, px => Math.max(600, Math.min(1400, 400 + .5 * px))), [o.block || 'center']); + if (px) { note(page, 'scroll', await boxOf(loc), {px, t0, t1: Date.now()}); await nap(rnd(...KNOBS.afterScrollMs)); } + return px; +} + +// zoom: true (frame the clicked element), a Locator (frame that region), or false/undefined (no camera note). +async function click(page, loc, o = {}) { + await loc.waitFor({state: 'visible', timeout: o.timeout || 15000}); + await scrollTo(page, loc, o); + const b = await boxOf(loc); if (!b) throw new Error('click target has no box'); + const x = b.x + b.width * clamp(o.fx ?? .5, .1, .9), y = b.y + b.height * clamp(o.fy ?? .5, .2, .8); + const t0 = Date.now(); await moveTo(page, x, y, o); + await nap(rnd(...KNOBS.preClickMs)); + const [rw, rh] = KNOBS.clickRegion, V = page.viewportSize(), tClick = Date.now(); + await page.evaluate(([x, y]) => window.__dp.ripple(x, y), [x, y]); + await page.mouse.click(x, y); + // A Locator zoom region is measured AFTER the click: it is often what the click reveals (a tab pane, a popup), + // invisible before it. The note keeps the click's time, so the push-in still starts before the press. + let zb = null; + if (o.zoom === true) zb = {x: clamp(x - Math.max(rw, b.width) / 2, 0, V.width), y: clamp(y - Math.max(rh, b.height) / 2, 0, V.height), width: Math.max(rw, b.width), height: Math.max(rh, b.height)}; + else if (o.zoom) { const z = o.zoom.first ? o.zoom.first() : o.zoom; + if (await z.waitFor({state: 'visible', timeout: o.zoomWait || 5000}).then(() => true, () => false)) { await nap(300); zb = await boxOf(z); } } + const r = st(page).rec; if (r) r.events.push({t: tClick, kind: 'click', bbox: b, x, y, t0, zoom: zb || undefined}); +} + +async function focus(page, loc, o = {}) { + await scrollTo(page, loc, o); note(page, 'focus', await boxOf(loc), {zoom: await boxOf(loc), until: o.until}); +} +// The narration line ends at t (epoch ms): hold the current shot at least until then. +function holdUntil(page, t) { const r = st(page).rec; if (r) r.events.push({t: Date.now(), kind: 'hold', until: t}); } + +// ---------------------------------------------------------------- capture (CDP screencast) +function jpegSize(file) { // width x height from the SOF marker; no image library needed + const b = fs.readFileSync(file); for (let i = 2; i < b.length - 9;) { if (b[i] !== 0xFF) { i++; continue; } const m = b[i + 1]; + if (m >= 0xC0 && m <= 0xC3) return {width: b.readUInt16BE(i + 7), height: b.readUInt16BE(i + 5)}; i += 2 + b.readUInt16BE(i + 2); } return null; +} +async function startCapture(page, o = {}) { + const dir = path.resolve(o.dir); fs.rmSync(dir, {recursive: true, force: true}); fs.mkdirSync(path.join(dir, 'frames'), {recursive: true}); + const v = page.viewportSize(), dsf = await page.evaluate(() => devicePixelRatio); + const cdp = await page.context().newCDPSession(page); + const frames = [], events = [], pending = []; + let n = 0, skew = []; + cdp.on('Page.screencastFrame', f => { + const file = `f${String(n++).padStart(6, '0')}.jpg`, t = f.metadata.timestamp * 1000; + skew.push(Date.now() - t); + pending.push(fs.promises.writeFile(path.join(dir, 'frames', file), Buffer.from(f.data, 'base64'))); + frames.push({file, t}); + cdp.send('Page.screencastFrameAck', {sessionId: f.sessionId}).catch(() => {}); + }); + await ready(page); + await cdp.send('Page.startScreencast', {format: 'jpeg', quality: o.quality || 88, maxWidth: Math.round(v.width * dsf), maxHeight: Math.round(v.height * dsf), everyNthFrame: 1}); + // The screencast only sends a frame when the compositor paints; nudge one so the take has a first frame. + await page.evaluate(() => window.__dp.place(window.__dp.pos().x + .01, window.__dp.pos().y)); + const rec = {dir, events, frames, t0: Date.now(), cuts: [], + begin() { rec.t0 = Date.now(); }, cut(a, b) { if (b > a) rec.cuts.push([a, b]); }, + async stop() { + // hold the last picture until now, then close the stream + await page.evaluate(() => window.__dp.place(window.__dp.pos().x, window.__dp.pos().y + .01)).catch(() => {}); + await nap(150); const end = Date.now(); + await cdp.send('Page.stopScreencast').catch(() => {}); await Promise.all(pending); await cdp.detach().catch(() => {}); + skew.sort((a, b) => a - b); const med = skew[skew.length >> 1] || 0; + const meta = {viewport: v, dsf, frameSize: frames.length ? jpegSize(path.join(dir, 'frames', frames[0].file)) : null, t0: rec.t0, end, cuts: rec.cuts, frames: frames.length, clockSkewMs: med, + captureFps: frames.length / Math.max(.001, (end - rec.t0) / 1000)}; + fs.writeFileSync(path.join(dir, 'frames.json'), JSON.stringify({meta, frames}, null, 0)); + fs.writeFileSync(path.join(dir, 'events.json'), JSON.stringify(events, null, 1)); + st(page).rec = null; return meta; + }}; + st(page).rec = rec; return rec; +} + +// ---------------------------------------------------------------- timeline helpers +function load(dir) { + const {meta, frames} = JSON.parse(fs.readFileSync(path.join(dir, 'frames.json'))); + const events = JSON.parse(fs.readFileSync(path.join(dir, 'events.json'))); + return {meta, frames, events}; +} +// Map wall-clock ms to output ms (cuts removed); null when inside a cut. +function mapper(meta) { + const cuts = [...meta.cuts].sort((a, b) => a[0] - b[0]); + const toOut = t => { let o = t - meta.t0; for (const [a, b] of cuts) { if (t >= b) o -= b - a; else if (t > a) return null; } return o; }; + const toWall = o => { let t = meta.t0 + o; for (const [a, b] of cuts) if (t >= a) t += b - a; return t; }; + const len = toOut(meta.end); + return {toOut, toWall, len}; +} + +// Shots: one zoom per shot, notes closer than mergeMs merged, nothing overlaps, every take ends zoomed out. +function shots(meta, events, k = {}) { + k = {...KNOBS, ...k}; const {toOut, len} = mapper(meta), V = meta.viewport, kb = k.keepBottom || 0, vis = (V.height - kb) / V.height; + // with a kept bottom band only the area above it shows the zoomed picture, so frame inside that area + const fit = b => Math.min(V.width / (b.width + 2 * k.zoomMarginPx), (V.height - kb) / (b.height + 2 * k.zoomMarginPx)); + const union = (a, b) => { const x = Math.min(a.x, b.x), y = Math.min(a.y, b.y); return {x, y, width: Math.max(a.x + a.width, b.x + b.width) - x, height: Math.max(a.y + a.height, b.y + b.height) - y}; }; + const notes = events.filter(e => e.zoom).map(e => ({t: toOut(e.t), box: e.zoom, until: e.until && toOut(e.until)})).filter(e => e.t != null).sort((a, b) => a.t - b.t); + // Boundaries: the camera must be back at 1x before the page scrolls or the cursor sets off for an unzoomed click, + // or the hold frames a region the content has already left. + const bounds = events.filter(e => (e.kind === 'scroll' && e.px > 40) || (e.kind === 'click' && !e.zoom)).map(e => ({t: toOut(e.t0 || e.t), end: toOut(e.t1 || e.t)})).filter(e => e.t != null).sort((a, b) => a.t - b.t); + const holds = events.filter(e => e.kind === 'hold').map(e => ({t: toOut(e.t), until: toOut(e.until)})).filter(e => e.t != null && e.until != null); + const out = [], skipped = []; + for (const n of notes) { + const last = out[out.length - 1]; + if (last && n.t - last.lastT < k.mergeMs && fit(union(last.box, n.box)) >= k.zoomMin) { last.box = union(last.box, n.box); last.lastT = n.t; if (n.until) last.until = Math.max(last.until || 0, n.until); continue; } + if (fit(n.box) < k.zoomMin) { skipped.push({t: n.t, why: 'region too large to earn a zoom', fit: +fit(n.box).toFixed(2)}); continue; } + out.push({firstT: n.t, lastT: n.t, box: n.box, until: n.until}); + } + let prevEnd = 0; const res = []; + for (const s of out) { + const z = clamp(fit(s.box), k.zoomMin, k.zoomMax), cx = s.box.x + s.box.width / 2, cy = s.box.y + s.box.height / 2; + const w = V.width / z, h = V.height / z, rect = {x: clamp(cx - w / 2, 0, V.width - w), y: clamp(cy - h * vis / 2, 0, V.height - kb - h * vis), w, h}; + const prevB = bounds.filter(x => x.end <= s.firstT).pop(), nextB = bounds.find(x => x.t > s.lastT); + let a = Math.max(prevEnd, prevB ? prevB.end : 0, s.firstT - k.leadMs), b = a + k.inMs; + let hold = Math.max(s.lastT + k.holdMs, b + k.holdMs, s.until || 0); + for (const h of holds) if (h.t >= a && h.t <= hold) hold = Math.max(hold, h.until); + const limit = Math.min(len, nextB ? nextB.t : len); + let c = hold, d = c + k.outMs; + if (d > limit) { d = limit; c = d - k.outMs; } + if (c - b < k.minHoldMs) { skipped.push({t: s.firstT, why: nextB && nextB.t < len ? 'next scroll or move comes too soon to hold' : 'no room to zoom out before the end', holdMs: Math.round(c - b)}); continue; } + res.push({z: +z.toFixed(3), rect, a, b, c, d}); prevEnd = d; + } + return {shots: res, skipped, len}; +} +// Camera rect (CSS px) at output time o. +function camera(shotList, V, o) { + const full = {x: 0, y: 0, w: V.width, h: V.height}; + for (const s of shotList) { + if (o < s.a || o > s.d) continue; + const u = o < s.b ? ease((o - s.a) / (s.b - s.a)) : o <= s.c ? 1 : 1 - ease((o - s.c) / (s.d - s.c)); + return {x: full.x + (s.rect.x - full.x) * u, y: full.y + (s.rect.y - full.y) * u, w: full.w + (s.rect.w - full.w) * u, h: full.h + (s.rect.h - full.h) * u}; + } + return full; +} + +// ---------------------------------------------------------------- output +const ENC = (crf = 18) => ['-c:v', 'libx264', '-crf', String(crf), '-preset', 'slow', '-pix_fmt', 'yuv420p', + '-colorspace', 'bt709', '-color_primaries', 'bt709', '-color_trc', 'bt709', '-color_range', 'tv', '-movflags', '+faststart']; + +// No camera: ffmpeg concat with per-frame durations, resampled to a constant fps. +function assemble(dir, out, o = {}) { + const {meta, frames} = load(dir), {toOut, len} = mapper(meta), fps = o.fps || 60, W = o.width || meta.viewport.width; + const kept = frames.map(f => ({...f, o: toOut(f.t)})).filter(f => f.o != null); + // the picture on screen at t0 is the last frame before it + const before = frames.filter(f => f.t <= meta.t0).pop(); if (before && !kept.some(f => f.o <= 0)) kept.unshift({...before, o: 0}); + const lines = []; + kept.forEach((f, i) => { const nx = i + 1 < kept.length ? kept[i + 1].o : len, d = Math.max(0, nx - Math.max(0, f.o)) / 1000; if (d > 0) lines.push(`file 'frames/${f.file}'`, `duration ${d.toFixed(4)}`); }); + lines.push(lines[lines.length - 2]); // concat demuxer drops the last duration otherwise + const list = path.join(dir, 'concat.txt'); fs.writeFileSync(list, lines.join('\n') + '\n'); + execFileSync(FF, ['-y', '-loglevel', 'error', '-f', 'concat', '-safe', '0', '-i', list, '-vf', `fps=${fps},scale=${W}:-2:flags=lanczos`, '-t', (len / 1000).toFixed(3), ...ENC(o.crf), out]); + return out; +} + +const COMPOSE_HTML = ``; + +// Camera render: each output frame drawn on a canvas from a fractional source rect (sub-pixel), piped to x264. +// o.keepBottom: CSS px of a fixed bottom band (caption strip) composited unzoomed on top of the zoomed picture. +async function render(dir, out, o = {}) { + const {chromium} = require('playwright'); + dir = path.resolve(dir); const {meta, frames, events} = load(dir), {toOut, len} = mapper(meta), V = meta.viewport, fps = o.fps || 60; + const plan = shots(meta, events, {...o.knobs, keepBottom: o.keepBottom || 0}); fs.writeFileSync(path.join(dir, 'shots.json'), JSON.stringify(plan, null, 1)); + const W = o.width || V.width, H = Math.round(W * V.height / V.width / 2) * 2, keep = o.keepBottom || 0; + const kept = frames.map(f => ({...f, o: toOut(f.t)})).filter(f => f.o != null); + const before = frames.filter(f => f.t <= meta.t0).pop(); if (before && !kept.some(f => f.o <= 0)) kept.unshift({...before, o: 0}); + // Served from a fake http origin: file:// images taint the canvas and toDataURL refuses to export it. + const browser = await chromium.launch(), page = await browser.newPage(); + await page.route('http://dp.local/**', r => { const u = new URL(r.request().url()).pathname; + if (u === '/') return r.fulfill({contentType: 'text/html', body: COMPOSE_HTML}); + r.fulfill({path: path.join(dir, 'frames', path.basename(u))}); }); + await page.goto('http://dp.local/'); await page.evaluate(([w, h, vw]) => setup(w, h, vw), [W, H, V.width]); + const ff = spawn(FF, ['-y', '-loglevel', 'error', '-f', 'image2pipe', '-framerate', String(fps), '-c:v', 'mjpeg', '-i', '-', ...ENC(o.crf), out], {stdio: ['pipe', 'inherit', 'inherit']}); + const done = new Promise((res, rej) => ff.on('close', c => c ? rej(new Error('ffmpeg exit ' + c)) : res())); + const N = Math.round(len / 1000 * fps); let j = 0, last = null, lastKey = ''; + const t0 = Date.now(); + try { + for (let i = 0; i < N; i++) { + const ot = i * 1000 / fps; while (j + 1 < kept.length && kept[j + 1].o <= ot) j++; + const cam = camera(plan.shots, V, ot), r = [cam.x, cam.y, cam.w, cam.h].map(v => +v.toFixed(3)); + const key = kept[j].file + r.join(','); + if (key !== lastKey) { last = Buffer.from(await page.evaluate(a => draw(...a), [kept[j].file, ...r, cam.w < V.width - .01 ? keep : 0]), 'base64'); lastKey = key; } + if (!ff.stdin.write(last)) await new Promise(r => ff.stdin.once('drain', r)); + } + } finally { ff.stdin.end(); await browser.close(); } + await done; + return {out, frames: N, shots: plan.shots.length, skipped: plan.skipped, renderSec: (Date.now() - t0) / 1000, captureFps: +meta.captureFps.toFixed(1)}; +} + +// Unique-picture rate of the source: how much real motion the 60 fps container carries. +function sourceStats(dir) { + const {meta, frames} = load(dir), {toOut} = mapper(meta), ts = frames.map(f => toOut(f.t)).filter(v => v != null); + const gaps = ts.slice(1).map((v, i) => v - ts[i]).filter(g => g > 0).sort((a, b) => a - b); + const q = p => gaps.length ? +gaps[Math.floor(p * (gaps.length - 1))].toFixed(1) : null; + return {frames: ts.length, captureFps: +meta.captureFps.toFixed(1), gapMsP50: q(.5), gapMsP90: q(.9), gapMsMax: q(1), clockSkewMs: meta.clockSkewMs}; +} + +module.exports = {KNOBS, launchArgs, prepare, ready, bottomPad, moveTo, scrollTo, click, focus, holdUntil, startCapture, assemble, render, shots, camera, sourceStats}; + +// CLI: node demo-polish.js render [--keep-bottom 120] | assemble | stats +if (require.main === module) { + const [cmd, dir, out, ...rest] = process.argv.slice(2), kb = rest.indexOf('--keep-bottom'); + (async () => { + if (cmd === 'render') console.log(JSON.stringify(await render(dir, out, {keepBottom: kb >= 0 ? +rest[kb + 1] : 0}))); + else if (cmd === 'assemble') console.log(assemble(dir, out)); + else if (cmd === 'stats') console.log(JSON.stringify(sourceStats(dir))); + else { console.error('usage: demo-polish.js render|assemble|stats [out.mp4] [--keep-bottom px]'); process.exit(2); } + })().catch(e => { console.error(e); process.exit(1); }); +} diff --git a/project-tests/e2e/demo-polish.selftest.js b/project-tests/e2e/demo-polish.selftest.js new file mode 100644 index 00000000..1c91028f --- /dev/null +++ b/project-tests/e2e/demo-polish.selftest.js @@ -0,0 +1,75 @@ +'use strict'; +// demo-polish.selftest.js — proves the camera planner keeps its rules. No browser, no app, no ffmpeg. +// +// node tests/e2e/demo-polish.selftest.js # exit 0 = every rule held +// +// The planner turns the recorder's event log into zoom shots. Each check below is one camera rule +// from skills/demo-recording-polish.md; a rule that breaks here shows up as a zoom that frames the +// wrong area, holds over a page that has scrolled away, or ends a scene zoomed in. +const P = require(__dirname + '/demo-polish.js'); + +let bad = 0; +const ok = (name, cond, info) => { if (!cond) bad++; console.log(`${cond ? 'ok ' : 'FAIL'} ${name}${cond ? '' : ' ' + JSON.stringify(info)}`); }; +const V = {width: 1920, height: 1080}, T0 = 1e12; +const meta = (lenMs, cuts = []) => ({viewport: V, t0: T0, end: T0 + lenMs, cuts}); +const card = {x: 100, y: 300, width: 500, height: 200}; +const focus = (t, box = card) => ({t: T0 + t, kind: 'focus', bbox: box, zoom: box}); +const scroll = (t, ms = 800) => ({t: T0 + t + ms, t0: T0 + t, t1: T0 + t + ms, kind: 'scroll', px: 400, bbox: card}); + +// 1. one note: lead, push-in, hold, zoom-out, capped zoom +{ const {shots: [s]} = P.shots(meta(20000), [focus(5000)]); + ok('one note gives one shot', !!s, s); + ok('push-in starts 400 ms before the note', s && s.a === 4600, s); + ok('push-in lasts 700 ms', s && s.b - s.a === 700, s); + ok('holds at least 2 s after the note', s && s.c >= 7000, s); + ok('zoom-out lasts 800 ms', s && s.d - s.c === 800, s); + ok('zoom never past 1.75x', s && s.z <= 1.75, s); } + +// 2. notes closer than 2.5 s merge into one shot framing both +{ const b2 = {x: 700, y: 300, width: 400, height: 200}; + const {shots} = P.shots(meta(20000), [focus(5000), focus(6200, b2)]); + ok('two notes 1.2 s apart merge', shots.length === 1, shots); + ok('merged shot frames both regions', shots[0] && shots[0].rect.x <= card.x && shots[0].rect.x + shots[0].rect.w >= b2.x + b2.width, shots[0]); } + +// 3. a region that only earns < 1.25x gets no zoom +{ const wide = {x: 64, y: 400, width: 1792, height: 300}; + const r = P.shots(meta(20000), [focus(5000, wide)]); + ok('full-width region is skipped, not zoomed', r.shots.length === 0 && r.skipped.length === 1, r); } + +// 4. the camera is back at 1x before the next scroll starts +{ const {shots: [s]} = P.shots(meta(20000), [focus(5000), scroll(9000)]); + ok('zoom-out ends by the next scroll', s && s.d <= 9000, s); + const r = P.shots(meta(20000), [focus(5000), scroll(5900)]); + ok('shot with no room to hold before a scroll is dropped', r.shots.length === 0 && /too soon/.test((r.skipped[0] || {}).why), r); } + +// 5. every take ends zoomed out +{ const {shots: [s]} = P.shots(meta(7000), [focus(4000)]); + ok('zoom-out finishes by the end of the take', s && s.d <= 7000, s); + const r = P.shots(meta(5000), [focus(4600)]); + ok('a note too close to the end is dropped', r.shots.length === 0, r); } + +// 6. narration holds the shot until the line ends +{ const {shots: [s]} = P.shots(meta(30000), [focus(5000), {t: T0 + 5100, kind: 'hold', until: T0 + 12000}]); + ok('hold extends to the end of the narration line', s && s.c >= 12000, s); } + +// 7. cuts: time after a cut shifts left, a note inside a cut is ignored +{ const m = meta(30000, [[T0 + 2000, T0 + 12000]]); + const r = P.shots(m, [focus(1000 + 0), focus(5000), focus(15000)]); + ok('note inside a cut ignored, later note shifted by the cut', r.shots.some(s => s.a === 15000 - 10000 - 400) && r.len === 20000, r); } + +// 8. a kept bottom band: the framed region sits in the visible area above it +{ const low = {x: 300, y: 800, width: 400, height: 120}; + const {shots: [s]} = P.shots(meta(20000), [focus(5000, low)], {keepBottom: 120}); + const visBottom = s && s.rect.y + s.rect.h * (V.height - 120) / V.height; + ok('zoomed region stays above the kept band', s && visBottom <= V.height - 120 + 0.01 && visBottom >= low.y + low.height, {s, visBottom}); } + +// 9. camera(): full frame outside shots, the shot rect while holding, eased in between +{ const {shots} = P.shots(meta(20000), [focus(5000)]), s = shots[0]; + const at = o => P.camera(shots, V, o); + ok('1x before the shot', at(s.a - 1).w === V.width); + ok('shot rect while holding', Math.abs(at((s.b + s.c) / 2).w - s.rect.w) < 1e-6); + const mid = at((s.a + s.b) / 2).w; ok('halfway into the push-in is between 1x and the shot', mid < V.width && mid > s.rect.w, mid); + ok('1x after the shot', at(s.d + 1).w === V.width); } + +console.log(bad ? `\n${bad} check(s) failed` : '\nall camera rules held'); +process.exit(bad ? 1 : 0); diff --git a/skills/demo-recording-polish.md b/skills/demo-recording-polish.md new file mode 100644 index 00000000..290acc0c --- /dev/null +++ b/skills/demo-recording-polish.md @@ -0,0 +1,190 @@ +# Demo recording polish — 60 fps capture, a visible cursor, eased scrolling, zoom afterwards + +**Applies to:** any project that records a demo film with Playwright (see `record-demo-video.md`) +**Purpose:** making a recorded walkthrough look produced rather than captured: smooth motion, a +pointer the viewer can follow, scrolling that glides, and a camera that pushes in where a page is +dense. The style is the one Screen Studio made familiar. +**Helpers:** `project-tests/e2e/demo-polish.js` (installed as `tests/e2e/demo-polish.js`), with +its selftest `demo-polish.selftest.js`. No dependencies beyond Playwright and ffmpeg. +**Source:** field run on a three-app Mendix demo, 2026-10-15. One read-only scene was recorded +both ways against the same data. The measurements are in "What it measured" below. + +## When to use this skill + +- A film is being recorded and someone will watch it: a pitch, a customer demo, a LinkedIn cut. +- Feedback on a film was "it looks janky", "I can't see where it clicks", "the scroll jumps", or + "I can't read that table". +- Not for test evidence. A journey video that only has to prove a path ran does not need any of + this. + +`record-demo-video.md` still owns the content rules: opening on the app, narration synced to the +pixels, human speed, deterministic data, checking claims against frames. This skill owns how the +picture moves. + +## Why the default recording looks rough + +Playwright's `recordVideo` is built for test evidence, not film. In playwright-core 1.59 its +`videoRecorder.js` encodes at a fixed **25 fps** with `vp8 -deadline realtime -speed 8 -b:v 1M`, +so the result is 25 fps at 1 Mbit/s. Fine text smears, and motion steps. On top of that, the +usual scripts: + +- scroll with `scrollIntoView({behavior:'smooth'})`, which has the browser's duration and curve + and cannot be tuned, or with raw `mouse.wheel` jumps, which teleport; +- show no pointer, so a click is a page that changes for no visible reason; +- have no way to zoom. A 1920 px dashboard shown at 1920 px is unreadable on a laptop screen, + and even more so on a phone. + +## The method + +### 1. Capture: CDP screencast at device pixels, assembled at 60 fps + +```js +const polish = require('./demo-polish'); +const browser = await chromium.launch({args: polish.launchArgs(2)}); +const ctx = await browser.newContext({viewport: {width: 1920, height: 1080}, deviceScaleFactor: 2}); +await polish.prepare(ctx); // cursor, ripple and scroll helpers; survive navigation +const page = await ctx.newPage(); +// ... log in and set up off camera ... +const rec = await polish.startCapture(page, {dir: 'take/'}); +// ... the scene ... +await rec.stop(); // frames/*.jpg + frames.json + events.json +``` + +- `Page.startScreencast` sends a JPEG each time the compositor paints, with a wall-clock + timestamp. `rec.stop()` writes the timestamps beside the frames, and the assembler gives each + frame its real duration, then resamples to a constant 60 fps (`libx264 -crf 18 -preset slow`, + yuv420p, tagged bt709). Narration offsets stay wall-clock, the same as before: a line said at + `t` lands at `t`. +- **The screencast ignores `deviceScaleFactor`.** Measured: with `deviceScaleFactor: 2` alone, + the frames are 1920x1080, in both the headless shell and new headless. Only the launch flag + `--force-device-scale-factor=2` (`polish.launchArgs(2)`) makes it send 3840x2160 frames. + Without the flag, a 1.75x zoom upscales a 1097 px crop to 1920 px and the text goes soft. + `frames.json` records `meta.frameSize`. Check it. +- The capture starts when you call `startCapture`, so the take starts on the app. The clapper + board in `record-demo-video.md` rule 1 is only needed for `recordVideo`. +- Idle stretches produce no frames. That is correct: the last frame is held until the next paint. + `polish.sourceStats(dir)` reports the gaps between frames during motion. That number, not the + container's 60/1, is the real smoothness. + +### 2. In the page: eased scroll, a drawn cursor, a click ripple + +| Helper | What it does | Knobs (in `KNOBS`) | +|---|---|---| +| `scrollTo(page, loc, {block})` | requestAnimationFrame scroll of the element's **real scroll container** (on Mendix that is `.mx-scrollcontainer-*`, never `window`), easeInOutCubic | duration `clamp(400 + 0.5 × px, 600, 1400)` ms, then wait 400–600 ms | +| `moveTo(page, x, y)` | drawn arrow cursor glides on a slightly bowed cubic Bezier; the real pointer lands at the end, so hover states are real | 500–900 ms per move by distance; arrow 30 px tall at 1080p | +| `click(page, loc, {zoom})` | scroll if needed → glide → pause → ripple → real `mouse.click` | pause 150–250 ms; ripple: a 55 px dark ring (4 px) with a white halo, scale 0.15→1 over 400 ms, opacity held near 0.6 to mid-way, then 0. The zoom region is measured **after** the click, once the target is visible | +| `focus(page, loc)` | a camera note without a click: "this region is worth a zoom" | — | +| `bottomPad(page, px)` | tells the scroller a fixed caption band covers the bottom px, so "center" means the visible area | — | + +The cursor is a DOM overlay at the top z-index with `pointer-events:none`. It is captured at 2x, +like everything else, so it stays sharp under a 1.75x zoom. + +### 3. Zoom afterwards, never in the page + +The recorder does not zoom. It logs `{t, kind, bbox}` for every click, focus and scroll to +`events.json`. `render()` plans the camera from that log and draws every output frame on a +canvas from a fractional source rectangle: + +```js +await polish.render('take/', 'take.mp4', {fps: 60, keepBottom: 120}); // writes take/shots.json too +polish.assemble('take/', 'flat.mp4'); // same frames, no camera +``` + +Why afterwards: + +- **A CSS transform on the page breaks the page.** `position:fixed` caption bands become + relative to the transformed ancestor, popups and dropdowns land in the wrong place, and + hit-testing for the next click moves. +- **Re-framing becomes a re-render, not a re-shoot.** The camera is a pure function of the log. + Change a knob, run `render` again; the app is not touched. That matters for any scene that + changes data. +- **Plain ffmpeg `zoompan` judders.** It rounds the crop to whole source pixels each frame, and a + slow push-in visibly steps. Drawing the frame with `drawImage` at fractional coordinates + removes the stepping. The compositor runs in headless Chromium, so it needs nothing beyond + Playwright. It served the frames over a `page.route` origin because file:// images taint the + canvas. +- **A kept bottom band stays still.** `keepBottom: 120` composites the bottom 120 px (the caption + strip) unzoomed over the zoomed picture, and frames the region inside the area above it. + +### Camera rules (enforced by the planner, checked by the selftest) + +| Rule | Value | +|---|---| +| Zoom | fit the region with 48 px margin, **capped at 1.75x**; a region that only earns < 1.25x gets **no** zoom (a full-width grid is not "dense", it is wide) | +| Push-in | starts 400 ms before the click, lasts 700 ms, easeInOutCubic | +| Hold | at least 2 s after the last note, or until the narration line ends (`say` → `holdUntil`) | +| Zoom-out | 800 ms | +| One zoom per shot | notes less than 2.5 s apart merge into one shot framing all of them | +| No cursor-follow | the camera holds still while the cursor moves inside the shot | +| Boundaries | the camera must be back at 1x **before** the next scroll or unzoomed cursor move starts; a shot that cannot hold 800 ms before then is dropped, not squeezed | +| Ends zoomed out | every take ends at 1x; a note too close to the end is dropped | + +Dropped notes are listed in `shots.json` → `skipped` with the reason. Read them: a note that +never zoomed is a decision the planner made for you. + +`zoom: true` on a click frames a region of at least 960x540 around the click (~1.7x), not the +bare button: a 1.75x push-in on a 140 px button is a close-up of a button. Pass a Locator to +frame a specific card or grid instead. + +## Where to zoom + +- Zoom where the viewer must **read** something small: a KPI row, a filter on a grid, a status + chip, a table's first rows. Don't zoom on navigation or on a page's first appearance. +- Aim for one or two zooms a minute. A camera that zooms on every click feels nervous. +- Never zoom over a page that is about to change. Click → navigate is a 1x moment. + +## Verify — the numbers and the stills + +```bash +ffprobe -v error -select_streams v -show_entries stream=r_frame_rate,avg_frame_rate,width,height -of compact take.mp4 +node tests/e2e/demo-polish.js stats take/ # capture fps, frame-gap p50/p90 during motion +node tests/e2e/demo-polish.selftest.js # camera rules +for t in 3 9 15 21; do ffmpeg -y -loglevel error -ss $t -i take.mp4 -frames:v 1 still-$t.png; done +``` + +Look at four stills: mid-scroll, cursor plus ripple, the zoomed hold, and the last frame +(must be at 1x). Things to look for: a zoom framing the wrong area (the region moved after the +note), a cursor resting on a tooltip, the caption band doubled (`keepBottom` missing), soft text +(`frameSize` is 1920 wide, so the launch flag is missing). + +## What it measured + +One read-only scene in a Mendix app, 55 s, recorded both ways against the same data on 2026-10-16 +(1920x1080 viewport, playwright-core 1.59.1, headless Chromium on macOS): + +| | old path (`recordVideo` → mux) | new path (`demo-polish`) | +|---|---|---| +| ffprobe | h264 1920x1080, `r_frame_rate=30/1`, nb_frames 1653, 2.48 MB (source webm vp8 `25/1`) | h264 1920x1080, `r_frame_rate=60/1`, nb_frames 3311, 4.22 MB | +| Unique frames, 1.0 s scroll | 15/32 | 36/62 | +| Unique frames, 0.8 s scroll | 18/26 | 33/50 | +| Unique frames, 1.3 s cursor glide + click | 5/41 (no cursor; only the page change) | 53/80 | +| Source frames | 1920x1080 | 3840x2160; frame gap during motion p50 19.4 ms, p90 28 ms | +| Camera | none | 2 shots (1.75x on a facts card, 1.36x on a grid pane), 0 skipped | +| Render | — | 18.5 s for 55 s of 60 fps output | + +Read it honestly: + +- **The motion is about 35–50 Hz in a 60 fps container.** A Mendix page painting at 4K is the + limit: a p50 gap of 19.4 ms is about 50 Hz. On a light local page the same capture managed + 16–18 ms, and JPEG quality 88, 70 and 55 changed nothing. The assembler repeats frames to reach + 60. The real rate is roughly double the old path's, not 2.4x. +- **The camera clamps to the frame.** A card at the right edge sits off-centre in its zoom. + Framing a whole pane around a short grid leaves empty space below it, so pass the grid, not the pane. +- **The ripple is deliberately neutral.** An orange ripple vanished against an orange highlight + outline the recorder already drew. Keep the ripple's colour away from any highlight colour, or + turn the highlight off in polish mode. + +## Failure modes + +| Symptom | Cause | Fix | +|---|---|---| +| Zoomed text is soft | screencast frames at 1920x1080 although `deviceScaleFactor: 2` | launch with `polish.launchArgs(2)`; check `meta.frameSize` | +| `toDataURL` "Tainted canvases may not be exported" | compositor page loads frames over file:// | serve them over `page.route` (the helper does) | +| The zoom shows the page after it scrolled away | camera hold crossed a scroll | the planner treats scroll starts as boundaries; log scrolls through `scrollTo`, not raw `mouse.wheel` | +| Caption band appears twice or zoomed | bottom band zoomed with the page | `render(..., {keepBottom: })` | +| A planned zoom never happened | region too wide (< 1.25x) or no room to hold before the next move | read `shots.json` → `skipped` | +| Stutter despite "60 fps" in ffprobe | the container is 60/1, the source painted less often | `stats` gap p90; lower JPEG quality or the DSF if p90 is far above 16.7 ms | +| `scrollTo` does nothing | the element's scroll container is not `window` and has no overflow | the helper walks up to the first scrollable ancestor; check the page really scrolls | +| Ripple invisible in the take | ripple colour equals a highlight outline on the same element, or opacity faded out within the first frames | `KNOBS.rippleColor` contrasting with every highlight; keep opacity up to mid-animation | +| `boundingBox` timeout on a zoom Locator | the region (a tab pane, a dialog) is hidden until the click | the helper measures the region after the click and waits up to 5 s for it to be visible | +| The first zoom is cut short | a zoom note at t≈0, or a scroll right after it | give the scene ~1.5 s at 1x first and about 3.5 s after the note before the next scroll | diff --git a/skills/record-demo-video.md b/skills/record-demo-video.md index 0d97f174..0422f3ee 100644 --- a/skills/record-demo-video.md +++ b/skills/record-demo-video.md @@ -223,7 +223,9 @@ Dwell must be long enough to *read*. A caption of N words needs roughly Type with a human-like per-character delay and click with an animated cursor-move-then-press helper — instant `fill()` and `click()` read as a machine. Inject a visible cursor: a click with no visible pointer is invisible -on playback. +on playback. `demo-recording-polish.md` has the tested helpers (a drawn cursor +on a Bezier path, a click ripple, eased scrolling, 60 fps capture and a zoom +rendered after the take) and the numbers behind each knob. ### 6. Show scrolling @@ -242,6 +244,10 @@ async function showcase(page, distance = 320) { Target the app's real scroll port, not `window` — on a Mendix page with Atlas UI that's `.mx-scrollcontainer-center`; scrolling `window` does nothing. +`scrollIntoView({behavior:'smooth'})` and raw `mouse.wheel` steps are the +rough versions: the first has a duration you cannot set, the second teleports. +`demo-recording-polish.md` → `scrollTo()` eases the real container over +`clamp(400 + 0.5 × px, 600, 1400)` ms. ### 7. Show state changing, don't just claim it @@ -519,6 +525,7 @@ be sorted. | Symptom | Cause | Fix | |---|---|---| +| Film looks janky: smeared text, stepping motion, no pointer, an unreadable dense page | `recordVideo` encodes 25 fps vp8 at 1 Mbit/s; no cursor; no zoom | `demo-recording-polish.md` — 2x screencast at 60 fps, drawn cursor, eased scroll, zoom afterwards | | First ~20s blank | `recordVideo` starts at context creation; cold runtime | Rule 1 — warm up, clapper board, trim | | Frame 0 still blank after trimming the measured lead-in | Playwright's video timeline does not track wall-clock time | Rule 1 — trim from the clapper, never from the clock | | Clapper never found in the recording even though it painted in the DOM | Chromium's screencast submits frames on compositor activity, not on a timer; an idle page with one style change can submit zero new frames | Rule 1 — force rasters with `page.screenshot()` across the hold | From 87741b5e6eda97de0a72483127b5dd8dd442aa82 Mon Sep 17 00:00:00 2001 From: MendixMau Date: Fri, 16 Oct 2026 09:42:17 +0800 Subject: [PATCH 2/2] feat(demo-recording-polish): centred zooms on the element, visible ripple, cursor parking, narration holds Round 2 after looking at the first take's stills: zoom frames the element with 8% padding, centred or not at all (edge targets skipped, cut targets capped at 1.25x); ripple 0->60 px over 450 ms, dark ring + white halo; park() rests the cursor in a side gutter or beside the next target; a narration line already playing when a shot starts holds it. Selftest 25 rules. Skill and changelog carry the re-measured 82 s take. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 2 +- project-tests/e2e/demo-polish.js | 47 +++++++++++----- project-tests/e2e/demo-polish.selftest.js | 22 +++++++- skills/demo-recording-polish.md | 66 +++++++++++++++-------- 4 files changed, 101 insertions(+), 36 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a19aa732..e194da6a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -36,7 +36,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m - fix(verify-module): the graph, coverage and journeys rung budgets are variables (`VM_TMO_GRAPH`, `VM_TMO_COVERAGE`, `VM_TMO_JOURNEYS`; defaults unchanged at 300/300/900, like the existing `VM_TMO_CONFORMANCE`), so a large build plan or a loaded machine can raise the 300 s that reported a clean coverage run as a timeout (TK-008) — five-app lab project (Mendix 11.15, mxcli v0.26.0) - fix(app-restart): mask the value of any `Name=value` whose name holds password/secret/token/key (e.g. `--constant Module.XPassword=v`), not only values after secret-named flags; the printed `start:` line leaked them (TK-003) — five-app lab project (Mendix 11.15, mxcli v0.26.0) - learn(bug-logs): **six BUG-DRAFT entries from a five-app lab project (mxcli v0.26.0, Mendix 11.15)**: `check` expression gaps (formatDecimal, Decimal toString, raw backslash in replaceAll), declare/retrieve partial checks, microflow splice and alter variable scope, create without commit, Data grid 2 write defects, describe round-trip and DateTime localization. Each gives the workaround; none is filed upstream yet, upstream issue numbers named where they exist (#580, #1489, #1111, #891). Read on demand through `bin/bug-lookup.sh`. — MendixMau, five-app lab report TK-027 to TK-036 -- new(demo-recording-polish): **`project-tests/e2e/demo-polish.js` and `skills/demo-recording-polish.md`: demo films at 60 fps with a visible cursor, a click ripple, eased scrolling and a Screen Studio-style zoom rendered after the take.** Playwright's `recordVideo` encodes 25 fps vp8 at 1 Mbit/s realtime (playwright-core 1.59 `videoRecorder.js`). The helpers capture with CDP `Page.startScreencast` at device pixels (only `--force-device-scale-factor=2` makes the screencast send 3840x2160; `deviceScaleFactor` alone gives 1920x1080, measured) and assemble the timestamped JPEGs at a constant 60 fps (x264 crf 18, bt709). They also draw a Bezier cursor with a 55 px ripple and scroll the real scroll container with easeInOutCubic over `clamp(400 + 0.5 × px, 600, 1400)` ms. Clicks, focus notes and scrolls are logged to `events.json`; `render()` plans one zoom per shot from that log (capped at 1.75x, 400 ms lead, 700 ms push-in, at least a 2 s hold or until the narration line ends, 800 ms out, notes under 2.5 s apart merged, back at 1x before the next scroll or move, every take ends at 1x) and draws each frame from a fractional source rectangle in headless Chromium, so the zoom does not judder the way ffmpeg `zoompan` does. A fixed caption band can be kept unzoomed. `demo-polish.selftest.js` checks 20 camera rules. Both files are in `MXTK_PROJECT_TESTS`; `record-demo-video.md` rules 5 and 6 link to the new skill. Measured on one read-only 55 s Mendix scene recorded both ways: old 30/1 (source vp8 25/1), new 60/1 from 3840x2160 frames, unique frames in a 1 s scroll 15/32 → 36/62 and in a 1.3 s cursor glide 5/41 → 53/80. The app paints at about 50 Hz at 4K (frame gap p50 19.4 ms), so the real motion rate is roughly double, not 60. — MendixMau, three-app Mendix demo films 2026-10-15 +- new(demo-recording-polish): **`project-tests/e2e/demo-polish.js` and `skills/demo-recording-polish.md`: demo films at 60 fps with a visible cursor, a click ripple, eased scrolling and a Screen Studio-style zoom rendered after the take.** Playwright's `recordVideo` encodes 25 fps vp8 at 1 Mbit/s realtime (playwright-core 1.59 `videoRecorder.js`). The helpers capture with CDP `Page.startScreencast` at device pixels (only `--force-device-scale-factor=2` makes the screencast send 3840x2160; `deviceScaleFactor` alone gives 1920x1080, measured) and assemble the timestamped JPEGs at a constant 60 fps (x264 crf 18, bt709). They also draw a Bezier cursor with a 60 px ripple (dark ring, white halo), park it in a side gutter or beside the next target between actions and scroll the real scroll container with easeInOutCubic over `clamp(400 + 0.5 × px, 600, 1400)` ms. Clicks, focus notes and scrolls are logged to `events.json`; `render()` plans one zoom per shot from that log (the element itself with 8% padding, centred or not at all, capped at 1.75x, 400 ms lead, 700 ms push-in, at least a 2 s hold or until the narration line ends, 800 ms out, notes under 2.5 s apart merged, back at 1x before the next scroll or move, every take ends at 1x) and draws each frame from a fractional source rectangle in headless Chromium, so the zoom does not judder the way ffmpeg `zoompan` does. A fixed caption band can be kept unzoomed. `demo-polish.selftest.js` checks 25 camera rules. Both files are in `MXTK_PROJECT_TESTS`; `record-demo-video.md` rules 5 and 6 link to the new skill. Measured on one read-only 82 s Mendix scene recorded both ways: old 30/1 (source vp8 25/1), new 60/1 from 3840x2160 frames; changed frames over the whole clip 223/2466 → 761/4940, in a 1.3 s cursor glide 1/39 → 33/78; two centred zooms (u/v .49–.54). The app paints at about 44 Hz at 4K (frame gap p50 22.6 ms), so the real motion rate is below 60. — MendixMau, three-app Mendix demo films 2026-10-15 - new(mxpatch, exec): **`project-bin/mxpatch.py`, the shared patch core, and a `mxcli fix hashes` check around every `exec.sh --patch` run: the two shipped workflow patchers no longer leave a stale ContentsHash.** mxpatch finds a unit by type + name (+ module) with a byte pre-filter, finds an element by Name, gets/sets a value by key, and writes the unit and its `.mpr` ContentsHash (base64 SHA-256 of the file) in one step; dry run by default, `already applied` on a re-run, and a refusal (exit 3) when the current value is not the expected old one. `wf-set-call-captions.py` and `wf-add-path-terminators.py` now write through it, refuse drift (a caption is replaced only while it is still the stamped microflow name or the `--expect-caption` value; `--expect-paths N` for terminators), and also take the `exec.sh --patch` form (` Module.Workflow`). The index is committed before the file swap, so a kill mid-write leaves a MISMATCH that `fix hashes` reports and a re-run repairs. `mxunit_bson.py` reads BSON 0x09 (UTC datetime). `exec.sh --patch` runs `mxcli fix hashes` (v0.25.0+, probed via `fix --help`) before and after the script and restores the snapshot on any issue the baseline did not have, before mxbuild, which cannot see it. Field evidence: captured mxcli v0.25.0/v0.26.0 runs on a public model (`tests/wave2/fixtures/mxpatch/CAPTURE.txt`): master's patchers leave a MISMATCH, the ported ones verify clean; the hash routine agrees with mxcli on 67 of 67 mismatched and 302 matching units of a second model. Widget-shape copying and pluggable-widget properties wait on a captured page unit. Field run ([#390](https://github.com/MendixMau/mxcli-project-toolkit/issues/390), mxcli v0.26.0, macOS): PASS, and the patch repaired a ContentsHash MISMATCH an old-style `--patch` had left on the unit it rewrote. Fixes from it: the backup of a rewritten unit is kept outside `mprcontents/` (in `$MXPATCH_BACKUP_DIR`, which `exec.sh --patch` points into the snapshot it just took, else a 0700 `/mxpatch-bak-/`; refused when it resolves inside `mprcontents/`, through a symlink or another case spelling; named with pid and counter so two writes in one second keep both), no longer as an untracked `.mxunit.bak` that `git add mprcontents` would commit; the BUILD-LOG row carries the hash bracket (`hashes: baseline 1 mismatch (pre-existing), verify clean (1 pre-existing repaired by this patch's rewrite)`); and the baseline message says that a unit the patch rewrites gets a fresh hash, and to repair the others with `fix hashes --repair`. A baseline MISMATCH that is still reported on a unit the patch rewrote counts as new (restore, exit 1), not as pre-existing. — jopterhorst ([#294](https://github.com/MendixMau/mxcli-project-toolkit/issues/294), [#295](https://github.com/MendixMau/mxcli-project-toolkit/issues/295)) - learn(pipelines/mendix-mpr/samples): **three minimal mxcli repros from the native-module bug hunt, the files the upstream draft cites** (split from [#293](https://github.com/MendixMau/mxcli-project-toolkit/pull/293), 6/6). `fsm-template-9.24.2/native-bughunt/` holds CAPTURE.txt and D1–D3, which are written from scratch in a `Repro` module: `describe entity` printing `index ( desc);`, the native Repeater `content` slot, and no flow writable to a 9.24.2 project. They are the repros for U3, U2 and U14 in `bug-logs/pending-github-issues/2026-10-12-mxcli-0.26-round-trip-report.md`. The other 472 sample files of #293 had no citing reader, because the skills and Rehost splits state their field results inline, so none of them ships. — MendixMau, Decompose/Rehost lab runs 2026-10-12 - learn(inbox, pending-github-issues): **ten field-note drops from the Decompose/Rehost lab runs, and one not-filed draft of 53 mxcli 0.26 defects for mendixlabs/mxcli** (split from [#293](https://github.com/MendixMau/mxcli-project-toolkit/pull/293), 5/6). `contrib/inbox/2026-10-1{2,3}-*.md`: increments 1–8 of the split-a-monolith runs on a public Marketplace template (FSM 9.24.2), the native-module bug hunt, and the GenAI Showcase regeneration (0.26.0 and the 2026-10-13 nightly); each now has the template's front matter and is at most 600 words. `bug-logs/pending-github-issues/2026-10-12-mxcli-0.26-round-trip-report.md`: the U1–U53 table the inbox drops refer to, marked DRAFT — NOT FILED, to be split into one issue per defect and retested before anything is posted. No BUG numbers assigned; triage promotes. — MendixMau, Decompose/Rehost lab runs 2026-10-12/13 diff --git a/project-tests/e2e/demo-polish.js b/project-tests/e2e/demo-polish.js index d322efbc..ac22217f 100644 --- a/project-tests/e2e/demo-polish.js +++ b/project-tests/e2e/demo-polish.js @@ -10,6 +10,7 @@ // await polish.scrollTo(page, locator); // eased, on the real scroll container // await polish.click(page, locator, {zoom: cardLocator}); // glide, pause, ripple, click; camera note // await polish.focus(page, denseCard); // camera note without a click +// await polish.park(page, {near: nextTarget}); // rest the cursor beside the next target, off the content // await rec.stop(); // writes frames.json + events.json // await polish.render('take/', 'take.mp4', {fps:60}); // composite: eased push-in/hold/zoom-out, sub-pixel // polish.assemble('take/', 'flat.mp4'); // same frames, no camera (fast, ffmpeg only) @@ -32,12 +33,15 @@ const KNOBS = { cursorPx: 30, // arrow height at 1080p moveMs: [500, 900], // per cursor move, scaled by distance preClickMs: [150, 250], // settle on the target before pressing - ripplePx: 55, rippleMs: 400, // 0 -> 55 px, ease-out, opacity .6 -> 0 + ripplePx: 60, rippleMs: 450, // 0 -> 60 px, ease-out; opacity held near .9 to half-way, then 0 rippleColor: 'rgba(17,24,39,.95)', rippleFill: 'rgba(17,24,39,.18)', // dark ring, white halo (in the overlay) scrollMs: px => clamp(400 + 0.5 * px, 600, 1400), afterScrollMs: [400, 600], zoomMax: 1.75, zoomMin: 1.25, // never past 1.75; a region that only earns < 1.25 gets no zoom at all - zoomMarginPx: 48, // breathing room around the framed region (CSS px) + zoomPad: .08, // padding around the framed element, as a share of its size per side + centreBand: .2, // the target's centre must land within 0.5 +- this of the zoomed frame, or no zoom + parkInsetPx: 32, // between actions the cursor rests this far in from the side edge, never mid-content + parkMs: 800, parkAfterMs: 450, leadMs: 400, // push-in starts this long before the click inMs: 700, holdMs: 2000, outMs: 800, mergeMs: 2500, // camera notes closer than this share one shot @@ -62,8 +66,8 @@ const OVERLAY = `(()=>{if(window.__dp)return;const K=${JSON.stringify({c: KNOBS. place(m*m*m*x0+3*m*m*t*p1.x+3*m*t*t*p2.x+t*t*t*x,m*m*m*y0+3*m*m*t*p1.y+3*m*t*t*p2.y+t*t*t*y);if(u>=1)break;}} function ripple(x,y){const r=document.createElement('div'); // dark ring + white halo: visible on light and dark UI, and never the same colour as a page's own highlight - r.style.cssText='position:fixed;left:'+x+'px;top:'+y+'px;width:'+K.r+'px;height:'+K.r+'px;margin:-'+K.r/2+'px 0 0 -'+K.r/2+'px;border-radius:50%;border:4px solid '+K.rc+';box-shadow:0 0 0 2px rgba(255,255,255,.9),inset 0 0 0 2px rgba(255,255,255,.9);background:'+K.rf+';z-index:2147483646;pointer-events:none'; - document.body.append(r);r.animate([{transform:'scale(0.15)',opacity:.6},{transform:'scale(.75)',opacity:.45,offset:.55},{transform:'scale(1)',opacity:0}],{duration:K.rm,easing:'cubic-bezier(.22,.61,.36,1)',fill:'forwards'}).onfinish=()=>r.remove();} + r.style.cssText='position:fixed;left:'+x+'px;top:'+y+'px;width:'+K.r+'px;height:'+K.r+'px;margin:-'+K.r/2+'px 0 0 -'+K.r/2+'px;border-radius:50%;border:4px solid '+K.rc+';box-shadow:0 0 0 3px rgba(255,255,255,.95),inset 0 0 0 2px rgba(255,255,255,.9);background:'+K.rf+';z-index:2147483646;pointer-events:none'; + document.body.append(r);r.animate([{transform:'scale(0.1)',opacity:.95},{transform:'scale(.7)',opacity:.85,offset:.5},{transform:'scale(1)',opacity:0}],{duration:K.rm,easing:'cubic-bezier(.22,.61,.36,1)',fill:'forwards'}).onfinish=()=>r.remove();} const scroller=el=>{for(let n=el.parentElement;n;n=n.parentElement){const s=getComputedStyle(n);if(/(auto|scroll)/.test(s.overflowY)&&n.scrollHeight>n.clientHeight+1)return n;}return document.scrollingElement;}; // Eased scroll of the element's real scroll container; returns how far it went so the caller can time the wait. async function scroll(el,block,msFor){if(getComputedStyle(el).display==='contents'&&el.firstElementChild)el=el.firstElementChild; @@ -83,7 +87,7 @@ const st = page => page.__dp || (page.__dp = {x: null, y: null, rec: null}); async function ready(page) { await page.evaluate(OVERLAY); const s = st(page); - if (s.x == null) { const v = page.viewportSize(); s.x = v.width * .62; s.y = v.height * .58; } + if (s.x == null) { const v = page.viewportSize(); s.x = KNOBS.parkInsetPx; s.y = v.height * .5; } // rest at the side, not on content await page.evaluate(([x, y]) => { window.__dp.ensure(); window.__dp.place(x, y); }, [s.x, s.y]); } // Tell the scroller about a fixed band at the bottom (a caption strip) so "center" means the visible area. @@ -101,6 +105,16 @@ async function moveTo(page, x, y, o = {}) { s.x = x; s.y = y; await page.mouse.move(x, y); // the real pointer lands too, so hover states are true } +// Park the cursor between actions: beside o.near (a Locator: just left of it, or right when it hugs the left edge), +// else in the nearest side gutter at the current height. Never left resting on the content the viewer is reading. +async function park(page, o = {}) { + await ready(page); const s = st(page), V = page.viewportSize(), k = KNOBS.parkInsetPx; + let x = s.x < V.width / 2 ? k : V.width - k, y = clamp(s.y, V.height * .12, V.height * .85); + if (o.near) { const b = await boxOf(o.near.first ? o.near.first() : o.near).catch(() => null); + if (b) { x = b.x > 3 * k ? b.x - k : b.x + b.width + k; y = clamp(b.y + b.height / 2, V.height * .12, V.height * .85); } } + await moveTo(page, x, y, {ms: o.ms || KNOBS.parkMs}); +} + async function scrollTo(page, loc, o = {}) { await ready(page); const t0 = Date.now(); const px = await loc.evaluate((el, [b]) => window.__dp.scroll(el, b, px => Math.max(600, Math.min(1400, 400 + .5 * px))), [o.block || 'center']); @@ -126,6 +140,7 @@ async function click(page, loc, o = {}) { else if (o.zoom) { const z = o.zoom.first ? o.zoom.first() : o.zoom; if (await z.waitFor({state: 'visible', timeout: o.zoomWait || 5000}).then(() => true, () => false)) { await nap(300); zb = await boxOf(z); } } const r = st(page).rec; if (r) r.events.push({t: tClick, kind: 'click', bbox: b, x, y, t0, zoom: zb || undefined}); + if (o.park !== false) { await nap(KNOBS.parkAfterMs); await park(page, {near: o.parkNear}); } } async function focus(page, loc, o = {}) { @@ -192,9 +207,12 @@ function mapper(meta) { function shots(meta, events, k = {}) { k = {...KNOBS, ...k}; const {toOut, len} = mapper(meta), V = meta.viewport, kb = k.keepBottom || 0, vis = (V.height - kb) / V.height; // with a kept bottom band only the area above it shows the zoomed picture, so frame inside that area - const fit = b => Math.min(V.width / (b.width + 2 * k.zoomMarginPx), (V.height - kb) / (b.height + 2 * k.zoomMarginPx)); - const union = (a, b) => { const x = Math.min(a.x, b.x), y = Math.min(a.y, b.y); return {x, y, width: Math.max(a.x + a.width, b.x + b.width) - x, height: Math.max(a.y + a.height, b.y + b.height) - y}; }; - const notes = events.filter(e => e.zoom).map(e => ({t: toOut(e.t), box: e.zoom, until: e.until && toOut(e.until)})).filter(e => e.t != null).sort((a, b) => a.t - b.t); + const P = 1 + 2 * k.zoomPad, fit = b => Math.min(V.width / (b.width * P), (V.height - kb) / (b.height * P)); + // A target partly outside the frame is clipped to what is visible and never zoomed past zoomMin. + const clip = b => { const x = Math.max(0, b.x), y = Math.max(0, b.y), r = Math.min(V.width, b.x + b.width), btm = Math.min(V.height - kb, b.y + b.height); + return {x, y, width: Math.max(0, r - x), height: Math.max(0, btm - y), cut: r - x < b.width - 4 || btm - y < b.height - 4}; }; + const union = (a, b) => { const x = Math.min(a.x, b.x), y = Math.min(a.y, b.y); return {x, y, width: Math.max(a.x + a.width, b.x + b.width) - x, height: Math.max(a.y + a.height, b.y + b.height) - y, cut: a.cut || b.cut}; }; + const notes = events.filter(e => e.zoom).map(e => ({t: toOut(e.t), box: clip(e.zoom), until: e.until && toOut(e.until)})).filter(e => e.t != null).sort((a, b) => a.t - b.t); // Boundaries: the camera must be back at 1x before the page scrolls or the cursor sets off for an unzoomed click, // or the hold frames a region the content has already left. const bounds = events.filter(e => (e.kind === 'scroll' && e.px > 40) || (e.kind === 'click' && !e.zoom)).map(e => ({t: toOut(e.t0 || e.t), end: toOut(e.t1 || e.t)})).filter(e => e.t != null).sort((a, b) => a.t - b.t); @@ -208,17 +226,22 @@ function shots(meta, events, k = {}) { } let prevEnd = 0; const res = []; for (const s of out) { - const z = clamp(fit(s.box), k.zoomMin, k.zoomMax), cx = s.box.x + s.box.width / 2, cy = s.box.y + s.box.height / 2; + const z = clamp(fit(s.box), k.zoomMin, s.box.cut ? k.zoomMin : k.zoomMax), cx = s.box.x + s.box.width / 2, cy = s.box.y + s.box.height / 2; const w = V.width / z, h = V.height / z, rect = {x: clamp(cx - w / 2, 0, V.width - w), y: clamp(cy - h * vis / 2, 0, V.height - kb - h * vis), w, h}; + // Centred or nothing: near a frame edge the camera rect clamps and the target slides off-centre. A lower zoom only + // widens the rect and makes that worse, so a target that cannot sit near the middle is not zoomed at all. + const u = (cx - rect.x) / w, v = (cy - rect.y) / (h * vis); + if (Math.abs(u - .5) > k.centreBand || Math.abs(v - .5) > k.centreBand) { skipped.push({t: s.firstT, why: 'target at the frame edge: cannot be centred', u: +u.toFixed(2), v: +v.toFixed(2)}); continue; } const prevB = bounds.filter(x => x.end <= s.firstT).pop(), nextB = bounds.find(x => x.t > s.lastT); let a = Math.max(prevEnd, prevB ? prevB.end : 0, s.firstT - k.leadMs), b = a + k.inMs; let hold = Math.max(s.lastT + k.holdMs, b + k.holdMs, s.until || 0); - for (const h of holds) if (h.t >= a && h.t <= hold) hold = Math.max(hold, h.until); + // a narration line that is playing when the shot starts (or starts during it) keeps the shot until it ends + for (const h of holds) if (h.t <= hold && h.until >= s.firstT) hold = Math.max(hold, h.until); const limit = Math.min(len, nextB ? nextB.t : len); let c = hold, d = c + k.outMs; if (d > limit) { d = limit; c = d - k.outMs; } if (c - b < k.minHoldMs) { skipped.push({t: s.firstT, why: nextB && nextB.t < len ? 'next scroll or move comes too soon to hold' : 'no room to zoom out before the end', holdMs: Math.round(c - b)}); continue; } - res.push({z: +z.toFixed(3), rect, a, b, c, d}); prevEnd = d; + res.push({z: +z.toFixed(3), rect, a, b, c, d, u: +u.toFixed(2), v: +v.toFixed(2)}); prevEnd = d; } return {shots: res, skipped, len}; } @@ -301,7 +324,7 @@ function sourceStats(dir) { return {frames: ts.length, captureFps: +meta.captureFps.toFixed(1), gapMsP50: q(.5), gapMsP90: q(.9), gapMsMax: q(1), clockSkewMs: meta.clockSkewMs}; } -module.exports = {KNOBS, launchArgs, prepare, ready, bottomPad, moveTo, scrollTo, click, focus, holdUntil, startCapture, assemble, render, shots, camera, sourceStats}; +module.exports = {KNOBS, launchArgs, prepare, ready, bottomPad, moveTo, park, scrollTo, click, focus, holdUntil, startCapture, assemble, render, shots, camera, sourceStats}; // CLI: node demo-polish.js render [--keep-bottom 120] | assemble | stats if (require.main === module) { diff --git a/project-tests/e2e/demo-polish.selftest.js b/project-tests/e2e/demo-polish.selftest.js index 1c91028f..d460e18a 100644 --- a/project-tests/e2e/demo-polish.selftest.js +++ b/project-tests/e2e/demo-polish.selftest.js @@ -52,17 +52,37 @@ const scroll = (t, ms = 800) => ({t: T0 + t + ms, t0: T0 + t, t1: T0 + t + ms, k { const {shots: [s]} = P.shots(meta(30000), [focus(5000), {t: T0 + 5100, kind: 'hold', until: T0 + 12000}]); ok('hold extends to the end of the narration line', s && s.c >= 12000, s); } +{ const {shots: [s]} = P.shots(meta(30000), [{t: T0 + 4000, kind: 'hold', until: T0 + 14000}, focus(5000)]); + ok('a line already playing when the shot starts holds it to the line end', s && s.c >= 14000, s); } + // 7. cuts: time after a cut shifts left, a note inside a cut is ignored { const m = meta(30000, [[T0 + 2000, T0 + 12000]]); const r = P.shots(m, [focus(1000 + 0), focus(5000), focus(15000)]); ok('note inside a cut ignored, later note shifted by the cut', r.shots.some(s => s.a === 15000 - 10000 - 400) && r.len === 20000, r); } // 8. a kept bottom band: the framed region sits in the visible area above it -{ const low = {x: 300, y: 800, width: 400, height: 120}; +{ const low = {x: 760, y: 700, width: 400, height: 120}; const {shots: [s]} = P.shots(meta(20000), [focus(5000, low)], {keepBottom: 120}); const visBottom = s && s.rect.y + s.rect.h * (V.height - 120) / V.height; ok('zoomed region stays above the kept band', s && visBottom <= V.height - 120 + 0.01 && visBottom >= low.y + low.height, {s, visBottom}); } +// 10. framing: the element itself, centred, with ~8% padding on every side +{ const box = {x: 700, y: 400, width: 500, height: 200}; + const {shots: [s]} = P.shots(meta(20000), [focus(5000, box)]); + const pad = Math.min(box.x - s.rect.x, s.rect.x + s.rect.w - box.x - box.width, box.y - s.rect.y, s.rect.y + s.rect.h - box.y - box.height); + ok('framed element sits inside the rect with at least 8% padding', s && pad >= 0.08 * box.height - 0.01, {s, pad}); + ok('framed element is centred (u, v within 0.5 +- 0.2)', s && Math.abs(s.u - .5) <= .2 && Math.abs(s.v - .5) <= .2, s); } + +// 11. a target hugging a frame edge cannot be centred, so it gets no zoom +{ const edge = {x: 1456, y: 327, width: 400, height: 170}; + const r = P.shots(meta(20000), [focus(5000, edge)]); + ok('edge target is skipped, not zoomed off-centre', r.shots.length === 0 && /frame edge/.test((r.skipped[0] || {}).why), r); } + +// 12. a target cut by the frame edge is clipped and never zoomed past zoomMin +{ const cut = {x: 600, y: -60, width: 700, height: 600}; + const {shots: [s]} = P.shots(meta(20000), [focus(5000, cut)]); + ok('half-cut target zooms at most zoomMin', s && s.z <= P.KNOBS.zoomMin, s); } + // 9. camera(): full frame outside shots, the shot rect while holding, eased in between { const {shots} = P.shots(meta(20000), [focus(5000)]), s = shots[0]; const at = o => P.camera(shots, V, o); diff --git a/skills/demo-recording-polish.md b/skills/demo-recording-polish.md index 290acc0c..96fb10c5 100644 --- a/skills/demo-recording-polish.md +++ b/skills/demo-recording-polish.md @@ -72,7 +72,8 @@ await rec.stop(); // frames/*.jpg + frames.json + event |---|---|---| | `scrollTo(page, loc, {block})` | requestAnimationFrame scroll of the element's **real scroll container** (on Mendix that is `.mx-scrollcontainer-*`, never `window`), easeInOutCubic | duration `clamp(400 + 0.5 × px, 600, 1400)` ms, then wait 400–600 ms | | `moveTo(page, x, y)` | drawn arrow cursor glides on a slightly bowed cubic Bezier; the real pointer lands at the end, so hover states are real | 500–900 ms per move by distance; arrow 30 px tall at 1080p | -| `click(page, loc, {zoom})` | scroll if needed → glide → pause → ripple → real `mouse.click` | pause 150–250 ms; ripple: a 55 px dark ring (4 px) with a white halo, scale 0.15→1 over 400 ms, opacity held near 0.6 to mid-way, then 0. The zoom region is measured **after** the click, once the target is visible | +| `click(page, loc, {zoom, park})` | scroll if needed → glide → pause → ripple → real `mouse.click` → park | pause 150–250 ms; ripple: a dark 4 px ring with a white halo, 0→60 px over 450 ms, opacity held near 0.9 to half-way, then 0. The zoom region is measured **after** the click, once the target is visible. Then the cursor parks (`park: false` to stay) | +| `park(page, {near})` | rests the cursor beside `near` (the next target), or in the nearest side gutter, 32 px in from the edge | 800 ms glide, 450 ms after a click | | `focus(page, loc)` | a camera note without a click: "this region is worth a zoom" | — | | `bottomPad(page, px)` | tells the scroller a fixed caption band covers the bottom px, so "center" means the visible area | — | @@ -110,13 +111,16 @@ Why afterwards: | Rule | Value | |---|---| -| Zoom | fit the region with 48 px margin, **capped at 1.75x**; a region that only earns < 1.25x gets **no** zoom (a full-width grid is not "dense", it is wide) | +| Zoom | fit the **element itself** with 8% padding per side, **capped at 1.75x**; a region that only earns < 1.25x gets **no** zoom (a full-width grid is not "dense", it is wide) | | Push-in | starts 400 ms before the click, lasts 700 ms, easeInOutCubic | | Hold | at least 2 s after the last note, or until the narration line ends (`say` → `holdUntil`) | | Zoom-out | 800 ms | | One zoom per shot | notes less than 2.5 s apart merge into one shot framing all of them | | No cursor-follow | the camera holds still while the cursor moves inside the shot | | Boundaries | the camera must be back at 1x **before** the next scroll or unzoomed cursor move starts; a shot that cannot hold 800 ms before then is dropped, not squeezed | +| Centred or nothing | the target's centre must land within 0.5 ± 0.2 of the zoomed frame (`shots.json` records `u`, `v`). Near a frame edge the camera clamps and the target slides off-centre, and a lower zoom only widens the rect and makes that worse, so an edge target gets no zoom | +| Cut targets | a target partly outside the frame is clipped to what shows and never zoomed past 1.25x | +| Narration holds | a line playing when the shot starts, or starting during it, keeps the shot until the line ends (bounded by the next scroll or move) | | Ends zoomed out | every take ends at 1x; a note too close to the end is dropped | Dropped notes are listed in `shots.json` → `skipped` with the reason. Read them: a note that @@ -124,7 +128,16 @@ never zoomed is a decision the planner made for you. `zoom: true` on a click frames a region of at least 960x540 around the click (~1.7x), not the bare button: a 1.75x push-in on a 140 px button is a close-up of a button. Pass a Locator to -frame a specific card or grid instead. +frame a specific card or grid instead: the grid, not the pane around it, or a short grid +leaves the bottom of the zoom empty. + +## Cursor and outline + +- **Park between actions.** A cursor left where it last clicked sits on the content the viewer is + reading. `click` parks it in the side gutter by default; before a beat whose target is known, + `park(page, {near: next})` puts it just beside that target, so the next glide is short. +- **No outline on clicked elements.** The cursor and ripple already say "here". Keep an outline + only for point-at beats with no click. ## Where to zoom @@ -142,37 +155,44 @@ node tests/e2e/demo-polish.selftest.js # camera rules for t in 3 9 15 21; do ffmpeg -y -loglevel error -ss $t -i take.mp4 -frames:v 1 still-$t.png; done ``` -Look at four stills: mid-scroll, cursor plus ripple, the zoomed hold, and the last frame +Look at five stills: mid-scroll, cursor plus ripple, each zoomed hold, the cursor at rest, and the last frame (must be at 1x). Things to look for: a zoom framing the wrong area (the region moved after the -note), a cursor resting on a tooltip, the caption band doubled (`keepBottom` missing), soft text +note), a zoomed target off-centre (`u`/`v` in `shots.json`), a cursor resting on content or a tooltip, the caption band doubled (`keepBottom` missing), soft text (`frameSize` is 1920 wide, so the launch flag is missing). ## What it measured -One read-only scene in a Mendix app, 55 s, recorded both ways against the same data on 2026-10-16 -(1920x1080 viewport, playwright-core 1.59.1, headless Chromium on macOS): +One read-only scene in a Mendix app, 82 s (hub → two tabs → a graph page), recorded both ways +against the same data on 2026-10-16 (1920x1080 viewport, playwright-core 1.59.1, headless +Chromium on macOS). The changed-frame counts use ffmpeg `mpdecimate=hi=64*4:lo=64*2:frac=0.1`. +The two takes differ by a few hundred ms in their timing, so the windows are approximate for the old take. | | old path (`recordVideo` → mux) | new path (`demo-polish`) | |---|---|---| -| ffprobe | h264 1920x1080, `r_frame_rate=30/1`, nb_frames 1653, 2.48 MB (source webm vp8 `25/1`) | h264 1920x1080, `r_frame_rate=60/1`, nb_frames 3311, 4.22 MB | -| Unique frames, 1.0 s scroll | 15/32 | 36/62 | -| Unique frames, 0.8 s scroll | 18/26 | 33/50 | -| Unique frames, 1.3 s cursor glide + click | 5/41 (no cursor; only the page change) | 53/80 | -| Source frames | 1920x1080 | 3840x2160; frame gap during motion p50 19.4 ms, p90 28 ms | -| Camera | none | 2 shots (1.75x on a facts card, 1.36x on a grid pane), 0 skipped | -| Render | — | 18.5 s for 55 s of 60 fps output | +| ffprobe | h264 1920x1080, `r_frame_rate=30/1`, nb_frames 2466, 4.63 MB (source webm vp8 `25/1`) | h264 1920x1080, `r_frame_rate=60/1`, nb_frames 4940, 5.75 MB | +| Changed frames, whole clip | 223/2466 | 761/4940 | +| Changed frames, 1.3 s cursor glide + click | 1/39 (no cursor; only the page change) | 33/78 | +| Changed frames, 0.8 s scroll | 6/24 | 32/48 | +| Changed frames, 1.0 s push-in | — (no camera) | 57/60 | +| Source frames | 1920x1080 | 3840x2160; frame gap during motion p50 22.6 ms, p90 31 ms | +| Camera | none | 2 shots: 1.256x on a 4-row grid (u .49, v .54), 1.379x on a network canvas (u .50, v .51); each held to its narration line's end (14 s, 10 s); 0 skipped | +| Render | — | 53 s for 82 s of 60 fps output | Read it honestly: -- **The motion is about 35–50 Hz in a 60 fps container.** A Mendix page painting at 4K is the - limit: a p50 gap of 19.4 ms is about 50 Hz. On a light local page the same capture managed - 16–18 ms, and JPEG quality 88, 70 and 55 changed nothing. The assembler repeats frames to reach - 60. The real rate is roughly double the old path's, not 2.4x. -- **The camera clamps to the frame.** A card at the right edge sits off-centre in its zoom. - Framing a whole pane around a short grid leaves empty space below it, so pass the grid, not the pane. +- **The motion is about 44 Hz in a 60 fps container.** A Mendix page painting at 4K is the limit: + the p50 gap is 22.6 ms. On a light local page the same capture managed 16–18 ms, and JPEG + quality 88, 70 and 55 changed nothing. The assembler repeats frames to reach 60. +- **Zooms show neighbours cut at the frame edge.** The target is whole and centred, but slivers + of the cards beside it are cut by the frame. That is what a zoom is; it is not a framing error. +- **Edge cards get no zoom.** In this app the facts card and the footprint card sit against a + frame edge. A 1.75x zoom put the facts card off-centre in the first round, so both became + point-at beats with an outline. - **The ripple is deliberately neutral.** An orange ripple vanished against an orange highlight - outline the recorder already drew. Keep the ripple's colour away from any highlight colour, or - turn the highlight off in polish mode. + outline the recorder already drew. Keep the ripple's colour away from any highlight colour. +- **Re-framing really is a re-render.** A planner bug (a narration hold logged 1 s before the + push-in was ignored, so a zoom held 2 s instead of 14) was fixed and re-rendered from the same + take without touching the app. ## Failure modes @@ -188,3 +208,5 @@ Read it honestly: | Ripple invisible in the take | ripple colour equals a highlight outline on the same element, or opacity faded out within the first frames | `KNOBS.rippleColor` contrasting with every highlight; keep opacity up to mid-animation | | `boundingBox` timeout on a zoom Locator | the region (a tab pane, a dialog) is hidden until the click | the helper measures the region after the click and waits up to 5 s for it to be visible | | The first zoom is cut short | a zoom note at t≈0, or a scroll right after it | give the scene ~1.5 s at 1x first and about 3.5 s after the note before the next scroll | +| A zoom slid off-centre against a frame edge | the camera clamps at the edge; lowering the zoom widens the rect and makes it worse | the planner skips targets outside the centre band (`centreBand`); make the beat a point-at, or pick an element away from the edge | +| A zoom held 2 s although the line runs on | the narration hold was logged before the push-in started | fixed: any line playing at the shot's start holds it; re-render the take |