Skip to content

ci: let rstack-ecosystem-ci run the E2E suites - #66

Closed
fi3ework wants to merge 2 commits into
mainfrom
ci/linux-e2e
Closed

fi3ework wants to merge 2 commits into
mainfrom
ci/linux-e2e

Conversation

@fi3ework

@fi3ework fi3ework commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

Summary

Two pieces that rstack-ecosystem-ci needs to run this repo's E2E suites against an unreleased
@rstest/core on ubuntu-latest (#40). This repo's own CI stays Windows + macOS.

 e2e/run.mjs
+  linux && !DISPLAY → re-exec self under `xvfb-run -a`
 e2e/setupFixtures.mjs
+  RSTACK_E2E_TOOLCHAIN_OVERRIDES="name=<absolute pkg dir>,…"   (malformed entry → throw)
+    pack → <fixture>/.toolchain/<name>.tgz
+    fixture pnpm-workspace.yaml: overrides { name: file:.toolchain/<name>.tgz }
+    per fixture: fail if pnpm-lock.yaml still holds name@<registry version>
+    per run: fail if no fixture resolved any override through a tarball

eco-ci's override goes into the consumer's root pnpm-workspace.yaml, but every E2E fixture is its own workspace root, so it never reaches them. The override therefore lives in each fixture's yaml and points at a tarball inside the fixture by relative path: the dependency-recovery suites copy a fixture to the OS tmpdir and reinstall with --frozen-lockfile, and only a fixture-relative file: spec survives that copy. Fixtures that do not depend on the package (e.g. rslint) are no-ops. The yaml is modified in place and not restored: the env is meant for disposable checkouts (ecosystem CI); locally, git checkout -- packages/vscode/e2e undoes it.

Evidence

  • Before: fixtures always install the pinned registry version; Linux had no display path.
    After: local runs on macOS with VSCODE_CLI=1 and RSTACK_E2E_TOOLCHAIN_OVERRIDES="@rstest/core=<rstest main checkout>/packages/core":
    [e2e] rstest-workspace-1: @rstest/core resolved through .toolchain (no registry version in pnpm-lock.yaml)
    [e2e] rstack: @rstest/core resolved through .toolchain (no registry version in pnpm-lock.yaml)   # transitive via rstack
    [e2e] rslint: @rstest/core not in graph, override unused
    [e2e] running slice: rstest   → Exit code: 0, Exit code: 0 (incl. dependency-recovery)
    [e2e] running slice: vscode   → Exit code: 0, Exit code: 0
    
    git status --porcelain after the rstest run (nothing untracked; .toolchain/ is ignored), then empty after git checkout -- packages/vscode/e2e:
     M packages/vscode/e2e/fixtures/rstack/pnpm-workspace.yaml
     M packages/vscode/e2e/fixtures/rstest-ownership/pnpm-workspace.yaml
     M packages/vscode/e2e/rstest/fixtures/workspace-1/pnpm-workspace.yaml
     M packages/vscode/e2e/rstest/fixtures/workspace-2/pnpm-workspace.yaml
    
    Error paths:
    $ RSTACK_E2E_TOOLCHAIN_OVERRIDES=@rstest/core node e2e/setupFixtures.mjs rstest
    Error: RSTACK_E2E_TOOLCHAIN_OVERRIDES: expected name=<absolute package dir>, got @rstest/core
    $ RSTACK_E2E_TOOLCHAIN_OVERRIDES=@rstest/core=<dir> node e2e/setupFixtures.mjs rslint
    Error: RSTACK_E2E_TOOLCHAIN_OVERRIDES: no fixture in this run resolved @rstest/core through a tarball; the override did nothing
    
    Smoke without the env: [smoke] OK, exit 0.
    xvfb evidence: three-platform run on the rebased branch, https://github.com/rstackjs/rstack-editor/actions/runs/38038161388 (E2E ubuntu-latest pass).

Merge Danger

Door: two-way

Blast Radius: E2E setup only (e2e/setupFixtures.mjs, e2e/run.mjs xvfb re-exec, CI comment); no extension source

On Linux without DISPLAY, e2e/run.mjs re-execs itself under `xvfb-run -a`,
so `pnpm test:e2e` works unchanged on rstack-ecosystem-ci's ubuntu-latest
runners (#40). This repo's CI stays Windows + macOS.
@fi3ework fi3ework changed the title ci: run E2E on Linux under xvfb ci: support running the E2E suites under xvfb on Linux Oct 10, 2026
@fi3ework fi3ework changed the title ci: support running the E2E suites under xvfb on Linux ci: let rstack-ecosystem-ci run the E2E suites Oct 10, 2026
RSTACK_E2E_TOOLCHAIN_OVERRIDES="name=<package dir>" packs each package into
<fixture>/.toolchain/ and adds a file: override to every fixture's pnpm-workspace.yaml,
so rstack-ecosystem-ci can run the suites against unreleased builds. Setup fails if the
lockfile still holds a registry version; the yaml is not restored (disposable checkouts).
@fi3ework

Copy link
Copy Markdown
Member Author

Closing without merging: the ecosystem-CI E2E suite this prepared for is not going ahead.

rstack-ecosystem-ci now runs this repo's test:unit + lint (type-checked) on every rstest main push (rstackjs/rstack-ecosystem-ci#99), and Renovate's rstack toolchain group opens a PR with the three-platform E2E on every release. That matches the protection rstack-cli has, and the remaining signal (a runtime-only break in a real VS Code, caught by neither type-check nor unit tests) did not justify keeping the fixture-injection code in setupFixtures.mjs plus ~8 min per upstream push.

Both commits (xvfb re-exec, RSTACK_E2E_TOOLCHAIN_OVERRIDES) only serve that suite, so neither lands. The branch stays as a reference if the decision is revisited.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant