Skip to content

fix(ci): preserve manual Android suite timing - #3554

Merged
thymikee merged 2 commits into
mainfrom
t3/android-manual-shell-3077
Oct 14, 2026
Merged

thymikee merged 2 commits into
mainfrom
t3/android-manual-shell-3077

Conversation

@thymikee

@thymikee thymikee commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Summary

The pinned emulator action starts a separate shell for each script line, losing Android setup/full-suite timestamps and reporting negative setup and zero suite durations.

Run the existing pipeline through one shell using the action's emulator serial. Record both timings even if full E2E fails, then preserve its exit code and stop before ordinary replay. Four files changed; prerequisite for #3555 and #3077 validation.

Validation

Tested head: f660374efbf42d270e9edcabaa8e3bd8f866510e, rebased onto main 15cfe9fa2 including merged test repairs #3590. Both layer commits are unchanged by range-diff.

Regression Before After
Lost timestamp state 8255c735: FAIL (-90s/0s) PASS (10s/25s)
Failed full suite loses timing f9260b1ee: 1 FAIL / 2 PASS 3 PASS; exit 13, no replay

Three focused cases cover two independent serials; hardcoding the first serial fails both other-serial cases. Mutation restored. Frozen install/build and exact-head pnpm check:affected --base 15cfe9fa2 --run pass all runnable gates, including 1,760 Vitest files / 14,891 passed tests / one skip. Fresh combined-stack review has no findings. Integration passed. iOS smoke failed during runner startup after preflight passed; one failed-job rerun was requested based on retained Xcode worker/listener evidence. Startup-stall cause remains unknown; no budgets or assertions changed. Coverage and remaining checks are pending.

Earlier hosted Android, runtime a3aba45a, reports setup 92s/full 398s. Pure workflow repair; no new device behavior.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread scripts/replays/android-manual.test.ts Outdated
Comment thread scripts/replays/android-manual.sh Outdated
@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.20 MB 5.20 MB 0 B
Package (unpacked) 5.20 MB 5.20 MB 0 B
Package (download) 1.56 MB 1.56 MB -2 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 17.3 ms 17.2 ms -0.2 ms
CLI --help 50.0 ms 48.9 ms -1.1 ms

@thymikee
thymikee force-pushed the t3/android-manual-shell-3077 branch from f9260b1 to 22fb210 Compare October 13, 2026 00:32
@thymikee thymikee changed the title fix(ci): preserve manual Android timing in one shell fix(ci): preserve manual Android suite timing Oct 13, 2026
@thymikee

Copy link
Copy Markdown
Member Author

I reviewed 22fb210 and found no problems in the code. The manual Android replay job now keeps the suite timing, and the script keeps the real exit status after it prints both timings. The test uses distinct serials, so a hardcoded serial would fail. All 22 checks pass, and there are no conflicts. Nothing else blocks merge. The earlier hosted manual Android run reproduced the timings, but I did not run the vitest case or the failure path on the hosted emulator, so exit-code preservation rests on the stubbed test and reading the code.

Both inline threads from cubic-dev-ai are fixed at this commit, so you can resolve them: #3554 (comment) (the test now uses emulator-7000 and emulator-8100, and the adb stub rejects a mismatch) and #3554 (comment) (the script saves the status with || FULL_STATUS=$?, prints both timings, then exits with it).

Two small, optional notes. The stubbed adb devices branch in android-manual.test.ts is never reached now, because the script no longer calls it. A short comment in the script saying the action exports ANDROID_SERIAL would help the next reader. android.yml still guesses the serial, which could be a separate change.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Oct 13, 2026
@thymikee
thymikee force-pushed the t3/android-manual-shell-3077 branch from 22fb210 to a85a0f0 Compare October 13, 2026 11:42
@thymikee

Copy link
Copy Markdown
Member Author

Thanks for the update. The Android serial and exit-code handling from the earlier pass (22fb210) are now fixed at a85a0f0, and I found nothing new in the code. The two inline threads on the distinct emulator serials and on keeping the full-suite exit status both look fixed at this head, so you can resolve them: #3554 (comment) and #3554 (comment).

I judged the stubbed test from the diff and the script. I did not run the vitest case or the hosted emulator path. I also did not open hosted run 38324090019, so I have not confirmed that the emulator action exports ANDROID_SERIAL to the script. The exit-code behavior on a real emulator rests on the stubbed test and on reading the code. I did not review the 122-file rebase delta, since it is upstream content.

No conflicts. The two Smoke Tests runs and Mutants (snapshot-occlusion) are still pending, not failed. This patch only touches the manual workflow, the Android manual script and its test, and a vitest exclusion, so those jobs can only be affected by the rebased-in base changes. Merge once those jobs finish green.

@thymikee
thymikee force-pushed the t3/android-manual-shell-3077 branch from a85a0f0 to f660374 Compare October 13, 2026 18:20
@thymikee
thymikee removed this pull request from stack #3556 October 14, 2026 06:25
@thymikee
thymikee merged commit 3d48bb9 into main Oct 14, 2026
22 of 23 checks passed
@thymikee
thymikee deleted the t3/android-manual-shell-3077 branch October 14, 2026 07:55
@github-actions

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-14 08:00 UTC

thymikee added a commit that referenced this pull request Oct 14, 2026
* origin/main: (83 commits)
  feat(ios): interpret React Native accessibility value semantics (#3589)
  fix(android): reinstall and reselect the test IME helper after it disappears (#3606)
  feat(snapshot): carry provider keyboard observations through acquisition (#3593)
  chore(web): bump managed agent-browser backend to 0.39.0 (#3591)
  fix(fold): report a dispatched pose as unverified when devicectl has no hinge-angle stream (#3587)
  feat(android): expose system-bar membership on snapshot nodes (#3585)
  fix(android): align ADB and Appium field values (#3584)
  test(closure): read committed JSON modules on the merge-base side (#3602)
  fix(ci): preserve manual Android suite timing (#3554)
  refactor(daemon): make the layer folders the declaration (PR 10) (#3520)
  refactor(move): extract the daemon server into @agent-device/daemon (11b) (#3546)
  refactor(daemon): package-local runtime fakes for the remaining daemon ports [11a-4] (#3544)
  refactor(daemon): cut daemon test edges into root data fixtures and fakes [11a-3] (#3517)
  refactor(11a-2): package-local fakes at the daemon ports; root-own command-surface composition tests (#3519)
  test(replay): verify Settings queries and bound scrolling (#3560)
  feat(snapshot): measure Android Appium viewport (#3586)
  fix(ios): verify a fill whose unidentified field moved while it kept focus (#3596)
  refactor(coverage): one fallow config with per-function CRAP ceilings (#3588)
  test: bound CLI promotion cases and align provider budget (#3590)
  fix(metro): report what the dev-server stop proved instead of assuming success (#3575)
  ...
thymikee added a commit that referenced this pull request Oct 14, 2026
* origin/main: (83 commits)
  feat(ios): interpret React Native accessibility value semantics (#3589)
  fix(android): reinstall and reselect the test IME helper after it disappears (#3606)
  feat(snapshot): carry provider keyboard observations through acquisition (#3593)
  chore(web): bump managed agent-browser backend to 0.39.0 (#3591)
  fix(fold): report a dispatched pose as unverified when devicectl has no hinge-angle stream (#3587)
  feat(android): expose system-bar membership on snapshot nodes (#3585)
  fix(android): align ADB and Appium field values (#3584)
  test(closure): read committed JSON modules on the merge-base side (#3602)
  fix(ci): preserve manual Android suite timing (#3554)
  refactor(daemon): make the layer folders the declaration (PR 10) (#3520)
  refactor(move): extract the daemon server into @agent-device/daemon (11b) (#3546)
  refactor(daemon): package-local runtime fakes for the remaining daemon ports [11a-4] (#3544)
  refactor(daemon): cut daemon test edges into root data fixtures and fakes [11a-3] (#3517)
  refactor(11a-2): package-local fakes at the daemon ports; root-own command-surface composition tests (#3519)
  test(replay): verify Settings queries and bound scrolling (#3560)
  feat(snapshot): measure Android Appium viewport (#3586)
  fix(ios): verify a fill whose unidentified field moved while it kept focus (#3596)
  refactor(coverage): one fallow config with per-function CRAP ceilings (#3588)
  test: bound CLI promotion cases and align provider budget (#3590)
  fix(metro): report what the dev-server stop proved instead of assuming success (#3575)
  ...

# Conflicts:
#	packages/daemon/src/server/http/daemon-runtime-lease-expiry.test.ts
#	packages/daemon/src/server/http/daemon-runtime.ts
#	packages/daemon/src/sessions/__tests__/lease-lifecycle.test.ts
#	packages/daemon/src/sessions/lease-lifecycle.ts
thymikee added a commit that referenced this pull request Oct 14, 2026
* origin/main: (79 commits)
  feat(ios): interpret React Native accessibility value semantics (#3589)
  fix(android): reinstall and reselect the test IME helper after it disappears (#3606)
  feat(snapshot): carry provider keyboard observations through acquisition (#3593)
  chore(web): bump managed agent-browser backend to 0.39.0 (#3591)
  fix(fold): report a dispatched pose as unverified when devicectl has no hinge-angle stream (#3587)
  feat(android): expose system-bar membership on snapshot nodes (#3585)
  fix(android): align ADB and Appium field values (#3584)
  test(closure): read committed JSON modules on the merge-base side (#3602)
  fix(ci): preserve manual Android suite timing (#3554)
  refactor(daemon): make the layer folders the declaration (PR 10) (#3520)
  refactor(move): extract the daemon server into @agent-device/daemon (11b) (#3546)
  refactor(daemon): package-local runtime fakes for the remaining daemon ports [11a-4] (#3544)
  refactor(daemon): cut daemon test edges into root data fixtures and fakes [11a-3] (#3517)
  refactor(11a-2): package-local fakes at the daemon ports; root-own command-surface composition tests (#3519)
  test(replay): verify Settings queries and bound scrolling (#3560)
  feat(snapshot): measure Android Appium viewport (#3586)
  fix(ios): verify a fill whose unidentified field moved while it kept focus (#3596)
  refactor(coverage): one fallow config with per-function CRAP ceilings (#3588)
  test: bound CLI promotion cases and align provider budget (#3590)
  fix(metro): report what the dev-server stop proved instead of assuming success (#3575)
  ...
thymikee added a commit that referenced this pull request Oct 14, 2026
* origin/main: (79 commits)
  feat(ios): interpret React Native accessibility value semantics (#3589)
  fix(android): reinstall and reselect the test IME helper after it disappears (#3606)
  feat(snapshot): carry provider keyboard observations through acquisition (#3593)
  chore(web): bump managed agent-browser backend to 0.39.0 (#3591)
  fix(fold): report a dispatched pose as unverified when devicectl has no hinge-angle stream (#3587)
  feat(android): expose system-bar membership on snapshot nodes (#3585)
  fix(android): align ADB and Appium field values (#3584)
  test(closure): read committed JSON modules on the merge-base side (#3602)
  fix(ci): preserve manual Android suite timing (#3554)
  refactor(daemon): make the layer folders the declaration (PR 10) (#3520)
  refactor(move): extract the daemon server into @agent-device/daemon (11b) (#3546)
  refactor(daemon): package-local runtime fakes for the remaining daemon ports [11a-4] (#3544)
  refactor(daemon): cut daemon test edges into root data fixtures and fakes [11a-3] (#3517)
  refactor(11a-2): package-local fakes at the daemon ports; root-own command-surface composition tests (#3519)
  test(replay): verify Settings queries and bound scrolling (#3560)
  feat(snapshot): measure Android Appium viewport (#3586)
  fix(ios): verify a fill whose unidentified field moved while it kept focus (#3596)
  refactor(coverage): one fallow config with per-function CRAP ceilings (#3588)
  test: bound CLI promotion cases and align provider budget (#3590)
  fix(metro): report what the dev-server stop proved instead of assuming success (#3575)
  ...
thymikee added a commit that referenced this pull request Oct 14, 2026
* origin/main: (24 commits)
  feat(ios): interpret React Native accessibility value semantics (#3589)
  fix(android): reinstall and reselect the test IME helper after it disappears (#3606)
  feat(snapshot): carry provider keyboard observations through acquisition (#3593)
  chore(web): bump managed agent-browser backend to 0.39.0 (#3591)
  fix(fold): report a dispatched pose as unverified when devicectl has no hinge-angle stream (#3587)
  feat(android): expose system-bar membership on snapshot nodes (#3585)
  fix(android): align ADB and Appium field values (#3584)
  test(closure): read committed JSON modules on the merge-base side (#3602)
  fix(ci): preserve manual Android suite timing (#3554)
  refactor(daemon): make the layer folders the declaration (PR 10) (#3520)
  refactor(move): extract the daemon server into @agent-device/daemon (11b) (#3546)
  refactor(daemon): package-local runtime fakes for the remaining daemon ports [11a-4] (#3544)
  refactor(daemon): cut daemon test edges into root data fixtures and fakes [11a-3] (#3517)
  refactor(11a-2): package-local fakes at the daemon ports; root-own command-surface composition tests (#3519)
  test(replay): verify Settings queries and bound scrolling (#3560)
  feat(snapshot): measure Android Appium viewport (#3586)
  fix(ios): verify a fill whose unidentified field moved while it kept focus (#3596)
  refactor(coverage): one fallow config with per-function CRAP ceilings (#3588)
  test: bound CLI promotion cases and align provider budget (#3590)
  fix(metro): report what the dev-server stop proved instead of assuming success (#3575)
  ...

# Conflicts:
#	src/remote/daemon-artifacts.ts
thymikee added a commit that referenced this pull request Oct 14, 2026
* origin/main: (83 commits)
  feat(ios): interpret React Native accessibility value semantics (#3589)
  fix(android): reinstall and reselect the test IME helper after it disappears (#3606)
  feat(snapshot): carry provider keyboard observations through acquisition (#3593)
  chore(web): bump managed agent-browser backend to 0.39.0 (#3591)
  fix(fold): report a dispatched pose as unverified when devicectl has no hinge-angle stream (#3587)
  feat(android): expose system-bar membership on snapshot nodes (#3585)
  fix(android): align ADB and Appium field values (#3584)
  test(closure): read committed JSON modules on the merge-base side (#3602)
  fix(ci): preserve manual Android suite timing (#3554)
  refactor(daemon): make the layer folders the declaration (PR 10) (#3520)
  refactor(move): extract the daemon server into @agent-device/daemon (11b) (#3546)
  refactor(daemon): package-local runtime fakes for the remaining daemon ports [11a-4] (#3544)
  refactor(daemon): cut daemon test edges into root data fixtures and fakes [11a-3] (#3517)
  refactor(11a-2): package-local fakes at the daemon ports; root-own command-surface composition tests (#3519)
  test(replay): verify Settings queries and bound scrolling (#3560)
  feat(snapshot): measure Android Appium viewport (#3586)
  fix(ios): verify a fill whose unidentified field moved while it kept focus (#3596)
  refactor(coverage): one fallow config with per-function CRAP ceilings (#3588)
  test: bound CLI promotion cases and align provider budget (#3590)
  fix(metro): report what the dev-server stop proved instead of assuming success (#3575)
  ...
thymikee added a commit that referenced this pull request Oct 14, 2026
* origin/main: (83 commits)
  feat(ios): interpret React Native accessibility value semantics (#3589)
  fix(android): reinstall and reselect the test IME helper after it disappears (#3606)
  feat(snapshot): carry provider keyboard observations through acquisition (#3593)
  chore(web): bump managed agent-browser backend to 0.39.0 (#3591)
  fix(fold): report a dispatched pose as unverified when devicectl has no hinge-angle stream (#3587)
  feat(android): expose system-bar membership on snapshot nodes (#3585)
  fix(android): align ADB and Appium field values (#3584)
  test(closure): read committed JSON modules on the merge-base side (#3602)
  fix(ci): preserve manual Android suite timing (#3554)
  refactor(daemon): make the layer folders the declaration (PR 10) (#3520)
  refactor(move): extract the daemon server into @agent-device/daemon (11b) (#3546)
  refactor(daemon): package-local runtime fakes for the remaining daemon ports [11a-4] (#3544)
  refactor(daemon): cut daemon test edges into root data fixtures and fakes [11a-3] (#3517)
  refactor(11a-2): package-local fakes at the daemon ports; root-own command-surface composition tests (#3519)
  test(replay): verify Settings queries and bound scrolling (#3560)
  feat(snapshot): measure Android Appium viewport (#3586)
  fix(ios): verify a fill whose unidentified field moved while it kept focus (#3596)
  refactor(coverage): one fallow config with per-function CRAP ceilings (#3588)
  test: bound CLI promotion cases and align provider budget (#3590)
  fix(metro): report what the dev-server stop proved instead of assuming success (#3575)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant