Skip to content

docs(claude-md): scope test verification by the types a change touches - #2389

Merged
datlechin merged 1 commit into
mainfrom
chore/verification-scope-rule
Aug 23, 2026
Merged

docs(claude-md): scope test verification by the types a change touches#2389
datlechin merged 1 commit into
mainfrom
chore/verification-scope-rule

Conversation

@datlechin

Copy link
Copy Markdown
Member

Phase 4 says to run "the suites you touched and their neighbours, never the whole target". Read literally, that means the suites whose files you edited. It should mean the suites that own the types you changed, and the difference just cost a red CI run.

What happened

A change to PersistedTab.init(from:) in #2386 made an unknown tabType decode as .query instead of throwing. The comment justifying it claimed the throw "would fail the whole aggregate file". That was never checked and is false: TabDiskState.init(from:) decodes through [LossyTab].compactMap(\.value), so a bad tab is dropped and the rest survive.

TabDiskStateDecodingTests exists to pin exactly that, in a test named "Drops tabs with an unknown legacy tab type and keeps the valid ones". Nobody ran it, because no file in that suite was edited. Local was green across fifteen suites; CI was red.

The change was also strictly worse than what it replaced. A tab written by a newer build would come back as an empty query tab carrying a title like Procedure: public.f(date) rather than being dropped.

Two rules added

Scope the suite list by type, not by file. grep -rl "TypeName" TableProTests names the suites that will judge the change, and it takes a second. A shared model always has tests you never opened.

When CI fails and local passed, do not reach for the whole target. A local full run of TableProTests reports around 60 failures with nothing to do with the change: quarantined suites, locale, pasteboard, network and timer-dependent cases. That number drowns the one failure that matters. Compare against a baseline instead: read the CI log for the real count, then run the suspect suites on a worktree at the merge base. A suite that fails in both is not yours.

That second rule was worth writing down because chasing it is what turned a one-line revert into a long detour. The local full run showed 64 failures; CI recorded exactly one issue. Running StructureChangeManagerUndoTests and SaveCompletionTests on a merge-base worktree reproduced their failures there too, which settled them as noise in one step.

Verification

The rule is prose in SKILL.md; there is nothing to build or test. git diff is four added lines.

Follows #2387, which made docs/STYLE.md discoverable and added verify.sh docs.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@datlechin
datlechin merged commit 18ecc68 into main Aug 23, 2026
10 checks passed
@datlechin
datlechin deleted the chore/verification-scope-rule branch August 23, 2026 02:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant