Skip to content

Point agent adapters to AGENTS.md and move multi-line workflow steps into tested scripts - #3890

Draft
anth-volk wants to merge 5 commits into
rules-registryfrom
rules-followups
Draft

anth-volk wants to merge 5 commits into
rules-registryfrom
rules-followups

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #3889

Summary

The top of the rules stack: Phase 0 (#3887) → rules registry (#3884) → this PR, which covers Phase 1 steps 1.3, 1.5 and 1.6. The sim counterpart is PolicyEngine/policyengine-sim-api#757.

  • 1.3: one source of agent instructions.

    • CLAUDE.md and .github/copilot-instructions.md are now short pointers to AGENTS.md. Both had duplicated the skill list and drifted from it.
    • The migration safety boundary moves from CLAUDE.md into AGENTS.md, so nothing is lost.
    • test_alembic_skill.py now checks AGENTS.md only.
    • The rules-registry guard enforces this through a new [agents] table: an adapter must mention AGENTS.md, must not reference other docs or carry the generated rules block, and must stay within 20 non-empty lines.
    • It's registered as api.agents.adapters-point-to-agents-md (enforced, with proof).
  • 1.5: api.process.pr-tracking-issue is registered as doc-only (tracking issue plus Fixes #N first line). CI enforcement comes in step 2.8.

  • 1.6: workflow logic lives in tested scripts. A new ci-scripts quality guard, configured in a [ci_scripts] table, runs three checks:

    • run: blocks with more than one line of code. Every non-empty line counts, including each line of a folded > block and each backslash-continued line. Commands chained with &&, || or ; also count as more than one line. Operators inside quotes or ${{ }} are ignored, and a single pipeline is one command.
    • Workflow scripts that no test file names.
    • shellcheck findings in .github/scripts/*.sh.

    Each check is ratcheted against a baseline in rules/baselines/: 10 multi-line or chained blocks, 1 untested script (cleanup_cloud_run_images.sh) and 1 shellcheck finding (deploy_observability_collector.sh, SC2129). Anything new fails, and so does a stale baseline entry, so the lists only shrink.

    The rules are api.ci.multiline-run-is-script, api.ci.workflow-scripts-tested and api.ci.shell-scripts-pass-shellcheck, all ratcheted, with proofs. The quality-guards jobs install a pinned shellcheck-py==0.11.0.1. The rule and how to test a workflow script are documented in docs/engineering/skills/ci-workflows.md.

  • Tool changes: the workflow reader now records each step's run: block. The tool stays byte-identical with sim's copy; the tool sha256 in rules.md matches.

Testing

  • python -m unittest discover -s scripts/rules/tests -t .: 74 tests pass, including the three CI-script proofs and the adapter proof. The shellcheck proof skips locally when shellcheck is missing, but fails in CI if it is missing there.

  • With shellcheck installed, python scripts/run_quality_guards.py passes all three guards: migration-contracts, rules-registry and ci-scripts.

  • python -m scripts.rules guard --base-ref origin/master passes.

  • pytest on the API tests that read workflows, scripts or adapters, plus the CRUD boundary tests: 204 pass.

  • Negative checks, each reverted afterwards:

    Change Result
    Make the changelog step two lines api.ci.multiline-run-is-script fails
    Call a new script that no test names api.ci.workflow-scripts-tested fails
    Add an unquoted variable to health_check.sh api.ci.shell-scripts-pass-shellcheck fails
    List a skill in CLAUDE.md the adapters check fails
  • ruff format --check . passes. actionlint reports nothing new.

🤖 Generated with Claude Code

anth-volk and others added 4 commits October 12, 2026 23:36
Add two checks to scripts/rules. The rules-registry guard now verifies that
agent adapters ([agents] in the registry) only point to AGENTS.md. A new
ci-scripts quality guard ([ci_scripts]) fails on run: blocks longer than one
logical line, workflow scripts that no test runs, and shellcheck findings,
each ratcheted against a baseline file that may only shrink. The workflow
reader now records each step's run: block.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both adapters duplicated the skill list and had drifted from AGENTS.md
(observability and the rules registry were missing). They now only point to
AGENTS.md; the safety boundary moves into AGENTS.md. The alembic skill test
checks AGENTS.md instead of every adapter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Document the rule in docs/engineering/skills/ci-workflows.md, install a
pinned shellcheck in the quality-guards jobs, and record today's exceptions
as baselines: 5 multi-line run: blocks, 1 untested script and 1 script with
a shellcheck finding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Steps 1.3, 1.5 and 1.6: api.agents.adapters-point-to-agents-md (enforced,
with proof), api.process.pr-tracking-issue (doc-only), and three ratcheted
api.ci.* rules with proofs. AGENTS.md gains the safety boundary and a
pointer to the CI workflows skill.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A folded (>) block and backslash-continued lines now count line by line,
and chaining commands on one line with &&, || or ; counts as more than one
line. Operators inside quotes or ${{ }} expressions are ignored, and a
single pipeline is still one command. The multi-line baseline grows from 5
to 10 entries: 2 folded blocks and 3 chained one-liners.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.56%. Comparing base (880260a) to head (344702f).

Additional details and impacted files
@@               Coverage Diff               @@
##           rules-registry    #3890   +/-   ##
===============================================
  Coverage           87.56%   87.56%           
===============================================
  Files                 198      198           
  Lines               12211    12211           
  Branches             2159     2159           
===============================================
  Hits                10692    10692           
  Misses                922      922           
  Partials              597      597           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch was successfully deployed

1 active deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Point agent adapters to AGENTS.md and move multi-line workflow steps into tested scripts

1 participant