Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 5 additions & 8 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
# Copilot Instructions

Follow the repository's canonical engineering skills under
`docs/engineering/skills/`.
Repository-wide guidance for AI agents lives in `AGENTS.md`. Read it first: it
lists the engineering skill to read for each kind of change, and its generated
Rules section lists every rule that governs this repository.

For tests, read `docs/engineering/skills/testing.md` before adding, moving, or
reviewing test files. Do not duplicate or override that testing guidance here.

For pull requests, read `docs/engineering/skills/github-prs.md` before opening,
replacing, or sharing a PR. This repository only accepts same-repository PRs
from branches in `PolicyEngine/policyengine-sim-api`; never create fork PRs.
This file is a Copilot adapter. Keep it a pointer: put guidance in `AGENTS.md`
or in a skill under `docs/engineering/skills/`, never here.
3 changes: 3 additions & 0 deletions .github/workflows/pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,9 @@ jobs:
- name: Test the rules registry tool
run: python -m unittest discover -s scripts/rules/tests -t .

- name: Install shellcheck (for the ci-scripts guard)
run: pip install "shellcheck-py==0.11.0.1"

- name: Run quality guards
run: python scripts/run_quality_guards.py

Expand Down
20 changes: 19 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ Before changing correlation identifiers, telemetry transport, spans, stage
names, logging, or observability failure handling, read
`docs/engineering/skills/observability.md`.

When adding or changing a GitHub Actions workflow step or a script under
`.github/scripts/`, read `docs/engineering/skills/ci-workflows.md`.

When adding, changing, or removing a rule, a guard test, a checker
configuration, or a CI job that enforces one, read
`docs/engineering/skills/rules-registry.md`.
Expand Down Expand Up @@ -67,6 +70,13 @@ same-repository draft PR.
(including the rules registry), Docker build, and local integration tests.
- There is currently no repository-wide changelog fragment requirement.

## Safety Boundaries

Do not commit directly to `main`.

Do not make production deployment changes or publish clients unless the user
explicitly asks for that operation.

## Rules

<!-- BEGIN GENERATED: rules (python -m scripts.rules render) -->
Expand Down Expand Up @@ -128,12 +138,20 @@ To add or change a rule, read `docs/engineering/skills/rules-registry.md`.
- `sim.quality.ruff-lint`: Python code passes ruff check with the rules selected in ruff.toml (ruff's defaults, made explicit); CI does not run ruff check yet. (advisory: ruff check (E4, E7, E9, F), not blocking)
- `sim.quality.tests-pass`: Each project's unit tests pass in its own locked environment. (blocking: CI job `test`)

### CI workflows and scripts

- `sim.ci.multiline-run-is-script`: A GitHub Actions run: block holds one line of code and no chained commands (&&, ||, ;); anything more lives in a tested script under .github/scripts/, called from a single-line step. (blocking: quality guard `ci-scripts`)
- `sim.ci.shell-scripts-pass-shellcheck`: Every shell script under .github/scripts/ passes shellcheck. (blocking: quality guard `ci-scripts`)
- `sim.ci.workflow-scripts-tested`: Every shell script a workflow calls is run by at least one test. (blocking: quality guard `ci-scripts`)

### Process and repository hygiene

- `sim.agents.adapters-point-to-agents-md`: CLAUDE.md and .github/copilot-instructions.md only point to AGENTS.md; skills and rules are listed in AGENTS.md alone. (blocking: quality guard `rules-registry`)
- `sim.process.agents-never-bypass`: Agents never bypass branch rules, even with admin credentials: no gh pr merge --admin and no bypass option; a blocked merge is reported to a human. (doc-only, not checked: follow it yourself)
- `sim.process.pr-tracking-issue`: Every PR has a tracking issue in this repository, and the first line of its description is Fixes #ISSUE_NUMBER. (doc-only, not checked: follow it yourself)
- `sim.process.rule-machinery-code-owned`: Changes to the rule machinery (rules/, scripts/rules/, scripts/run_quality_guards.py and CODEOWNERS) need approval from a core developer other than the author. (doc-only, not checked: follow it yourself)
- `sim.process.rules-registry`: Every rule is registered in rules/registry.toml, is enforced where it claims to be, and its rendered views are current. (blocking: quality guard `rules-registry`)
- `sim.process.same-repo-draft-prs`: PRs are opened as drafts from branches in PolicyEngine/policyengine-sim-api, never from forks, and start with 'Fixes #<issue>'. (doc-only, not checked: follow it yourself)
- `sim.process.same-repo-draft-prs`: PRs are opened as drafts from branches in PolicyEngine/policyengine-sim-api, never from forks. (doc-only, not checked: follow it yourself)
- `sim.process.stacked-prs-full-ci-at-bottom`: In a stack of pull requests, only the lowest unmerged one runs the full PR workflow and the Modal image smoke. Every pr.yml job is needed by the CI complete gate, which fails on the pull requests above it and otherwise passes only when every job passed, so a stack merges bottom-up and nothing merges untested. (blocking: pytest `projects/policyengine-simulation-executor/tests/test_ci_complete_gate.py`)

<!-- END GENERATED: rules -->
29 changes: 5 additions & 24 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,8 @@
# Claude Instructions

These instructions apply repository-wide.
Repository-wide guidance for AI agents lives in `AGENTS.md`. Read it first: it
lists the engineering skill to read for each kind of change, and its generated
Rules section lists every rule that governs this repository.

## Canonical Guidance

Repository-wide AI-facing engineering guidance lives in `AGENTS.md`.
Canonical skills live under `docs/engineering/skills/`.

Use those files as the source of truth. This file is a Claude adapter and should
stay thin; do not duplicate detailed testing, CI, formatting, or architecture
rules here.

## Required Skill Lookup

Before opening, replacing, or sharing a PR, read
`docs/engineering/skills/github-prs.md`.

When adding, moving, or reviewing tests, read
`docs/engineering/skills/testing.md`.

## Safety Boundaries

Do not commit directly to `main`.

Do not make production deployment changes or publish clients unless the user
explicitly asks for that operation.
This file is a Claude adapter. Keep it a pointer: put guidance in `AGENTS.md` or
in a skill under `docs/engineering/skills/`, never here.
1 change: 1 addition & 0 deletions changelog_entry.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
- bump: patch
changes:
added:
- Made CLAUDE.md and the Copilot instructions thin pointers to AGENTS.md, required workflow logic longer than one line to live in tested and shellchecked scripts, and registered the tracking-issue rule
- Added a rules registry (rules/registry.toml) with a standard-library tool that validates it, checks every rule is enforced where it claims, and renders docs/generated/rules.md and the Rules block in AGENTS.md
changed:
- Added CODEOWNERS for the rule machinery, pinned CI's ruff to 0.15.22 with an explicit rule selection, and documented that agents never bypass branch rules
Expand Down
2 changes: 2 additions & 0 deletions docs/engineering/skills/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ first, then keep adapters thin.

Current skills:

- `ci-workflows.md`: one line of code per workflow `run:` block, tested
workflow scripts, and shellcheck.
- `github-prs.md`: same-repository draft PR workflow, issue linkage, PR head
verification, and title conventions.
- `testing.md`: test placement, dependency boundaries, and expected validation
Expand Down
52 changes: 52 additions & 0 deletions docs/engineering/skills/ci-workflows.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# CI Workflows

Read this before adding or changing a GitHub Actions workflow step, or a script
under `.github/scripts/`.

## One line of code per `run:` block

A workflow `run:` block may hold only one line of code, with no chained
commands. Anything more goes in a shell script under `.github/scripts/`, called
from a single-line step:

```yaml
- name: Extract package versions
id: versions
run: .github/scripts/modal-extract-versions.sh projects/policyengine-simulation-executor
```

Inline shell in YAML can't be linted, unit-tested or run locally; a script can.

Every non-empty, non-comment line counts, including each line of a folded
(`>`) block and each backslash-continued line. Chaining commands on one line
with `&&`, `||` or `;` also counts as more than one line. Operators inside
quotes or `${{ }}` expressions don't count, and a single pipeline (`a | b`) is
one command.

## Writing a workflow script

- Start with `#!/usr/bin/env bash` and `set -euo pipefail`.
- Take inputs from arguments or environment variables, not from `${{ }}`
expressions inside the script.
- It must pass `shellcheck` (the `ci-scripts` quality guard runs it).

## Testing a workflow script

Every script a workflow calls must be referenced by name in at least one test.
Test it the way the existing scripts are tested: start it with `subprocess`,
put fake versions of the commands it calls (`gcloud`, `pip`, `curl`, ...) first
on `PATH`, and assert on what they were called with. Cover the success path and
at least one failure path.

Examples: `projects/policyengine-simulation-executor/tests/test_modal_scripts.py`
and `projects/policyengine-simulation-entry/tests/test_deployment_assets.py`.

## Enforcement and baselines

The `ci-scripts` quality guard (`python scripts/run_quality_guards.py`, or
`python -m scripts.rules ci-scripts`) checks all three rules above. Existing
violations are listed in `rules/baselines/ci-*.txt`. Anything new fails, and so
does a baseline entry that no longer applies, so each list only shrinks. When
you convert an existing multi-line block into a script, or add a missing test,
remove its baseline entry and lower the rule's `baseline.count` in
`rules/registry.toml`.
48 changes: 39 additions & 9 deletions docs/generated/rules.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
# Rules

Generated by `python -m scripts.rules render` (scripts/rules v0.1.0, tool sha256 `803fdc2cb511`) from `rules/registry.toml`. Do not edit by hand.
Generated by `python -m scripts.rules render` (scripts/rules v0.1.0, tool sha256 `afc55ba219f1`) from `rules/registry.toml`. Do not edit by hand.

Every rule that governs `policyengine-sim-api` is listed here with how it is enforced. Blocking rules fail CI. The Gaps section lists what is written down but not yet enforced or proven. To add or change a rule, see `docs/engineering/skills/rules-registry.md`.

## Summary

| Tier | enforced | ratcheted | advisory | doc-only | Total |
| --- | ---: | ---: | ---: | ---: | ---: |
| mechanical | 25 | 0 | 1 | 13 | 39 |
| mechanical | 26 | 3 | 1 | 14 | 44 |
| metric | 0 | 0 | 0 | 0 | 0 |
| judgment | 0 | 0 | 0 | 0 | 0 |

**Blocking:** 25 of 39 rules.
**Blocking:** 29 of 44 rules.

## Gaps

Expand All @@ -27,8 +27,9 @@ Every rule that governs `policyengine-sim-api` is listed here with how it is enf
- `sim.observability.id-is-diagnostic-only`: observability_id has diagnostic meaning only: never use it for idempotency, authorization, database identity, routing or business logic.
- `sim.observability.registered-stages-only`: Span names come from the stage plans in policyengine_simulation_observability.stages; runtime code never adds span-name string literals.
- `sim.process.agents-never-bypass`: Agents never bypass branch rules, even with admin credentials: no gh pr merge --admin and no bypass option; a blocked merge is reported to a human.
- `sim.process.pr-tracking-issue`: Every PR has a tracking issue in this repository, and the first line of its description is Fixes #ISSUE_NUMBER.
- `sim.process.rule-machinery-code-owned`: Changes to the rule machinery (rules/, scripts/rules/, scripts/run_quality_guards.py and CODEOWNERS) need approval from a core developer other than the author.
- `sim.process.same-repo-draft-prs`: PRs are opened as drafts from branches in PolicyEngine/policyengine-sim-api, never from forks, and start with 'Fixes #<issue>'.
- `sim.process.same-repo-draft-prs`: PRs are opened as drafts from branches in PolicyEngine/policyengine-sim-api, never from forks.
- `sim.quality.every-project-tested`: Every project and library with unit tests runs them in the PR test matrix.
- `sim.quality.ruff-format-tests`: Test code is formatted with ruff format too; CI currently checks only src directories.

Expand Down Expand Up @@ -59,7 +60,9 @@ Every rule that governs `policyengine-sim-api` is listed here with how it is enf

### Ratcheted (existing violations still tolerated)

- None.
- `sim.ci.multiline-run-is-script`: 20 remaining in `rules/baselines/ci-multiline-run.txt`
- `sim.ci.shell-scripts-pass-shellcheck`: 1 remaining in `rules/baselines/ci-shellcheck.txt`
- `sim.ci.workflow-scripts-tested`: 4 remaining in `rules/baselines/ci-untested-scripts.txt`

### Exceptions and review dates

Expand Down Expand Up @@ -259,33 +262,60 @@ flowchart LR
e_CI_job_test --> j_CI_pr_yml_test
```

## CI workflows and scripts

| Rule | Statement | Tier | Status | Blocking | Enforced by | CI | Proof |
| --- | --- | --- | --- | --- | --- | --- | --- |
| `sim.ci.multiline-run-is-script` | A GitHub Actions run: block holds one line of code and no chained commands (&&, \|\|, ;); anything more lives in a tested script under .github/scripts/, called from a single-line step. | mechanical | ratcheted | yes | quality guard `ci-scripts` | `pr.yml:quality-guards` | yes |
| `sim.ci.shell-scripts-pass-shellcheck` | Every shell script under .github/scripts/ passes shellcheck. | mechanical | ratcheted | yes | quality guard `ci-scripts` | `pr.yml:quality-guards` | yes |
| `sim.ci.workflow-scripts-tested` | Every shell script a workflow calls is run by at least one test. | mechanical | ratcheted | yes | quality guard `ci-scripts` | `pr.yml:quality-guards` | yes |

```mermaid
flowchart LR
r_sim_ci_multiline_run_is_script("sim.ci.multiline-run-is-script (ratcheted)")
e_quality_guard_ci_scripts["quality guard ci-scripts"]
j_CI_pr_yml_quality_guards["CI pr.yml:quality-guards"]
r_sim_ci_shell_scripts_pass_shellcheck("sim.ci.shell-scripts-pass-shellcheck (ratcheted)")
r_sim_ci_workflow_scripts_tested("sim.ci.workflow-scripts-tested (ratcheted)")
r_sim_ci_multiline_run_is_script --> e_quality_guard_ci_scripts
e_quality_guard_ci_scripts --> j_CI_pr_yml_quality_guards
r_sim_ci_shell_scripts_pass_shellcheck --> e_quality_guard_ci_scripts
r_sim_ci_workflow_scripts_tested --> e_quality_guard_ci_scripts
```

## Process and repository hygiene

| Rule | Statement | Tier | Status | Blocking | Enforced by | CI | Proof |
| --- | --- | --- | --- | --- | --- | --- | --- |
| `sim.agents.adapters-point-to-agents-md` | CLAUDE.md and .github/copilot-instructions.md only point to AGENTS.md; skills and rules are listed in AGENTS.md alone. | mechanical | enforced | yes | quality guard `rules-registry` | `pr.yml:quality-guards` | yes |
| `sim.process.agents-never-bypass` | Agents never bypass branch rules, even with admin credentials: no gh pr merge --admin and no bypass option; a blocked merge is reported to a human. | mechanical | doc-only | no | nothing | — | — |
| `sim.process.pr-tracking-issue` | Every PR has a tracking issue in this repository, and the first line of its description is Fixes #ISSUE_NUMBER. | mechanical | doc-only | no | nothing | — | — |
| `sim.process.rule-machinery-code-owned` | Changes to the rule machinery (rules/, scripts/rules/, scripts/run_quality_guards.py and CODEOWNERS) need approval from a core developer other than the author. | mechanical | doc-only | no | nothing | — | — |
| `sim.process.rules-registry` | Every rule is registered in rules/registry.toml, is enforced where it claims to be, and its rendered views are current. | mechanical | enforced | yes | quality guard `rules-registry` | `pr.yml:quality-guards` | **unproven** |
| `sim.process.same-repo-draft-prs` | PRs are opened as drafts from branches in PolicyEngine/policyengine-sim-api, never from forks, and start with 'Fixes #<issue>'. | mechanical | doc-only | no | nothing | — | — |
| `sim.process.same-repo-draft-prs` | PRs are opened as drafts from branches in PolicyEngine/policyengine-sim-api, never from forks. | mechanical | doc-only | no | nothing | — | — |
| `sim.process.stacked-prs-full-ci-at-bottom` | In a stack of pull requests, only the lowest unmerged one runs the full PR workflow and the Modal image smoke. Every pr.yml job is needed by the CI complete gate, which fails on the pull requests above it and otherwise passes only when every job passed, so a stack merges bottom-up and nothing merges untested. | mechanical | enforced | yes | pytest `projects/policyengine-simulation-executor/tests/test_ci_complete_gate.py` | `pr.yml:test`, `pr.yml:ci-complete` | yes |

```mermaid
flowchart LR
r_sim_agents_adapters_point_to_agents_md("sim.agents.adapters-point-to-agents-md (enforced)")
e_quality_guard_rules_registry["quality guard rules-registry"]
j_CI_pr_yml_quality_guards["CI pr.yml:quality-guards"]
r_sim_process_agents_never_bypass("sim.process.agents-never-bypass (doc-only)")
gap_unenforced>"not enforced"]
r_sim_process_pr_tracking_issue("sim.process.pr-tracking-issue (doc-only)")
r_sim_process_rule_machinery_code_owned("sim.process.rule-machinery-code-owned (doc-only)")
r_sim_process_rules_registry("sim.process.rules-registry (enforced)")
e_quality_guard_rules_registry["quality guard rules-registry"]
j_CI_pr_yml_quality_guards["CI pr.yml:quality-guards"]
r_sim_process_same_repo_draft_prs("sim.process.same-repo-draft-prs (doc-only)")
r_sim_process_stacked_prs_full_ci_at_bottom("sim.process.stacked-prs-full-ci-at-bottom (enforced)")
e_pytest_projects_policyengine_simulation_executor_tests_test_ci_complete_gate_py["pytest projects/policyengine-simulation-executor/tests/test_ci_complete_gate.py"]
j_CI_pr_yml_test["CI pr.yml:test"]
j_CI_pr_yml_ci_complete["CI pr.yml:ci-complete"]
r_sim_agents_adapters_point_to_agents_md --> e_quality_guard_rules_registry
e_quality_guard_rules_registry --> j_CI_pr_yml_quality_guards
r_sim_process_agents_never_bypass -.-> gap_unenforced
r_sim_process_pr_tracking_issue -.-> gap_unenforced
r_sim_process_rule_machinery_code_owned -.-> gap_unenforced
r_sim_process_rules_registry --> e_quality_guard_rules_registry
e_quality_guard_rules_registry --> j_CI_pr_yml_quality_guards
r_sim_process_same_repo_draft_prs -.-> gap_unenforced
r_sim_process_stacked_prs_full_ci_at_bottom --> e_pytest_projects_policyengine_simulation_executor_tests_test_ci_complete_gate_py
e_pytest_projects_policyengine_simulation_executor_tests_test_ci_complete_gate_py --> j_CI_pr_yml_test
Expand Down
Loading
Loading