Update optional derivation narration to Claude Sonnet 5.5

[MaxGhenis]

Fixes #580

Optional derivation narration defaults to an outdated Sonnet model and sends temperature with a tight token budget. Update it to Sonnet 5.5 with explicit low effort and compatible requests. Rebuilt from the 2026-09-28 model-pin audit against current main, including its approved DECIDE-to-FIX override for opt-in narration text.

File	Old → new	Role
src/policyengine/derivations/narrate.py	claude-sonnet-4-6 → claude-sonnet-5-5	Shared default for synchronous and asynchronous narration
tests/test_derivations.py	Sonnet 4.6 expectation → Sonnet 5.5 expectations	Mocked default assertion and offline compatibility coverage

Pin LiteLLM 1.103.0 in the optional narration and development extras. Regenerate uv.lock from main: 27 new packages belong to LiteLLM's dependency closure, and its requirement upgrades pydantic-settings from 2.10.1 to 2.15.0. Preserve main's project and country-package versions: policyengine 6.2.6, core 3.33.0, UK 2.125.1, US 2.2.1, and SPM calculator 1.0.0.

Both supported Sonnet routes use explicit Anthropic routing, native output_config.effort=low, and a 4,096-token default budget for the short narrative, including thinking. They omit thinking so adaptive thinking applies. No request on these routes sends temperature, top_p, top_k, legacy thinking settings, forced tool choice, or assistant prefill. Explicit temperature is rejected for these routes. LiteLLM joins response blocks by type; narration rejects refusal and empty text. Because this pinned wrapper silently discards effort when parameter dropping is enabled, narration fails before requesting in that configuration. Real sync/async SDK calls through HTTPX MockTransport verify the final JSON, both model spellings, multiple text blocks, and refusal handling. The provider-prefix convention follows LiteLLM's Anthropic routing documentation.

No environment variable selects the model default; callers can override it with model=. Explicit custom-model temperature and token arguments remain supported. Keep LITELLM_DROP_PARAMS unset or false and litellm.drop_params false. ANTHROPIC_API_BASE and ANTHROPIC_BASE_URL can change the endpoint, not the explicit model ID. Arbitrary custom-model overrides remain caller-controlled.

Validation used uv 0.12.14, a locked Python 3.13.9 development environment, and at most four pytest workers. Caches and fixtures were kept under scratch and removed afterward.

• POLICYENGINE_SKIP_COUNTRY_IMPORTS=1 PYTEST_ADDOPTS='-n 0 -o cache_dir=.lane-scratch/derivations-cache --basetemp=.lane-scratch/derivations-temp' uv run --no-sync pytest --noconftest tests/test_derivations.py -v: 36 passed.
• POLICYENGINE_SKIP_COUNTRY_IMPORTS=1 uv run --no-sync pytest --noconftest tests/test_derivations.py -k narrat -v: 30 passed on Python 3.13.9 with four workers.
• .lane-scratch/compat-current/py3.11/bin/python -m pytest --noconftest tests/test_derivations.py -k narrat -v -o cache_dir=.lane-scratch/compat-current/pytest-cache-py3.11 --basetemp=.lane-scratch/compat-current/pytest-tmp-py3.11: 30 passed, 6 deselected.
• .lane-scratch/compat-current/py3.12/bin/python -m pytest --noconftest tests/test_derivations.py -k narrat -v -o cache_dir=.lane-scratch/compat-current/pytest-cache-py3.12 --basetemp=.lane-scratch/compat-current/pytest-tmp-py3.12: 30 passed, 6 deselected.
• .lane-scratch/compat-current/py3.14/bin/python -m pytest --noconftest tests/test_derivations.py -k narrat -v -o cache_dir=.lane-scratch/compat-current/pytest-cache-py3.14 --basetemp=.lane-scratch/compat-current/pytest-tmp-py3.14: 30 passed, 6 deselected. These three runs disabled country imports, enabled the local cost map, and removed provider credentials.
• PYTEST_ADDOPTS='-n 4 -o cache_dir=.lane-scratch/pytest-cache --basetemp=.lane-scratch/pytest-temp' PYTHONUNBUFFERED=1 make test: 1,306 passed, 9 skipped, 1 failed, 89% coverage. The sole failure is a Git-tag fixture discovering parent-checkout tags through its scratch directory; origin/main's files reproduce it.
• GIT_CEILING_DIRECTORIES="$PWD/.lane-scratch/bump-fixed-temp" POLICYENGINE_SKIP_COUNTRY_IMPORTS=1 PYTEST_ADDOPTS='-n 0 -o cache_dir=.lane-scratch/bump-cache --basetemp=.lane-scratch/bump-fixed-temp' uv run --no-sync pytest --noconftest tests/test_bump_version.py -v: 8 passed, resolving that environment-only failure.
• make format and PATH="$PWD/.lane-scratch/ruff-cli/bin:$PATH" make lint: passed with CI's Ruff 0.16.10. Locked Ruff 0.12.11 reports 11 pre-existing UP038 findings in unchanged files.
• uv run --no-sync mypy src/policyengine: 118 pre-existing errors versus 120 with main's narration file; no new errors. CI treats this check as informational.
• uv lock --check, .github/check-changelog.sh, python scripts/release_lock.py, and uv pip check: passed.
• POLICYENGINE_SKIP_COUNTRY_IMPORTS=1 uv run --no-sync policyengine bundle verify --country us --country uk --packages-only --json: all 5 package versions matched.
• python -c "import policyengine; from policyengine.core import Dataset, Policy, Simulation; from policyengine.outputs import aggregate, poverty, inequality; print('import OK')": base-only installs and smoke imports passed on Python 3.11, 3.12, and 3.14; package/narration imports also passed on 3.13.
• env QUARTO_DENO="$PWD/.lane-scratch/quarto-wrapper/deno-wrapper" DENO_DIR="$PWD/.lane-scratch/deno" XDG_CACHE_HOME="$PWD/.lane-scratch/xdg-cache" make docs: 30 pages built. The scratch-only wrapper resolves a macOS Quarto cache failure also reproduced on main.

The initial standard derivation invocation was interrupted during unrelated country startup; the complete test file and full suite subsequently finished. The nine suite skips concern unavailable Belgium source-stack integration and explicit SPM development-authentication setup. Credentialed python scripts/check_release_credentials.py and python scripts/bundle.py check --published-spm --include-tros --strict-tros remain skipped without HUGGING_FACE_TOKEN and must run in CI.

Simulation code, country-model pins, UK region filtering, and public result models were deliberately left unchanged. No other outdated live model call site was found on current main. Replaced-ID searches find no matches in HEAD or the final working tree. No live model-provider call was made, and no provider credentials or metered usage were used.

After independent review

Independent review's global-sampling blocker was reproduced in all 12 mocked HTTP combinations. Narration now rejects effective LiteLLM AnthropicConfig temperature, top_p, and top_k before either completion function for both Sonnet spellings, without mutating shared configuration. Added 24 regression cases including zero values; custom-model overrides remain supported, and existing HTTPX tests verify compatible global token defaults remain unchanged.

Offline, serial Python 3.13.9 validation used locked LiteLLM 1.103.0, removed provider credentials, local cost data, disabled pytest plugin autoload, and blocked sockets. .venv/bin/python -m pytest --noconftest tests/test_derivations.py -k narrat -v -o cache_dir=.lane-scratch/narration-cache --basetemp=.lane-scratch/narration-temp: 54 passed, 6 deselected. .venv/bin/python -m pytest --noconftest tests/test_derivations.py -k 'not narrat and not derive_returns and not top_level_contributions' -v -o cache_dir=.lane-scratch/deterministic-cache --basetemp=.lane-scratch/deterministic-temp: 4 passed, 56 deselected. This verifies 58 tests. The full-file coverage run was interrupted after 1,428.34 seconds during unchanged US-model startup; origin/main's identical US fixture/tests also exceeded a 300-second limit, so no complete 60-test pass is claimed.

make format, make lint, locked Ruff checks on both changed Python files, .github/check-changelog.sh, uv lock --check --python .venv/bin/python, python scripts/release_lock.py, and uv pip check --python .venv/bin/python passed; dependency consistency checked 187 packages. Bundle verification matched all five pins. Clean base-only installs, dependency checks, and CI smoke imports passed on Python 3.11–3.14, with 24 packages per environment. env -u PYTHONPATH .venv/bin/python -m mypy src/policyengine --no-incremental --cache-dir .lane-scratch/mypy-cache reported 118 existing errors; substituting main's narration file reported 120, with zero new diagnostic lines. Complete current-main sources reported 134 errors. Credentialed release verification remains for CI without HUGGING_FACE_TOKEN; the full suite was excluded as instructed. Scratch was removed, fixes remain uncommitted, and no live provider calls or GitHub writes were made.

---

Part of the 2026-09-28 cross-repo model-pin audit, rebuilt on today's main (2026-10-11) after the audit's build lanes were cut off. Current models were re-checked against the provider's docs on 2026-10-03. Opened as a draft per docs/engineering/skills/github-prs.md.

🤖 Generated with Claude Code