Skip to content

fix(middleware): preserve serving-hop usage iterations - #2012

Draft
Afloat16 wants to merge 2 commits into
anthropics:mainfrom
Afloat16:fix-serving-hop-usage-iterations-2026-10-09
Draft

Afloat16 wants to merge 2 commits into
anthropics:mainfrom
Afloat16:fix-serving-hop-usage-iterations-2026-10-09

Conversation

@Afloat16

@Afloat16 Afloat16 commented Oct 9, 2026 •

Copy link
Copy Markdown

When refusal fallback serves a stream with a reported usage.iterations breakdown, _HopReader replaces that breakdown with one aggregate fallback_message entry. A server-tool loop can consequently lose its sampling/compaction records, per-iteration token values, cache TTL details and reported models.

Retain the reported records and their order, and relabel only the final message/fallback_message sampling entry. When no sampling entry is reported, retain the other records and append the existing aggregate completion entry. The shared reader covers sync and async beta streaming. Aggregate usage, stream framing and fallback request construction retain their current behavior. The change adds no public name, parameter or option.

Initial validation at d151df2:

  • Public strict sync/async synthetic HTTP/SSE regressions: original production has 10 failures and 136 passing controls in the affected scope; the new-constant metadata tripwire alone is deselected on original production. All 147 affected cases pass in each final version/Pydantic scope.
  • Full ./scripts/test matrix: Python 3.10.22/Pydantic 2.12.5 has 9742 passed, 29 skipped and 1 expected failure; Python 3.10.22/Pydantic 1.10.26 has 9215 passed and 519 skipped; Python 3.14.8/Pydantic 2.12.5 has 9745 passed, 26 skipped and 1 expected failure. The separate newest-Python MCP>=2 run has 39 passed.
  • ./scripts/format, whole ./scripts/lint, package build, public-symbol comparison and ./scripts/detect-breaking-changes against main all pass. The actual lint script runs Ruff, dependency-cap checks, strict Pyright and import smoke; it does not invoke mypy.

Tests use synthetic transport responses and the bundled loopback mock fixture. Existing opt-in live-provider tests remain disabled, and external provider/service-billing behavior was not validated. Normal remote CI remains in force.

Coordinated sibling ports:

These ports correct the same serving-hop projection in the SDKs' existing language-specific middleware; each includes its own runtime changelog note. The implementation and regressions were developed with substantial AI assistance.

Validation after merging current main b4b7916 (SDK 1.13.0), published at 754b710:

  • The ordinary two-parent merge preserves the middleware and regression bytes and keeps both the Unreleased fix and the new upstream release entry.
  • Fresh full native runs on tree 2ea88b13a1b84aaeb14963f233cf1d2898cb6ff7 repeat all three Python/Pydantic matrices and the 39-case MCP check above with the same pass/skip/expected-failure counts. Native lint, package build, public-symbol comparison and the complete native compatibility checker against b4b7916 pass.
  • In the fresh two-file streaming scope, current upstream has 10 expected regression failures and 94 passing controls; the candidate has 105 passes with Pydantic 2, or 104 passes and one existing skip with Pydantic 1. Initial proxy-dependent diagnostics were replaced by isolated synthetic/loopback runs. Unrelated upstream formatter changes were kept outside this PR.
  • New-head upstream Actions require maintainer approval. The fork run has five queued jobs targeting ubuntu-latest-4-core with no assigned runner or executed test steps; remote CI has not passed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant