Skip to content

feat(claude): refuse Claude 5.x fallbacks and Mythos 5 on Bedrock - #4656

Closed
aheritier wants to merge 3 commits into
docker:feat/claude5x-model-tablefrom
aheritier:feat/claude5x-safety-defaults
Closed

aheritier wants to merge 3 commits into
docker:feat/claude5x-model-tablefrom
aheritier:feat/claude5x-safety-defaults

Conversation

@aheritier

@aheritier aheritier commented Oct 19, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #4652. Part of #4583 and epic #4580.

Stacked on #4657 (base branch feat/claude5x-model-table). This PR's diff shows only its own three commits; GitHub retargets it to main when #4657 merges.

What

Refuse two Claude 5.x configurations that docker-agent cannot serve correctly yet, at client construction and before any inference call:

  1. Server-side fallbacks involving Claude 5.x are rejected when the primary or any fallback is one of the nine 5.x models (gateway and Vertex spellings included), until feat(Claude): end-to-end server fallback for Claude 5.x #4584 handles a mid-response model switch. The error points to feat(Claude): end-to-end server fallback for Claude 5.x #4584. Absent or empty lists and chains without a 5.x model are unchanged. Malformed lists stay ignored, as before.
  2. claude-mythos-5 on Bedrock is refused. AWS serves it only through Bedrock Mantle, not the Converse API used here. claude-mythos-5-1 still works. The model table gains a BedrockMantleOnly field.

Both checks run where every construction path already validates options: validateThinkingOptions for Anthropic Direct, Vertex and Model Garden, and bedrock.NewClient for Bedrock. Clones, routing and gateway branches all pass through them.

⚠️ Breaking change

Fable 5/5.1 with fallbacks (the headline refusal-fallback use case) now fails at startup. This follows the rollout policy approved in #4580: the request is refused rather than half-supported until #4584. examples/anthropic-fallbacks.yaml and the docs example now use claude-opus-4-8 → claude-opus-4-7, and the docs carry a warning callout.

Not in this PR

Making thinking_prefix_mismatch: error the 5.x default moved to #4653. The runtime edits the conversation prefix (num_history_items trimming, hooks, compaction), so error would break those sessions until signed history can be replayed exactly. Both issues are updated.

Validation

  • task build passes.
  • golangci-lint v2.14.0 reports 0 issues, and go run ./lint . reports no offenses.
  • go test ./e2e -run '^TestWasmProviderDependencies$|^TestDependencies$' passes.
  • task test fails only the 10 pre-existing pkg/teamloader TestLoadExamples Gemini cases (GOOGLE_API_KEY or GEMINI_API_KEY environment variable is required under .env.test). They fail identically on main.
  • An independent review found no critical or important issues; its three minor findings are fixed in 782564e.

docker#4584

Signed-off-by: Arnaud Héritier <aheritier@apache.org>
Signed-off-by: Arnaud Héritier <aheritier@apache.org>
Copilot AI balanced review requested due to automatic review settings October 19, 2026 22:08
@aheritier
aheritier requested a review from a team as a code owner October 19, 2026 22:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The scoped safeguards use existing construction validation paths, preserve permitted configurations, and have focused regression coverage with no unresolved blocking findings.

No open issues 🎉

What changed in this PR

Adds startup safeguards for unsupported Claude 5.x configurations, building on the request-policy changes in #4655.

Changes:

  • Rejects non-empty fallback chains involving known Claude 5.x models until #4584.
  • Refuses Mythos 5 on Bedrock Converse while allowing Mythos 5.1.
  • Adds regression tests and updates documentation, schema descriptions, and fallback examples.
File Description
pkg/​runtime/​structured_output.go Stops unsupported Claude 5.x reminder retries.
pkg/​runtime/​structured_output_test.go Tests failure after one plain-text response.
pkg/​runtime/​loop.go Passes model identity to structured-output handling.
pkg/​modelinfo/​thinking_levels.go Enables Claude 5.x effort levels.
pkg/​modelinfo/​modelinfo.go Uses explicit Claude 5.x capabilities.
pkg/​modelinfo/​claude5.go Defines model policies and Mantle-only status.
pkg/​modelinfo/​claude5_test.go Tests model recognition and policies.
pkg/​model/​provider/​providerutil/​prefill.go Detects final assistant prefills.
pkg/​model/​provider/​providerutil/​prefill_test.go Tests prefill detection.
pkg/​model/​provider/​providers/​claude5_requests_test.go Checks requests through provider factories.
pkg/​model/​provider/​factory.go Warns when thinking cannot be disabled.
pkg/​model/​provider/​factory_test.go Tests warning conditions.
pkg/​model/​provider/​bedrock/​client.go Enforces Claude policies and refuses Mythos 5.
pkg/​model/​provider/​bedrock/​client_test.go Tests request policies and Mythos restrictions.
pkg/​model/​provider/​anthropic/​vertex/​vertex_test.go Tests fallback rejection on Vertex.
pkg/​model/​provider/​anthropic/​thinking.go Preserves thinking intent and default effort.
pkg/​model/​provider/​anthropic/​thinking_binding.go Adds fallback validation during construction.
pkg/​model/​provider/​anthropic/​thinking_binding_test.go Updates binding expectations and warning coverage.
pkg/​model/​provider/​anthropic/​system_context_test.go Uses a legacy model for prefill coverage.
pkg/​model/​provider/​anthropic/​features_integration_test.go Tests thinking, prefills, and fallback restrictions.
pkg/​model/​provider/​anthropic/​fallbacks.go Rejects fallback chains involving Claude 5.x.
pkg/​model/​provider/​anthropic/​fallbacks_test.go Covers rejected and permitted fallback configurations.
pkg/​model/​provider/​anthropic/​client.go Rejects Claude 5.x assistant prefills locally.
pkg/​model/​provider/​anthropic/​capabilities.go Applies explicit model capabilities.
pkg/​model/​provider/​anthropic/​capabilities_test.go Tests capability predicates.
examples/​anthropic-fallbacks.yaml Switches to a permitted legacy fallback chain.
docs/​providers/​bedrock/​index.md Documents policies and Mantle restrictions.
docs/​providers/​anthropic/​index.md Documents Claude 5.x behavior and fallback refusal.
agent-schema.json Updates provider-option descriptions.

Review effort: Balanced

- Drop the Fable example from the server-side fallback comment; that
  configuration is now rejected.
- Build the Bedrock Mantle-only error from the configured model ID.
- Move unit-test fixtures off Claude 5.x fallbacks that the constructor
  now rejects.

Signed-off-by: Arnaud Héritier <aheritier@apache.org>
@aheritier

Copy link
Copy Markdown
Collaborator Author

Superseded by #4658: same commits, moved to a docker/docker-agent branch because GitHub stacks cannot include fork PRs.

@aheritier aheritier closed this Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(anthropic): Claude 5.x safety defaults (prefix error, temporary fallback block, Mythos 5 on Converse)

2 participants