Repository navigation
Conversation
Signed-off-by: Arnaud Héritier <aheritier@apache.org>
…bilities Signed-off-by: Arnaud Héritier <aheritier@apache.org>
… table Signed-off-by: Arnaud Héritier <aheritier@apache.org>
Signed-off-by: Arnaud Héritier <aheritier@apache.org>
Signed-off-by: Arnaud Héritier <aheritier@apache.org>
Signed-off-by: Arnaud Héritier <aheritier@apache.org>
…actory applyModelDefaults turns a configured none into a nil budget plus WithNoThinking, so the Anthropic and Bedrock clients never saw it and Claude 5.x kept its default thinking. Restore the explicit none for the Claude 5.x table so it is sent as disabled or adaptive low. Signed-off-by: Arnaud Héritier <aheritier@apache.org>
…orts Signed-off-by: Arnaud Héritier <aheritier@apache.org>
Signed-off-by: Arnaud Héritier <aheritier@apache.org>
…retry A tool-mode reminder retry ends the request on the model's plain-text answer, which Claude 5.x rejects as an assistant prefill. Fail with the coded structured-output error after the first inference instead. Signed-off-by: Arnaud Héritier <aheritier@apache.org>
Signed-off-by: Arnaud Héritier <aheritier@apache.org>
…and 0 budgets Signed-off-by: Arnaud Héritier <aheritier@apache.org>
…e thinking Signed-off-by: Arnaud Héritier <aheritier@apache.org>
Signed-off-by: Arnaud Héritier <aheritier@apache.org>
Importing vertexai for IsModelGardenConfig linked the OpenAI SDK into the lean embedder surface (e2e TestDependencies). Check the publisher directly. Signed-off-by: Arnaud Héritier <aheritier@apache.org>
Signed-off-by: Arnaud Héritier <aheritier@apache.org>
There was a problem hiding this comment.
🟡 Changes recommended
Request validation remains incomplete, and thinking-off and unknown-model policies conflict with the linked requirements.
4 open issues
What changed in this PR
Adds an explicit nine-model Claude 5.x capability table and applies request policies across Anthropic Direct, Vertex, and Bedrock.
Changes:
- Updates thinking, effort, sampling, and prefix-check behavior.
- Adds local prefill rejection and limits structured-output retries.
- Adds request tests and documents transport behavior and limitations.
| File | Description |
|---|---|
| pkg/runtime/structured_output.go | Stops reminder retries for Claude 5.x. |
| pkg/runtime/structured_output_test.go | Tests failure after one inference. |
| pkg/runtime/loop.go | Passes serving model to structured-output handling. |
| pkg/modelinfo/thinking_levels.go | Enables all effort tiers for table models. |
| pkg/modelinfo/modelinfo.go | Uses table for thinking capabilities. |
| pkg/modelinfo/claude5.go | Defines model policies and ID lookup. |
| pkg/modelinfo/claude5_test.go | Tests lookup and capabilities. |
| pkg/model/provider/providerutil/prefill.go | Adds shared assistant-tail detection. |
| pkg/model/provider/providerutil/prefill_test.go | Tests conversation-tail detection. |
| pkg/model/provider/providers/claude5_requests_test.go | Captures requests through provider factories. |
| pkg/model/provider/factory.go | Warns about unsupported thinking-off settings. |
| pkg/model/provider/factory_test.go | Tests warning conditions. |
| pkg/model/provider/bedrock/client.go | Applies Claude 5.x Converse policies. |
| pkg/model/provider/bedrock/client_test.go | Tests Converse request behavior. |
| pkg/model/provider/anthropic/thinking.go | Restores disabled budgets and controls effort. |
| pkg/model/provider/anthropic/thinking_binding_test.go | Updates binding and budget-warning tests. |
| pkg/model/provider/anthropic/system_context_test.go | Uses a legacy model for prefill testing. |
| pkg/model/provider/anthropic/features_integration_test.go | Tests thinking policies and prefill rejection. |
| pkg/model/provider/anthropic/client.go | Rejects detected assistant prefills locally. |
| pkg/model/provider/anthropic/capabilities.go | Consults the explicit capability table. |
| pkg/model/provider/anthropic/capabilities_test.go | Tests table-backed capability predicates. |
| docs/providers/bedrock/index.md | Documents Converse policies and verification limits. |
| docs/providers/anthropic/index.md | Documents Claude 5.x request behavior. |
Review effort: Balanced
| if _, ok := modelinfo.LookupClaude5(model); ok { | ||
| return true | ||
| } | ||
| m, ok := parseClaudeModel(model) | ||
| return ok && (m.atLeast("opus", 5, 0) || m.atLeast("sonnet", 5, 0) || m.family == "fable" || m.family == "mythos") |
There was a problem hiding this comment.
Intentional: the maintainer decided that IDs outside the nine-model table keep today's name heuristics, so nothing that works now regresses. Making the table the only source of 5.x behavior would drop sampling omission and adaptive defaults for IDs like claude-haiku-5 or claude-mythos-5-preview, causing 400s. #4651 has been updated to record this decision.
| if budget.IsDisabled() { | ||
| if isClaude5 && !c5.CanDisableThinking { | ||
| return "low", true |
There was a problem hiding this comment.
Intentional, per maintainer decision: a configured none on a model that can't disable thinking is sent as adaptive/low, with a startup warning. A local error would break title generation and MCP sampling, which request no-thinking internally, on Opus/Sonnet 5.5, Fable and Mythos. #4651 has been updated to record this decision.
| req := captureFinalRequest(t, latest.ModelConfig{ | ||
| Provider: "anthropic", Model: "claude-haiku-5-5", | ||
| ThinkingBudget: &latest.ThinkingBudget{Effort: "high"}, Temperature: new(0.2), | ||
| }, anthropicEnv) | ||
| assert.Equal(t, "adaptive", field(req.body, "thinking", "type")) | ||
| assert.Equal(t, "high", field(req.body, "output_config", "effort")) | ||
| assert.NotContains(t, req.body, "temperature") |
There was a problem hiding this comment.
Fixed in e89cd77. The standard-path cases now use Sonnet 5 (no automatic binding) with interleaved_thinking: false and assert the standard anthropic-beta header, including a token-budget case that checks effort is omitted. The Haiku 5.5 case is renamed to say it goes through Beta.
| // FinalAssistantPrefill reports whether the conversation ends with an | ||
| // assistant message that is not a tool call, i.e. a response prefill. | ||
| // Trailing system messages are ignored. | ||
| func FinalAssistantPrefill(messages []chat.Message) bool { | ||
| for _, msg := range slices.Backward(messages) { | ||
| switch msg.Role { | ||
| case chat.MessageRoleSystem: | ||
| continue | ||
| case chat.MessageRoleAssistant: | ||
| return len(msg.ToolCalls) == 0 |
There was a problem hiding this comment.
A history ending on an assistant tool_use with no tool_result is invalid for every Claude model, and the API rejects it with an explicit missing-tool-result error. It isn't a prefill. The runtime always appends tool results before the next call, so a 5.x-only 'prefill' check here would mislabel the error without preventing anything.
aheritier
added
area/docs
… the factory level Signed-off-by: Arnaud Héritier <aheritier@apache.org>
Pull request was closed
Closes #4651. Part of #4583 (Claude 5.x core) and epic #4580.
What
Replace the name-based guesses for Claude 5.x with an explicit table of the nine models (
modelinfo.LookupClaude5). Anthropic Direct, Vertex and Bedrock Converse now send each of them a request it accepts. IDs outside the table keep exactly today's behavior.All nine use adaptive thinking by default, reject token budgets and sampling params, and accept every effort level up to
max.Bugs fixed (each with a test that failed before)
thinking_budget: highwas sent asbudget_tokens, which every 5.x model rejects.budget_tokenson Sonnet 5/5.5, Haiku 5.5, Fable and Mythos.temperature/top_p/top_kwere sent to 5.x models.thinking_budget: nonenever reached the Claude clients through the factory.applyModelDefaultsturns it into a nil budget plusWithNoThinking, so on 5.x thinking stayed on at the server default, on Direct, Vertex and Bedrock. It is now sent asdisabledwhere the model allows it and as adaptive/lowotherwise.thinking: disabled, which they reject.xhigh/maxon Haiku 5.5, Sonnet 5 and Sonnet 5.5.Behavior changes to review
noneon a model that can't disable thinking (Opus/Sonnet 5.5, Fable, Mythos) is sent as adaptive withloweffort. When the value came from config, a startup warning says so. Internal calls (titles, MCP sampling) get the same mapping silently.max_tokensis not raised, and the warning now says the token ceiling is not enforced.ErrorCodeStructuredOutputFailedafter the first plain-text answer, instead of retrying with reminders. Each retry would end on the model's answer, which 5.x rejects. feat(Claude): Claude 5.x core — request policy and signed history #4583 asks for "one inference, then a graceful model error", with no role rewrite.drop_blockbinding. Haiku, Opus and Sonnet 5.5 now get it, the same way Fable 5.1 already did. The default policy itself (drop_blockvserror) is feat(anthropic): Claude 5.x safety defaults (prefix error, temporary fallback block, Mythos 5 on Converse) #4652.Tests
@date, Bedrockus.anthropic.…-v1:0, gateway prefixes and near-miss IDs.openai/claude-…IDs deliberately don't match.providers.NewDefaultRegistry()) for Direct standard, Direct Beta, Vertex Model Garden (path,anthropic_version, bearer auth, no API key, de-duplicated betas), Bedrock Converse and a title clone, plus legacy controls.Validation
task buildpasses.golangci-lintv2.14.0 reports 0 issues, andgo run ./lint .reports no offenses.go test -race -shuffle=on ./pkg/modelinfo/ ./pkg/model/provider/... ./pkg/runtime/passes.go test ./e2e -run '^TestWasmProviderDependencies$|^TestDependencies$'passes.task test-wasm-providers: the WASM JS packages pass. Running it needed a minimal environment, because Node rejected my shell's environment size.task testfails onlypkg/teamloaderTestLoadExamples(10 Gemini examples,GOOGLE_API_KEY or GEMINI_API_KEY environment variable is requiredunder.env.test). The base commit7eedf8dfails identically on this machine.Not verified against live services: Bedrock
thinking: {"type":"disabled"}and the thinking-binding beta on Vertex. The docs say so.