Skip to content

fix(agent_tool): wrap input_schema payload in ReAct prompt and propagate tool_choice to LiteLLM - #5924

Closed
ecanlar wants to merge 13 commits into
google:mainfrom
ecanlar:fix/agent-tool-input-schema-tool-choice-litellm
Closed

ecanlar wants to merge 13 commits into
google:mainfrom
ecanlar:fix/agent-tool-input-schema-tool-choice-litellm

Conversation

@ecanlar

@ecanlar ecanlar commented Jun 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes two issues that prevent Claude-family models from entering the ReAct tool-calling loop when used via LiteLLM inside a nested AgentTool:

  1. AgentTool with input_schema — the serialized JSON payload sent as the first message causes Claude to interpret the request as already complete and respond directly without calling any tools.
  2. tool_choice not propagated — llm_request.config.tool_config.function_calling_config.mode was not forwarded to LiteLLM's completion_args, so callers could not enforce tool use at the request level.

Changes

src/google/adk/tools/agent_tool.py

Wrap the serialized input_schema JSON in a natural-language instruction that explicitly asks the inner agent to use its available tools before producing a response. This keeps Claude in ReAct mode regardless of the message content format.

src/google/adk/models/lite_llm.py

Read llm_request.config.tool_config.function_calling_config.mode and map it to LiteLLM's tool_choice parameter:

  • ANY → "required"
  • NONE → "none"
  • AUTO → provider default (unchanged, key omitted from completion_args)

_get_completion_inputs now returns a 5-tuple (messages, tools, response_format, generation_params, tool_choice).

Unit Tests Added

tests/unittests/tools/test_agent_tool.py

  • test_run_async_no_input_schema_passes_request_unchanged: without input_schema, the content passed to the inner runner is args['request'] verbatim.
  • test_run_async_with_input_schema_wraps_in_natural_language: with input_schema, the text begins with "Process the following structured request", contains "Request:\n" followed by the JSON payload, and is not a bare JSON blob.
  • test_run_async_with_input_schema_text_not_raw_json: asserts the text does not start with {.

tests/unittests/models/test_litellm.py

  • test_get_completion_inputs_tool_choice_none_without_tool_config: tool_choice is None with no tool_config.
  • test_get_completion_inputs_tool_choice_required_for_any_mode: returns "required" for ANY mode.
  • test_get_completion_inputs_tool_choice_none_for_none_mode: returns "none" for NONE mode.
  • test_get_completion_inputs_tool_choice_none_for_auto_mode: returns None for AUTO mode.
  • test_generate_content_async_propagates_tool_choice_required: acompletion receives tool_choice="required" for ANY.
  • test_generate_content_async_propagates_tool_choice_none_mode: acompletion receives tool_choice="none" for NONE.
  • test_generate_content_async_omits_tool_choice_for_auto_mode: tool_choice key absent from completion_args for AUTO.
  • test_generate_content_async_omits_tool_choice_without_tool_config: tool_choice key absent when no tool_config.

Also updated all existing _get_completion_inputs call sites (10 occurrences) to unpack the new 5-tuple.

Pytest Results

tests/unittests/tools/test_agent_tool.py + tests/unittests/models/test_litellm.py
1 failed (pre-existing: test_custom_schema[GOOGLE_AI] — unrelated to this PR),
302 passed, 1 skipped in 2.98s

New tests: 11 passed (3 agent_tool + 8 litellm)

The 1 pre-existing failure (test_custom_schema[GOOGLE_AI]) is a pydantic.ValidationError that reproduces on the base branch before any of these changes and is unrelated to this fix.

Related

…ol_choice to LiteLLM

When AgentTool uses input_schema, the inner agent receives a raw JSON blob
that causes Claude models to skip the tool-calling loop (ReAct). Fix by
wrapping the payload in a natural-language instruction.

Also propagate tool_config.function_calling_config.mode to LiteLLM's
tool_choice parameter so callers can enforce tool_choice='required'.
Addresses google#773.

Fixes: google#5926
@ecanlar
ecanlar force-pushed the fix/agent-tool-input-schema-tool-choice-litellm branch from ded18ca to 6168c49 Compare June 1, 2026 09:51
@adk-bot adk-bot added models [Component] This issue is related to model support tools [Component] This issue is related to tools labels Jun 1, 2026
@adk-bot

adk-bot commented Jun 1, 2026

Copy link
Copy Markdown
Collaborator

Response from ADK Triaging Agent

Hello @ecanlar, thank you for creating this PR!

We appreciate your contribution to fixing the LiteLLM/Claude AgentTool issue. To help our reviewers process this pull request more efficiently, please ensure it complies with our contribution guidelines.

Currently, the following items are missing:

  1. Unit Tests: The PR modifies behavior in lite_llm.py and agent_tool.py, but does not add or update any unit tests (e.g., under tests/unittests/). Please add unit tests covering the new tool wrapping and tool_choice behavior.
  2. Pytest Results Summary: Please run the tests and include a summary of the passed pytest or tox results in the PR description.
  3. Logs/Screenshots: Please provide console logs or screenshots demonstrating that the inner agent successfully enters the ReAct loop and calls tools after applying the fix.

Providing this information helps ensure high code quality, prevents future regressions, and makes the review process smoother. Thank you!

…tool_choice propagation

- Update existing _get_completion_inputs call sites to handle new 5-tuple
  return value (adds tool_choice as 5th element)
- Add 3 tests for AgentTool.run_async: verifies message is passed verbatim
  without input_schema, and is wrapped in a natural-language instruction
  with input_schema (PR google#5924 fix)
- Add 8 tests for LiteLLM tool_choice propagation: covers _get_completion_inputs
  returning correct tool_choice for ANY/NONE/AUTO modes and None when no
  tool_config; covers generate_content_async correctly including/omitting
  tool_choice in completion_args
rohityan and others added 2 commits June 1, 2026 19:11
- Fix: only apply the ReAct wrapper in agent_tool.py when output_schema is not set on the inner agent, preventing breaking of single-shot structured output mode
- Add regression test test_run_async_with_input_and_output_schema_passes_raw_json documenting that raw JSON is passed when both input_schema and output_schema are set
- Apply pre-commit formatting fixes (isort + pyink)
@ecanlar
ecanlar force-pushed the fix/agent-tool-input-schema-tool-choice-litellm branch from 23d582a to 67d087a Compare June 2, 2026 09:44
ecanlar and others added 2 commits June 3, 2026 08:26
…st HTTP call

Move the ValueError from module import time to the lazy _get_session()
factory so that importing the package (e.g. pytest collection) does not
fail in environments where GITHUB_TOKEN is not set.
@ecanlar
ecanlar force-pushed the fix/agent-tool-input-schema-tool-choice-litellm branch from f8c61e3 to 7c23d83 Compare June 5, 2026 11:05
@ecanlar

ecanlar commented Jun 5, 2026

Copy link
Copy Markdown
Contributor Author

Good catch — confirmed the guard was not in place. tool_choice and tools were derived from two independent config fields (config.tool_config and config.tools) with no cross-check, so a caller setting mode=ANY with no function_declarations would reach acompletion() with tools=None, tool_choice="required" — exactly the combination LiteLLM rejects.

Fixed in the latest commit:

  • _get_completion_inputs: coerce tool_choice to None when tools is falsy before returning.
  • generate_content_async: null tool_choice alongside tools inside the "functions" additional-arg branch.

Two new tests cover both paths; 10 tool_choice tests passing.

@rohityan

rohityan commented Jun 5, 2026 •

Copy link
Copy Markdown
Collaborator

Hi @ecanlar , Thank you for your contribution! We appreciate you taking the time to submit this pull request.

@rohityan

rohityan commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator

/adk-pr-analyze

@rohityan rohityan added request clarification [Status] The maintainer need clarification or more information from the author and removed request clarification [Status] The maintainer need clarification or more information from the author labels Jun 5, 2026
@rohityan

rohityan commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Hi @ecanlar , Your PR has been received by the team and is currently under review. We will provide feedback as soon as we have an update to share.

@rohityan rohityan added the needs review [Status] The PR/issue is awaiting review from the maintainer label Jun 9, 2026
@rohityan
rohityan requested a review from GWeale June 9, 2026 18:23
@rohityan

rohityan commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Hi @GWeale , can you please review this.

@rohityan rohityan added request clarification [Status] The maintainer need clarification or more information from the author and removed needs review [Status] The PR/issue is awaiting review from the maintainer labels Jun 11, 2026
@rohityan
rohityan removed the request for review from GWeale June 11, 2026 23:47
@rohityan

rohityan commented Jun 11, 2026 •

Copy link
Copy Markdown
Collaborator

Hi @ecanlar , can you please fix formatting errors.

@rohityan
rohityan requested a review from GWeale June 11, 2026 23:57
@rohityan rohityan added needs review [Status] The PR/issue is awaiting review from the maintainer and removed request clarification [Status] The maintainer need clarification or more information from the author labels Jun 11, 2026
ecanlar and others added 2 commits June 12, 2026 07:40
LiteLLM rejects tool_choice="required"/"none" when the tools list is
falsy. tool_choice and tools were derived from independent config fields
with no cross-check, so a caller setting mode=ANY with no
function_declarations would reach acompletion() with tools=None and
tool_choice="required".

Add a guard in _get_completion_inputs to coerce tool_choice to None
when tools is falsy, and mirror the same coercion in
generate_content_async after the "functions" additional-arg override
that already nulls tools.

Add two tests covering both paths; update four existing tool_choice
tests to include a FunctionDeclaration so they remain valid under the
new guard.
@ecanlar
ecanlar force-pushed the fix/agent-tool-input-schema-tool-choice-litellm branch from b680ca5 to 9cd8f31 Compare June 12, 2026 07:26
@ecanlar

ecanlar commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

i fixed the formatting erros. @rohityan

@i-yliu i-yliu assigned i-yliu and unassigned rohityan Jun 18, 2026
copybara-service Bot pushed a commit that referenced this pull request Jul 28, 2026
…ice to LiteLLM

Merge #5924
END_PUBLIC
## Summary

Fixes two issues that prevent Claude-family models from entering the ReAct tool-calling loop when used via LiteLLM inside a nested `AgentTool`:

1. **`AgentTool` with `input_schema`** — the serialized JSON payload sent as the first message causes Claude to interpret the request as already complete and respond directly without calling any tools.
2. **`tool_choice` not propagated** — `llm_request.config.tool_config.function_calling_config.mode` was not forwarded to LiteLLM's `completion_args`, so callers could not enforce tool use at the request level.

## Changes

### `src/google/adk/tools/agent_tool.py`
Wrap the serialized `input_schema` JSON in a natural-language instruction that explicitly asks the inner agent to use its available tools before producing a response. This keeps Claude in ReAct mode regardless of the message content format.

### `src/google/adk/models/lite_llm.py`
Read `llm_request.config.tool_config.function_calling_config.mode` and map it to LiteLLM's `tool_choice` parameter:
- `ANY` → `"required"`
- `NONE` → `"none"`
- `AUTO` → provider default (unchanged, key omitted from `completion_args`)

`_get_completion_inputs` now returns a 5-tuple `(messages, tools, response_format, generation_params, tool_choice)`.

## Unit Tests Added

### `tests/unittests/tools/test_agent_tool.py`
- `test_run_async_no_input_schema_passes_request_unchanged`: without `input_schema`, the content passed to the inner runner is `args['request']` verbatim.
- `test_run_async_with_input_schema_wraps_in_natural_language`: with `input_schema`, the text begins with `"Process the following structured request"`, contains `"Request:\n"` followed by the JSON payload, and is not a bare JSON blob.
- `test_run_async_with_input_schema_text_not_raw_json`: asserts the text does not start with `{`.

### `tests/unittests/models/test_litellm.py`
- `test_get_completion_inputs_tool_choice_none_without_tool_config`: `tool_choice` is `None` with no `tool_config`.
- `test_get_completion_inputs_tool_choice_required_for_any_mode`: returns `"required"` for `ANY` mode.
- `test_get_completion_inputs_tool_choice_none_for_none_mode`: returns `"none"` for `NONE` mode.
- `test_get_completion_inputs_tool_choice_none_for_auto_mode`: returns `None` for `AUTO` mode.
- `test_generate_content_async_propagates_tool_choice_required`: `acompletion` receives `tool_choice="required"` for `ANY`.
- `test_generate_content_async_propagates_tool_choice_none_mode`: `acompletion` receives `tool_choice="none"` for `NONE`.
- `test_generate_content_async_omits_tool_choice_for_auto_mode`: `tool_choice` key absent from `completion_args` for `AUTO`.
- `test_generate_content_async_omits_tool_choice_without_tool_config`: `tool_choice` key absent when no `tool_config`.

Also updated all existing `_get_completion_inputs` call sites (10 occurrences) to unpack the new 5-tuple.

## Pytest Results

```
tests/unittests/tools/test_agent_tool.py + tests/unittests/models/test_litellm.py
1 failed (pre-existing: test_custom_schema[GOOGLE_AI] — unrelated to this PR),
302 passed, 1 skipped in 2.98s

New tests: 11 passed (3 agent_tool + 8 litellm)
```

The 1 pre-existing failure (`test_custom_schema[GOOGLE_AI]`) is a `pydantic.ValidationError` that reproduces on the base branch before any of these changes and is unrelated to this fix.

## Related

- Fixes #5926
- Addresses #773 (expose tool_choice to callers via FunctionCallingConfig)
- Related #1063 (fixed FunctionDeclaration description in v1.20, different issue)

Co-authored-by: George Weale <gweale@google.com>
COPYBARA_INTEGRATE_REVIEW=#5924 from ecanlar:fix/agent-tool-input-schema-tool-choice-litellm 9cd8f31
PiperOrigin-RevId: 955433983
@adk-bot

adk-bot commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Thank you @ecanlar for your contribution! 🎉

Your changes have been successfully imported and merged via Copybara in commit 550189c.

Closing this PR as the changes are now in the main branch.

@adk-bot adk-bot added the merged [Status] This PR is merged label Jul 28, 2026
@adk-bot adk-bot closed this Jul 28, 2026
FrigaZzz pushed a commit to FrigaZzz/adk-python that referenced this pull request Aug 11, 2026
…ice to LiteLLM

Merge google#5924
END_PUBLIC
## Summary

Fixes two issues that prevent Claude-family models from entering the ReAct tool-calling loop when used via LiteLLM inside a nested `AgentTool`:

1. **`AgentTool` with `input_schema`** — the serialized JSON payload sent as the first message causes Claude to interpret the request as already complete and respond directly without calling any tools.
2. **`tool_choice` not propagated** — `llm_request.config.tool_config.function_calling_config.mode` was not forwarded to LiteLLM's `completion_args`, so callers could not enforce tool use at the request level.

## Changes

### `src/google/adk/tools/agent_tool.py`
Wrap the serialized `input_schema` JSON in a natural-language instruction that explicitly asks the inner agent to use its available tools before producing a response. This keeps Claude in ReAct mode regardless of the message content format.

### `src/google/adk/models/lite_llm.py`
Read `llm_request.config.tool_config.function_calling_config.mode` and map it to LiteLLM's `tool_choice` parameter:
- `ANY` → `"required"`
- `NONE` → `"none"`
- `AUTO` → provider default (unchanged, key omitted from `completion_args`)

`_get_completion_inputs` now returns a 5-tuple `(messages, tools, response_format, generation_params, tool_choice)`.

## Unit Tests Added

### `tests/unittests/tools/test_agent_tool.py`
- `test_run_async_no_input_schema_passes_request_unchanged`: without `input_schema`, the content passed to the inner runner is `args['request']` verbatim.
- `test_run_async_with_input_schema_wraps_in_natural_language`: with `input_schema`, the text begins with `"Process the following structured request"`, contains `"Request:\n"` followed by the JSON payload, and is not a bare JSON blob.
- `test_run_async_with_input_schema_text_not_raw_json`: asserts the text does not start with `{`.

### `tests/unittests/models/test_litellm.py`
- `test_get_completion_inputs_tool_choice_none_without_tool_config`: `tool_choice` is `None` with no `tool_config`.
- `test_get_completion_inputs_tool_choice_required_for_any_mode`: returns `"required"` for `ANY` mode.
- `test_get_completion_inputs_tool_choice_none_for_none_mode`: returns `"none"` for `NONE` mode.
- `test_get_completion_inputs_tool_choice_none_for_auto_mode`: returns `None` for `AUTO` mode.
- `test_generate_content_async_propagates_tool_choice_required`: `acompletion` receives `tool_choice="required"` for `ANY`.
- `test_generate_content_async_propagates_tool_choice_none_mode`: `acompletion` receives `tool_choice="none"` for `NONE`.
- `test_generate_content_async_omits_tool_choice_for_auto_mode`: `tool_choice` key absent from `completion_args` for `AUTO`.
- `test_generate_content_async_omits_tool_choice_without_tool_config`: `tool_choice` key absent when no `tool_config`.

Also updated all existing `_get_completion_inputs` call sites (10 occurrences) to unpack the new 5-tuple.

## Pytest Results

```
tests/unittests/tools/test_agent_tool.py + tests/unittests/models/test_litellm.py
1 failed (pre-existing: test_custom_schema[GOOGLE_AI] — unrelated to this PR),
302 passed, 1 skipped in 2.98s

New tests: 11 passed (3 agent_tool + 8 litellm)
```

The 1 pre-existing failure (`test_custom_schema[GOOGLE_AI]`) is a `pydantic.ValidationError` that reproduces on the base branch before any of these changes and is unrelated to this fix.

## Related

- Fixes google#5926
- Addresses google#773 (expose tool_choice to callers via FunctionCallingConfig)
- Related google#1063 (fixed FunctionDeclaration description in v1.20, different issue)

Co-authored-by: George Weale <gweale@google.com>
COPYBARA_INTEGRATE_REVIEW=google#5924 from ecanlar:fix/agent-tool-input-schema-tool-choice-litellm 9cd8f31
PiperOrigin-RevId: 955433983
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merged [Status] This PR is merged models [Component] This issue is related to model support needs review [Status] The PR/issue is awaiting review from the maintainer tools [Component] This issue is related to tools

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: AgentTool with input_schema + LiteLLM/Claude skips inner agent tool calls

4 participants