[BUG]: LiteLlm.capabilities hardcodes output_schema_and_tools=True, bypassing the SetModelResponseTool workaround #6954

Description

LiteLlm.capabilities returns output_schema_and_tools=True unconditionally, regardless of which model it wraps. ADK trusts that and applies response_schema alongside the tool declarations, so SetModelResponseTool never engages. On models that cannot honour both constraints the agent loops, calling tools repeatedly instead of producing structured output.

This is the same failure as #3413 and #5054, but on a path none of the existing work covers:

• fix: output_schema + tools infinite loop on Gemini 2.x #5057 fixed the equivalent gate on the native Gemini path. LiteLlm bypasses that gate entirely, so merging it would not have helped anyone using ADK through LiteLLM. It was closed as stale because the code moved into models/_capabilities, which is the path this report is against.
• fix: harden SetModelResponseTool fallback to prevent infinite loops #5091 hardens SetModelResponseTool (stronger instructions, forced tool_config at round N-1, a 25-round cutoff). All of that applies only once the fallback is active. Here it never activates, because LiteLlm reports the capability as supported.

Filing per the request on #5057: "the repro @wuliang229 asked for is what a version gate needs first... open a new issue if you see this happen again."

Environment

• google-adk 2.8.0
• litellm 1.98.0
• Python 3.13.14
• Model: openrouter/google/gemini-3.1-flash-lite (Gemini 3.1 Flash Lite via OpenRouter)

Reproduction

Needs an OpenRouter key in OPENROUTER_API_KEY. Any provider LiteLLM can reach should do; the model family matters more than the provider (see the note on model dependence below).

import asyncio
from pydantic import BaseModel, Field
from google.adk import Agent
from google.adk.models.lite_llm import LiteLlm
from google.adk.runners import InMemoryRunner
from google.genai import types

MODEL = "openrouter/google/gemini-3.1-flash-lite"

class CityTime(BaseModel):
    city: str = Field(description="City name")
    time: str = Field(description="Local time")

def get_current_time(city: str) -> dict:
    """Returns the current time in a specified city."""
    return {"city": city, "time": "10:30 AM"}

agent = Agent(
    model=LiteLlm(MODEL),
    name="repro",
    instruction="Use get_current_time to look up the time, then answer.",
    tools=[get_current_time],
    output_schema=CityTime,
)

async def main():
    runner = InMemoryRunner(agent=agent, app_name="repro")
    await runner.session_service.create_session(
        app_name="repro", user_id="u", session_id="s")
    tool_calls = 0
    async for ev in runner.run_async(
        user_id="u", session_id="s",
        new_message=types.Content(role="user",
                                  parts=[types.Part(text="What time is it in Sydney?")]),
    ):
        if ev.content and ev.content.parts:
            for p in ev.content.parts:
                if p.function_call:
                    tool_calls += 1
                    print(f"  tool call {tool_calls}: {p.function_call.name}")
    print(f"total tool calls: {tool_calls}")

asyncio.run(main())

Observed: get_current_time is called repeatedly with identical arguments before the agent settles. Four runs of this exact code gave 4, 12, 18 and 19 tool calls. It does eventually produce valid output, but the count is neither stable nor bounded, and on some runs it did not terminate within several minutes.

Adding output_schema_and_tools=False (via the subclass below) makes it 2 function calls, every run: get_current_time plus set_model_response.

Supplementary: confirming the cause without API calls

This inspects the outgoing request and short-circuits before anything is sent, so it needs no key and no credit. Useful for verifying a fix quickly.

import asyncio
from pydantic import BaseModel
from google.adk import Agent
from google.adk.models.lite_llm import LiteLlm
from google.adk.models.base_llm import LlmCapabilities
from google.adk.models.llm_response import LlmResponse
from google.adk.runners import InMemoryRunner
from google.genai import types

MODEL = "openrouter/google/gemini-3.1-flash-lite"

class CityTime(BaseModel):
    city: str
    time: str

def get_current_time(city: str) -> dict:
    """Returns the current time in a specified city."""
    return {"city": city, "time": "10:30 AM"}

class SchemaSafeLiteLlm(LiteLlm):
    @property
    def capabilities(self) -> LlmCapabilities:
        return LlmCapabilities(output_schema_and_tools=False)

def spy(callback_context, llm_request):
    cfg = llm_request.config
    schema_set = bool(getattr(cfg, "response_schema", None)
                      or getattr(cfg, "response_json_schema", None))
    names = [fd.name
             for t in (getattr(cfg, "tools", None) or [])
             for fd in (getattr(t, "function_declarations", None) or [])]
    print(f"  response_schema_set={schema_set}  tools_offered={names}")
    return LlmResponse(content=types.Content(role="model", parts=[types.Part(text="{}")]))

async def inspect(label, model):
    print(f"\n{label}  output_schema_and_tools={model.capabilities.output_schema_and_tools}")
    agent = Agent(model=model, name="probe", instruction="x",
                  tools=[get_current_time], output_schema=CityTime,
                  before_model_callback=spy)
    runner = InMemoryRunner(agent=agent, app_name=label)
    await runner.session_service.create_session(
        app_name=label, user_id="u", session_id="s")
    async for _ in runner.run_async(
        user_id="u", session_id="s",
        new_message=types.Content(role="user", parts=[types.Part(text="time in Sydney?")]),
    ):
        pass

async def main():
    await inspect("stock LiteLlm", LiteLlm(MODEL))
    await inspect("SchemaSafeLiteLlm", SchemaSafeLiteLlm(MODEL))

asyncio.run(main())

Output:

stock LiteLlm  output_schema_and_tools=True
  response_schema_set=True   tools_offered=['get_current_time']

SchemaSafeLiteLlm  output_schema_and_tools=False
  response_schema_set=False  tools_offered=['set_model_response', 'get_current_time']

Cost

Each retry resends the whole conversation, so the waste is worse than the call count suggests. Measured for the single question above, via OpenRouter:

Input tokens grow ~300x against ~26x on cost, so this degrades sharply with a realistic tool payload rather than the toy one used here. The failure is also silent: valid output is eventually returned, so nothing surfaces except the bill.

Expected behaviour

LiteLlm should not assert the capability on behalf of every model it wraps. When the wrapped model cannot pair response_schema with tools, ADK's existing SetModelResponseTool workaround should engage, as it does on the native path.

With the capability declared False, behaviour is correct and stable: 2 function calls per turn (get_current_time plus set_model_response), and a second conversation turn still calls the tool rather than answering from history.

Root cause

src/google/adk/models/lite_llm.py:3075

@property
@override
def capabilities(self) -> LlmCapabilities:
    # LiteLLM reconciles tools + response_format per provider: providers with
    # native support get both passed through, and the rest are converted to a
    # json tool call with tool_choice enforcement.
    return LlmCapabilities(output_schema_and_tools=True)

src/google/adk/flows/llm_flows/basic.py:137

if not agent.tools or model.capabilities.output_schema_and_tools:
    llm_request.set_output_schema(agent.output_schema)

The comment states the reasoning: LiteLLM is expected to reconcile the two per provider. For this provider and model it does not, and both constraints reach the model raw, as the reproduction shows.

By contrast the native path is conservative. gemini_output_schema_and_tools requires Vertex AI and a Gemini model id, so on other backends it correctly returns False and the workaround engages.

Note on model dependence

The model matters too, consistent with the table in #5054: flash models prefer calling tools over set_model_response, while gemini-3.1-pro-preview succeeds on the first call. The gemini-3.1-flash-lite used here is in the susceptible family. That is an argument for LiteLlm being conservative by default rather than asserting support for every wrapped model, since it cannot
know which family it is proxying.

Possible fixes

1. Default LiteLlm.capabilities to output_schema_and_tools=False, so the reliable SetModelResponseTool path is used unless a user opts out. Safe, at the cost of one extra function call for providers that could have handled it natively.
2. Make it constructor-configurable, e.g. LiteLlm(model, output_schema_and_tools=False), so users can declare it without subclassing.

Deriving it from LiteLLM's own metadata does not work. LiteLLM reports both capabilities independently, and has no API for the combination:

litellm.supports_response_schema(model="openrouter/google/gemini-3.1-flash-lite")   # True
litellm.supports_function_calling(model="openrouter/google/gemini-3.1-flash-lite")  # True

Both are True while the combination still loops, so the assumption in the current code comment cannot be rescued by consulting LiteLLM.

Happy to open a PR — combining 1 and 2 (a constructor argument defaulting to False) is a small diff, needs no network to test, and makes LiteLlm consistent with the conservative native gate. Glad to keep the default at True instead if preserving current behaviour matters more.

Related

• Agent with output_schema + tools enters infinite loop ADK v1.18.0 #3413 — output_schema + tools infinite loop
• [BUG]: output_schema=str combined with tools causes infinite call_llm/execute_tools loop #5054 — same loop with output_schema=str; contains the per-model comparison table
• fix: output_schema + tools infinite loop on Gemini 2.x #5057 — equivalent gate on the native Gemini path; closed as stale after the check moved into models/_capabilities. Does not cover the LiteLlm path.
• fix: harden SetModelResponseTool fallback to prevent infinite loops #5091 — hardens SetModelResponseTool; helps only once the fallback is active, which it never is here.