LiteLLM Streaming Content Duplication in Tool Call Responses

Summary

When using LiteLLM models with ADK's planning features (e.g., PlanReActPlanner) in streaming mode, planning and reasoning content appears twice in responses when tool calls are made:

First during streaming as individual text chunks (lines 1288-1296)
Again in the aggregated tool-call message with content=text (line 1352)

This violates OpenAI/LiteLLM conventions and creates unnecessary duplication in conversation history.

Environment

ADK Version: 1.19.0
Affected File: lite_llm.py
Python Version: 3.11+
Models Affected: All non-Gemini models accessed via LiteLLM (Claude, GPT, etc.) when using planning workflows
Feature: Streaming responses with tool calls

Expected Behavior

According to OpenAI/LiteLLM API specifications:

When a message contains only tool calls (no user-facing answer text), the content field should be None
Planning/reasoning text like <PLANNING>I need to search...</PLANNING> is internal reasoning, not the final answer
Tool-call messages should follow this structure:

{
  "role": "assistant",
  "content": None,  # No content for tool-only messages
  "tool_calls": [...]
}

Actual Behavior

The aggregated response at line 1348-1359 sets content=text, including all accumulated planning/reasoning text:

aggregated_llm_response_with_tool_call = (
    _message_to_generate_content_response(
        ChatCompletionAssistantMessage(
            role="assistant",
            content=text,  # Includes planning text, causing duplication
            tool_calls=tool_calls,
        ),
        model_version=part.model,
        thought_parts=list(reasoning_parts)
        if reasoning_parts
        else None,
    )
)

Result: Planning text appears twice:

During streaming (lines 1288-1296): <PLANNING>I need to search...</PLANNING> streamed chunk-by-chunk
In aggregated message (line 1352): Same text included in content field

Impact

1. Content Duplication

Frontend receives the same planning text twice
Requires additional filtering logic in application code
Poor user experience if not handled

2. API Convention Violation

OpenAI/Claude/GPT APIs expect content=None for tool-only messages
Current implementation sends content=<planning_text>, which is semantically incorrect
Tool-call messages should not contain answer text in content

3. Conversation History Bloat

Planning text unnecessarily stored in message content field
Already preserved separately in thought_parts (line 1357)
Increases storage and memory overhead

4. Semantic Confusion

content=text implies "model generated answer text AND called tools"
Reality: model only generated internal reasoning before calling tools
Misrepresents the actual interaction flow

Steps to Reproduce

Create an agent with LiteLLM model:

from google.adk.agents import Agent
from google.adk.models import LiteLlm
from google.adk.planners import PlanReActPlanner

agent = Agent(
    model=LiteLlm(model="vertex_ai/claude-3-5-sonnet-v2@20241022"),
    planner=PlanReActPlanner(),
    tools=[search_tool, ...]
)

Enable streaming and send a query requiring tools:

async for response in agent.run_streaming("What's the weather in Boston?"):
    print(response.content)

Observe in logs:
- Planning text like <PLANNING>I need to search for weather</PLANNING> streamed as chunks
- Same planning text appears again in aggregated response content field when tool calls are made
Check conversation history:
- Tool-call message has content="<PLANNING>..." instead of content=None

Root Cause

Lines 1268-1303: Accumulate all text chunks (including planning) into text variable:

text = ""
...
elif isinstance(chunk, TextChunk):
    text += chunk.text  # Accumulates planning/reasoning text
    yield _message_to_generate_content_response(...)  # Already streamed to user

Line 1352: Includes accumulated text again in aggregated message:

content=text,  # Duplicates already-streamed planning text

Line 1357: Planning already preserved separately:

thought_parts=list(reasoning_parts) if reasoning_parts else None,

Proposed Fix

Change line 1352 to set content=None for tool-only messages:

aggregated_llm_response_with_tool_call = (
    _message_to_generate_content_response(
        ChatCompletionAssistantMessage(
            role="assistant",
            content=None,  # ✅ FIX: No duplication, follows OpenAI/LiteLLM spec
            tool_calls=tool_calls,
        ),
        model_version=part.model,
        thought_parts=list(reasoning_parts)
        if reasoning_parts
        else None,
    )
)

Comparison with Non-Streaming

Non-streaming path (around line 770) correctly handles this by:

Creating a single response with complete tool call information
No opportunity for duplication (no incremental streaming)

Streaming path (lines 1268-1400) has the duplication issue because:

Text chunks yielded immediately during streaming
Then included again in final aggregated message

The fix brings streaming behavior in line with non-streaming and API conventions.

Additional Context

This issue specifically affects planning workflows where models generate reasoning text before calling tools
Does not affect simple tool-call scenarios without planning text
thought_parts parameter already exists to preserve reasoning separately from message content
Frontend applications using ADK planning need to implement workarounds to deduplicate content

Recommended Fix (Summary)

# Line 1348-1359
aggregated_llm_response_with_tool_call = (
    _message_to_generate_content_response(
        ChatCompletionAssistantMessage(
            role="assistant",
            content=None,  # ✅ FIX: Avoid duplication, follow OpenAI spec
            tool_calls=tool_calls,
        ),
        model_version=part.model,
        thought_parts=list(reasoning_parts)
        if reasoning_parts
        else None,
    )
)

This single-line change eliminates content duplication, aligns with API standards, and maintains semantic correctness for tool-call messages in streaming responses.

fix: Set content=None for tool-only messages in streaming to avoid du…

@thesynapses

@thesynapses Thanks for reporting
The duplication occurs at lite_llm.py:1649 where content=text includes already-streamed planning text in the aggregated tool-call message.

Your PR #3698 with the fix (content=None) correctly follows OpenAI/LiteLLM conventions and is ready for review.

Until it's merged, you can work around this by deduplicating content in your application code using a set to track seen text.
The fix will eliminate duplication and properly preserve planning in thought_parts only.

Thanks!

fix: Set content=None for tool-only messages in streaming to avoid du…

This issue has been automatically marked as stale because it has not had recent activity for 7 days after a maintainer requested clarification. It will be closed if no further activity occurs within 7 days.

For duplication bugs in streaming tool responses, I’d instrument chunk IDs and tool-call correlation IDs at the adapter boundary first. In similar cases, duplication came from replayed partials after reconnect or from mixing delta + aggregate buffers.

Useful triage data:

provider + model + streaming mode
whether duplication appears in raw stream vs post-processor output
exact sequence of tool_call / tool_result events

If raw stream is clean but final output duplicates, the merge layer is the likely culprit.

fix: Set content=None for tool-only messages in streaming to avoid du…

@thesynapses

@thesynapses, agree with your diagnosis — pulled main and confirmed that setting content=None on tool-only messages clears up the duplication cleanly.

PR #3698 already has the right fix, it's just drifted from main since November given how much lite_llm.py has changed since.

Could you rebase it once more and confirm it's still clean on your end so we can push for review?

fix(litellm): avoid duplicate streamed tool-call content

@surajksharma07

@surajksharma07 I rebased the fix onto current main and confirmed it is clean.

GitHub would not allow me to reopen #3698 after replacing its stale head (422 Validation Failed), including after temporarily restoring the PR's recorded head. I therefore opened #6670 from the same branch as the replacement.

The new PR contains one commit and only two files:

content=None for the final aggregated tool-call response
focused regression coverage proving streamed text is not repeated in that final response

Validation: 361 passed in tests/unittests/models/test_litellm.py; repository-pinned Pyink and Isort checks also pass.

PR: #6670

@thesynapses

@thesynapses Confirmed on my end too. #6670's diff is exactly the content=None fix at the current line plus a tightened regression assertion and CI's clean. Sorry about the #3698 reopen mess, that's on GitHub's 422 behavior not anything you did wrong. Given how long this has already dragged through rebases not asking for another round trip, pushing this to review now.

Team is looking more into it.

test: verify pre-tool-call text preservation in LiteLLM streaming

Navigation Menu

Repository navigation

LiteLLM Streaming Content Duplication in Tool Call Responses #3697

Summary

Environment

Expected Behavior

Actual Behavior

Impact

1. Content Duplication

2. API Convention Violation

3. Conversation History Bloat

4. Semantic Confusion

Steps to Reproduce

Root Cause

Proposed Fix

Comparison with Non-Streaming

Additional Context

Recommended Fix (Summary)

14 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

LiteLLM Streaming Content Duplication in Tool Call Responses #3697

Description

Summary

Environment

Expected Behavior

Actual Behavior

Impact

1. Content Duplication

2. API Convention Violation

3. Conversation History Bloat

4. Semantic Confusion

Steps to Reproduce

Root Cause

Proposed Fix

Comparison with Non-Streaming

Additional Context

Recommended Fix (Summary)

Activity

klateefa commented on Dec 23, 2025

adk-bot commented on Feb 17, 2026

adk-bot commented on Feb 21, 2026

14 remaining items

adk-bot commented on Mar 8, 2026

saschabuehrle commented on Mar 8, 2026

surajksharma07 commented on Jul 28, 2026

thesynapses commented on Aug 10, 2026

surajksharma07 commented on Aug 12, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions