MCP Toolset OAuth2 User Interaction Flow Defects #3331

Describe the bug

The current Google ADK Python implementation has critical defects in the MCP (Model Context Protocol) Toolset OAuth2 authentication flow that prevent proper user interaction during the authentication process. MCP Toolset tool discovery occurs during the preprocessing phase using ReadonlyContext, which lacks the request_credential() method necessary for OAuth2 user interaction. This creates a fundamental architectural problem where authentication is required before tools can be discovered, but the authentication mechanism is not available at that stage.

To Reproduce

Steps to reproduce the behavior:

1. Install ADK Python with MCP Toolset support
2. Create an MCP Toolset that requires OAuth2 authentication:

   from google.adk.tools.mcp_tool import MCPToolset
   from google.adk.auth import AuthScheme, AuthCredential
   
   # Configure OAuth2 MCP Toolset
   toolset = MCPToolset(
       connection_params=connection_params,
       auth_scheme=oauth2_scheme,
       auth_credential=oauth2_credential
   )

3. Attempt to use the agent with the MCP Toolset
4. Observe MCP session creation failure during tool discovery phase with the following error stacktrace:

Traceback (most recent call last):
  File "/.../google/adk/flows/llm_flows/base_llm_flow.py", line 435, in _preprocess_async
    tools = await _convert_tool_union_to_tools(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../google/adk/agents/llm_agent.py", line 121, in _convert_tool_union_to_tools
    return await tool_union.get_tools_with_prefix(ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../google/adk/tools/base_toolset.py", line 114, in get_tools_with_prefix
    tools = await self.get_tools(readonly_context)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../google/adk/tools/mcp_tool/mcp_toolset.py", line 159, in get_tools
    session = await self._mcp_session_manager.create_session()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../google/adk/tools/mcp_tool/mcp_session_manager.py", line 362, in create_session
    await session.initialize()
  File "/.../mcp/client/session.py", line 151, in initialize
    result = await self.send_request(
             ^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError: Cancelled via cancel scope

Root Cause Analysis:

The error occurs because MCP Toolset attempts to create an MCP session during tool discovery in the preprocessing phase, but the session creation fails when OAuth2 authentication is required.

File: src/google/adk/flows/llm_flows/base_llm_flow.py (Lines 435-437)
Function: _preprocess_async()

# Problematic code in base_llm_flow.py
tools = await _convert_tool_union_to_tools(
    tool_union, ReadonlyContext(invocation_context)  # Problem: ReadonlyContext lacks auth capabilities
)

Call Chain:

1. _preprocess_async() → _convert_tool_union_to_tools() → tool_union.get_tools_with_prefix(ctx)
2. get_tools_with_prefix(ctx) → get_tools(readonly_context) → _mcp_session_manager.create_session()
3. create_session() → session.initialize() → MCP session initialization fails due to missing authentication
4. ReadonlyContext lacks request_credential() method to trigger OAuth2 authentication flow

Core Issues:

1. MCP session creation requires authentication: OAuth2-protected MCP servers require authentication before session initialization
2. ReadonlyContext lacks authentication capability: No request_credential() method available during preprocessing
3. Session initialization timing mismatch: MCP session creation happens in _preprocess_async, but authentication event processing occurs in _postprocess_async
4. Tool discovery dependency: MCP Toolsets must establish authenticated sessions before tools can be discovered

Expected behavior

1. Seamless MCP Toolset Authentication: MCP Toolsets should be able to trigger OAuth2 authentication flows during tool discovery phase with proper user interaction
2. Consistent Authentication Experience: OAuth2 authentication for MCP Toolsets should integrate with ADK's standard adk_request_credential event system
3. Runtime Compatibility: Authentication flows should work consistently across different execution environments (CLI, web server, deployed agents)
4. User-Friendly Flow: Users should receive clear authentication prompts and be able to complete OAuth2 flows seamlessly

Screenshots
Not applicable for this authentication flow issue.

Desktop (please complete the following information):
Not applicable for this authentication flow issue.

Model Information:

• Are you using LiteLLM: No
• Which model is being used: Gemini 2.0 Flash / Gemini 2.5 Pro

Additional context

Proposed Solution:

Modify BaseToolset.process_llm_request to Support Event Generation

Leverage the existing toolset processing mechanism in _preprocess_async to handle authentication:

1. Modify BaseToolset.process_llm_request signature: Change return type from None to AsyncGenerator[Event, None]
2. Update _preprocess_async to handle toolset events: Modify lines 479-481 in base_llm_flow.py to process events from toolsets:

   # Current code (line 479-481):
   await tool_union.process_llm_request(
       tool_context=tool_context, llm_request=llm_request
   )
   
   # Proposed change:
   async with Aclosing(
       tool_union.process_llm_request(
           tool_context=tool_context, llm_request=llm_request
       )
   ) as agen:
     async for event in agen:
       yield event

This addresses fundamental architectural limitations that prevent proper OAuth2 integration with MCP Toolsets, affecting both developer experience and end-user functionality in production deployments.