Skip to content

before_run_callback early-exit is silently ignored on the node execution path (root LlmAgent and Workflow) — reopen of #6013 #6828

Closed

Description

@Dratatus

Describe the bug
Runner has two execution paths. The legacy path (_exec_with_plugin) honors the documented before_run_callback contract: a plugin returning types.Content halts the run and that content becomes the final response. The node execution path (_run_node_async) calls plugin_manager.run_before_run_callback(...) but discards the return value, so the run continues as if the plugin had returned None.

This was reported in #6013 (confirmed by adk-bot as high priority) but that issue was closed by the stale bot on 2026-07-04 without a fix, and the fix PR #6032 has been waiting for human review since June and no longer merges cleanly.

Why this matters
Every root that dispatches through the node path is affected: a Workflow root and, since 2.7.0, any root LlmAgent. In practice, plugin-based guardrails (safety filters that block a turn in before_run_callback, e.g. Model Armor-style input screening) are silently bypassed for the most common application shape. We hit this while hardening a production healthcare analytics assistant: the blocked turn reaches the model anyway.

To Reproduce
Minimal repro on google-adk 2.7.0 (also reproduces on current main):

import asyncio
from google.adk.agents.llm_agent import Agent
from google.adk.agents.invocation_context import InvocationContext
from google.adk.apps.app import App
from google.adk.plugins.base_plugin import BasePlugin
from google.adk.runners import Runner
from google.adk.sessions.in_memory_session_service import InMemorySessionService
from google.genai import types

class HaltPlugin(BasePlugin):
    def __init__(self):
        super().__init__(name="halt")
    async def before_run_callback(self, *, invocation_context: InvocationContext):
        return types.Content(role="model", parts=[types.Part(text="halted by plugin")])

async def main():
    agent = Agent(name="root_agent", model="gemini-2.5-flash")
    ss = InMemorySessionService()
    runner = Runner(app=App(name="repro", root_agent=agent, plugins=[HaltPlugin()]),
                    session_service=ss)
    session = await ss.create_session(app_name="repro", user_id="u")
    async for event in runner.run_async(
        user_id="u", session_id=session.id,
        new_message=types.Content(role="user", parts=[types.Part(text="hi")]),
    ):
        print(event.author, event.content)

asyncio.run(main())

Expected behavior
The run halts with the plugin's content as the final response and the model is never called — the behavior _exec_with_plugin already implements, and what the plugin docs promise ("Returning a value to halt execution of the runner and ends the runner with that event").

Actual behavior
The plugin's content is discarded and the model generates a normal response.

Desktop:

  • OS: macOS 15
  • Python version: 3.12
  • ADK version: 2.7.0 (also reproduced on main @ 4599a52)

Additional context
We have prepared a PR that rebases the stale fix #6032 onto current main (with credit to its author) and extends it with a root-LlmAgent regression test. Both new tests fail without the fix and pass with it; the full tests/unittests/test_runners.py and tests/unittests/workflow/ suites pass (738 passed). Happy to iterate on review feedback.

Activity

  1. added 2 commits that reference this issue on Aug 20, 2026
    3173ae7
    847b851
  2. Dratatus commented on Aug 20, 2026

    @Dratatus
    Author

    Closing for now — will be resubmitted together with the fix by the colleague who did the original investigation.

  3. javiervillam-axiom-robotics commented on Aug 21, 2026

    @javiervillam-axiom-robotics

    Independent confirmation against 2.7.1, plus what it cost in production — posting here rather than opening a third issue, since this one is already tracked pending a fix.

    Still present in 2.7.1

    runners.py:

    • _run_node_async → await ic.plugin_manager.run_before_run_callback(invocation_context=ic) — the return value is awaited and dropped.
    • _exec_with_plugin → early_exit_result = await plugin_manager.run_before_run_callback(...), then if isinstance(early_exit_result, types.Content): ... — honoured.

    run_async selects mode='chat' for a plain LlmAgent, which routes to _run_node_async. So a plugin that returns a Content from before_run_callback on the most common agent shape has its return silently discarded, while base_plugin.py's docstring states the opposite: "The content to return to the user. When the content is present, the provided content will be used as agent response and appended to event history as agent response."

    Why it is worth prioritising: it is silent in the worst way

    We used before_run_callback for a Gemini-as-a-judge prompt-injection screen. Its documented contract — the attack never reaches the agent — did not hold, and nothing surfaced that:

    • the plugin logged BLOCKED user message: category=jailbreak and returned its refusal Content, so the logs said the screen worked;
    • the model call started ~15 ms later anyway;
    • the user got the model's answer, not the refusal.

    A guardrail that logs success while doing nothing is worse than one that raises. The only reason we found it is that we happened to be measuring token usage on those turns.

    Workaround, for anyone hitting this before the fix lands

    Move the short-circuit to before_model_callback and return an LlmResponse instead of a Content. That is honoured on both modes because the code lives in the shared BaseLlmFlow (_handle_before_model_callback runs plugin callbacks first and, on a non-None return, yields the response and returns before the model is called), and base_plugin.py documents returning an LlmResponse there as the supported way to skip the model call.

    Two caveats that cost us time and are not obvious from the docs:

    1. before_model_callback fires once per MODEL CALL, not once per turn — so a per-turn screen re-runs on every post-tool continuation. We latch the invocation id in temp: state (which BaseSessionService._apply_temp_state keeps in-memory for the invocation and trims before persistence) so the judge classifies once. Without that, a 3-tool turn pays 4 classifier round trips on the same user message.
    2. The screen now runs after the LlmRequest has been assembled rather than before the runner starts. No model call is made, so the cost is request-building only — but it is not a free move if anything in your request assembly is expensive.

    Suggestion

    If the fix ends up being "honour the return in _run_node_async too", it would be worth adding a test that covers the branch on both paths — run_async with a plain LlmAgent is the shape most users have, and it is the one that was uncovered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions