Repository navigation
feat: add an injectable scheduler for ADK's internal fan-out - #1587
Conversation
332d0a1 to
d20e32f
Compare
d20e32f to
23b6695
Compare
23b6695 to
725e21b
Compare
|
Thanks @damianmomotgoogle for the review! All six threads are addressed in 1b89d23 (single commit, rebased onto current Full |
ADK hard-codes the RxJava IO scheduler wherever it leaves the caller's thread: ParallelAgent subscribes each sub-agent on Schedulers.io(), tool calls in ToolExecutionMode.PARALLEL_SUBSCRIBE are subscribed on it unless the LlmAgent has an executor, and the live send loop is started on it. An embedding application has no way to influence that fan-out, for example to cap it, to run it on an existing worker pool, or to keep every ADK task on a dispatcher it controls, which a durable-execution host must. adk-go exposes this control point as platform.WithTaskRunner / platform.RunTasks (google/adk-go#1050). Add the Java analog as an RxJava Scheduler carried on InvocationContext. Runner.builder().scheduler(...) sets it once, every InvocationContext the runner creates carries it (and toBuilder() copies it), and InvocationContext.scheduler() returns it or Schedulers.io() when none is configured, so the default behavior is unchanged. ParallelAgent uses the invocation's scheduler unless its own builder set one (its default moves from an eager Schedulers.io() to "inherit"), Functions resolves the PARALLEL_SUBSCRIBE worker from the LlmAgent executor and then the invocation, and BaseLlmFlow's live send loop goes through the same helper. AgentTool builds the runner it nests with Runner.builder() and hands it the caller's scheduler, so a nested agent fans out on the same threads as its caller. It is an RxJava Scheduler rather than a task-runner function because every hop site already takes one, and because a Scheduler can also express the delayed scheduling RxJava's time-based operators need, which a plain Executor cannot. Agent-level settings keep precedence, so existing ParallelAgent.Builder.scheduler() and LlmAgent.Builder.executor() callers see the same threading. A null scheduler means "inherit the invocation's" on ParallelAgent.Builder and "use the default" on Runner.Builder, and InvocationContext.equals()/hashCode() compare the configured scheduler field, so a context still equals its toBuilder() copy.
725e21b to
1b89d23
Compare
Link to Issue or Description of Change
Problem:
ADK hard-codes the RxJava IO scheduler wherever it leaves the caller's thread:
ParallelAgentsubscribes each sub-agent onSchedulers.io(), tool calls inToolExecutionMode.PARALLEL_SUBSCRIBEare subscribed on it unless theLlmAgenthas anexecutor, and the live send loop is started on it. An embedding application has no way to influence that fan-out, for example to cap it against rate-limited backends, to dispatch it onto an existing worker pool, or to keep every ADK task on a dispatcher it controls, which a durable-execution host must do to stay deterministic.adk-goexposes exactly this control point asplatform.WithTaskRunner/platform.RunTasks(google/adk-go#1050), next to its time/UUID provider seams. This is the Java analog, and the companion to #1255 (time and UUID providers); the two are independent and either can land first. On alignment withadk-python: its asyncio model has no equivalent seam (parallel tool calls run as tasks on the caller's event loop, and the optional tool thread pool is opt-in viaRunConfig), soadk-go'splatform.WithTaskRunneris the closest analog to mirror.Solution:
Carry an RxJava
ScheduleronInvocationContext, configured once on theRunner:Runner.builder().scheduler(@Nullable Scheduler)— the scheduler for the runner's invocations;null(the default) selectsSchedulers.io(). EveryInvocationContextthe runner creates (new, resumed, and live invocations) carries it, andtoBuilder()copies it to sub-agent and branch contexts.InvocationContext.scheduler()— returns the configured scheduler, orSchedulers.io()when none is set, so the default behavior is unchanged.InvocationContext.Builder.scheduler(@Nullable Scheduler)is the plumbing.ParallelAgentuses the invocation's scheduler unless its ownBuilder.scheduler(...)was set. Its builder default moves from an eagerSchedulers.io()to "unset, inherit", andscheduler(null)means unset, so existing callers keep working; the effective default is the same. (One benign change: onmain,scheduler(null)stored the null and a run with sub-agents then failed with an NPE fromsubscribeOn(null); now such an agent inherits the invocation's scheduler.)Functions.resolveWorkerScheduler, used for thePARALLEL_SUBSCRIBEtool fan-out and for the hop that starts the live send loop inBaseLlmFlow(both previously fell back toSchedulers.io()).AgentToolbuilds the runner it nests withRunner.builder()(same in-memory artifact, session, and memory services, plugins, app name, and null compaction/cache/resumability configs as theInMemoryRunnerit replaces, and the caller'sRunConfighandling is unchanged) and passes the caller's invocation scheduler, so nested fan-out inherits it.InvocationContext.equals()/hashCode()include the configured scheduler field as-is: a context always equals itstoBuilder()copy (noRxJavaPluginshandler runs during a comparison), and an unset scheduler is not equal to an explicitSchedulers.io().Why an RxJava
Schedulerrather than a Go-style task-runner function: every Java hop site already takes aScheduler, and aSchedulercan also express the delayed scheduling RxJava's time-based operators need, which a plainExecutorcannot (Schedulers.from(Executor)needs a helper thread to time delays). A host that wants ADK's own hops removed installsSchedulers.trampoline(), which runs them inline on the thread that would otherwise hand off (sub-agents and tools are then subscribed sequentially, though anything they start asynchronously can still overlap); a host with its own dispatcher implementsScheduleronce. A single-threaded scheduler must not be blocked from its own thread (for example by callingblockingGet()on a run from that thread, or from a tool), or the queued fan-out tasks never run.Not changed here: hops outside the Runner → agent → flow path, for example the MCP retry timers (
Flowable.timer/delayon the computation scheduler, tool-side), the Vertex session client's retry timer, the model clients' HTTP executors, the agent-analytics plugin's own executors, and the dev server's ownSchedulers.io()hops. Also observed while testing, pre-existing and left alone:ParallelAgentdoes not re-activate the OpenTelemetry context inside itssubscribeOnhop (asFunctionsdoes for tool calls), so sub-agent spans are parented to the root under any non-inline scheduler, including today's default; aTracing.withContext(parentContext)inside the hop fixes it and I'm happy to send that separately.Testing Plan
Unit Tests:
New tests (all under
core):InvocationContextTest—scheduler()defaults toSchedulers.io(), a custom scheduler is carried and copied bytoBuilder(),nullselects the default, and equality compares the configured scheduler (a context equals itstoBuilder()copy; unset differs from an explicitSchedulers.io()).ParallelAgentTest— without an agent-level scheduler the sub-agents run on the invocation's scheduler (non-resumable and resumable paths; the resumable one also pins the exact event sequence under the trampoline); an agent-level scheduler takes precedence;scheduler(null)inherits.FunctionsTest—PARALLEL_SUBSCRIBEtool calls run on the invocation's scheduler (for anLlmAgentwithout an executor and for a non-LLM agent); anLlmAgent.executor()takes precedence. Shared body in one helper that returns the worker count.RunnerTest—Runner.builder().scheduler(...)reaches aParallelAgentrun;scheduler(null)falls back to the default; therunLivesend loop runs on the runner's scheduler (trampoline-backed, and the test model is asserted to have received the request) and anLlmAgent.executor()takes precedence on that hop.AgentToolTest— the run anAgentToolnests inherits the caller's scheduler.testing/RecordingScheduler— a testSchedulerthat delegates to another scheduler (the trampoline in these tests, so they are single-threaded and assert exact worker counts) and counts the workers ADK requested from it.Local results (
./mvnw -Prelease clean package, OpenJDK 17 (Homebrew 17.0.20.1), all modules):BUILD SUCCESS;core:Tests run: 2063, Failures: 0, Errors: 0, Skipped: 24(the skips are pre-existing); every other module passes with 0 failures.git diff --exit-codeafter the build: clean (formatter made no changes).Manual End-to-End (E2E) Tests:
N/A; this is a low-level concurrency seam with no runner UI surface. The default path (
Schedulers.io()when no scheduler is installed) is unchanged, and the installed-scheduler path is exercised by the unit tests above.Checklist
Additional context
Motivated by parity with
adk-go(platform.WithTaskRunner, google/adk-go#1050) and by the durable-execution use case discussed on #1255, where a host must be able to keep ADK's fan-out on threads it controls. #1255 and this PR touch the same spots inInvocationContextandRunner(a new field, builder setter, and constructor parameter each) and the same runner-construction block inAgentTool, so whichever lands second needs a small, mechanical rebase; I'll do it promptly.