Skip to content

fix: collect eval state from workflow nodes - #6001

Closed
he-yufeng wants to merge 1 commit into
google:mainfrom
he-yufeng:fix/workflow-empty-state
Closed

he-yufeng wants to merge 1 commit into
google:mainfrom
he-yufeng:fix/workflow-empty-state

Conversation

@he-yufeng

Copy link
Copy Markdown
Contributor

Summary

Fixes #5995.

create_empty_state() only walked sub_agents, so graph-based Workflow roots crashed when the dev server tried to add the current session to an eval set. Workflow children live in workflow.graph.nodes, not sub_agents.

This updates the state traversal to:

  • keep the existing sub_agents walk for normal agents
  • also walk graph.nodes for Workflow-style roots and nested graph nodes
  • track visited objects so shared graph/agent nodes are not processed repeatedly

Testing

python -m pytest tests\unittests\cli\utils\test_state.py -q
python -m py_compile src\google\adk\cli\utils\state.py tests\unittests\cli\utils\test_state.py
python -m pyink --check src\google\adk\cli\utils\state.py tests\unittests\cli\utils\test_state.py
git diff --check

@adk-bot adk-bot added the eval [Component] This issue is related to evaluation label Jun 7, 2026
@adk-bot

adk-bot commented Jun 7, 2026

Copy link
Copy Markdown
Collaborator

I have successfully performed a comprehensive, read-only analysis of Pull Request #6001 and its related issue #5995.

I have created a premium, structured review report for you. You can read the full, detailed analysis in the newly generated artifact:
👉 pr_analysis_report.md

Key Highlights & Summary of Findings

  1. Google CLA Compliance Gate (Passed):
    • Verified that the contributor @he-yufeng has successfully signed and passed the standard Google CLA check (re-verified via standard GitHub checks status).
  2. Root Cause Resolved:
    • The state initialization utility create_empty_state() in state.py previously assumed the root agent is a standard BaseAgent and traversed child elements via .sub_agents.
    • In graph-based Workflows (introduced in ADK 2.x), child nodes live in .graph.nodes, which produced an AttributeError and crashed the ADK Dev Server with a 500 Internal Server Error when developers attempted to save active sessions to evaluation sets.
    • The PR resolves this systematically by dynamically querying and traversing both sub_agents and graph.nodes. It also elegantly handles cyclic or shared node graphs using an object identity cycle-tracker (visited set).
  3. Principles & Style Compliance (Pass with Nits):
    • Quality: The solution is robust and does not introduce any breaking changes to the public API namespace.
    • Testing: Excellent behavior-focused test coverage added in the new unit test file, utilizing 100% real components (Workflow, LlmAgent, and edge bounds) without mocking.
    • Styling/Typing Nits:
      • Nit 1: The new test file tests/unittests/cli/utils/test_state.py is missing the mandatory from __future__ import annotations header import.
      • Nit 2: Parameter typing in create_empty_state(...) was relaxed to Any instead of leveraging ADK's BaseNode interface (the common ancestor for both BaseAgent and Workflow), which could be annotated more strongly.

Next Steps and Recommendations

  • Recommendation: Approve with Nits. The patch is high-impact, low-risk, and handles DAG structures extremely cleanly. We can request

@rohityan rohityan self-assigned this Jun 9, 2026
@rohityan rohityan added the request clarification [Status] The maintainer need clarification or more information from the author label Jun 9, 2026
@rohityan

rohityan commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Hi @he-yufeng , Thank you for your contribution! We appreciate you taking the time to submit this pull request.
Can you address these 2 suggestions.
Nit 1: The new test file tests/unittests/cli/utils/test_state.py is missing the mandatory from __future__ import annotations header import.
Nit 2: Parameter typing in create_empty_state(...) was relaxed to Any instead of leveraging ADK's BaseNode interface (the common ancestor for both BaseAgent and Workflow), which could be annotated more strongly.

@he-yufeng
he-yufeng force-pushed the fix/workflow-empty-state branch from 4f54a2a to 7adfbb5 Compare June 10, 2026 14:19
@he-yufeng

Copy link
Copy Markdown
Contributor Author

Pushed 7adfbb5e after rebasing the branch cleanly onto current main.

This addresses both review nits:

  • added from __future__ import annotations to the new test file
  • narrowed the traversal helper and public function parameter from Any to BaseNode

Validation:

  • python -m py_compile src\google\adk\cli\utils\state.py tests\unittests\cli\utils\test_state.py
  • PYTHONPATH=src python -m pytest tests\unittests\cli\utils\test_state.py -q (3 passed)
  • python -m pre_commit run isort --files src\google\adk\cli\utils\state.py tests\unittests\cli\utils\test_state.py
  • python -m pre_commit run pyink --files src\google\adk\cli\utils\state.py tests\unittests\cli\utils\test_state.py
  • git diff --check upstream/main..HEAD

@he-yufeng
he-yufeng force-pushed the fix/workflow-empty-state branch 5 times, most recently from feec696 to dbd31bb Compare June 12, 2026 13:10
@he-yufeng

Copy link
Copy Markdown
Contributor Author

Rebased this on current main and re-ran the focused local checks:

  • python -m py_compile on the touched Python files
  • PYTHONPATH=src python -m pytest tests\unittests\cli\utils\test_state.py -q
  • pre_commit run isort on the touched files
  • python -m pyink --check on the touched files
  • git diff --check upstream/main..HEAD

The focused test run passed with 3 tests.

@he-yufeng
he-yufeng force-pushed the fix/workflow-empty-state branch 3 times, most recently from 2af17a5 to cc27387 Compare June 15, 2026 14:43
@he-yufeng

Copy link
Copy Markdown
Contributor Author

Rebased this onto current main and re-ran the focused checks.

Current head: cc27387f

Validation:

  • python -m py_compile src\google\adk\cli\utils\state.py tests\unittests\cli\utils\test_state.py
  • PYTHONPATH=src python -m pytest tests\unittests\cli\utils\test_state.py -q --basetemp .tmp\pytest-6001-round03 -p no:cacheprovider -> 3 passed
  • python -m pre_commit run isort --files src\google\adk\cli\utils\state.py tests\unittests\cli\utils\test_state.py
  • python -m pyink --check src\google\adk\cli\utils\state.py tests\unittests\cli\utils\test_state.py
  • git diff --check upstream/main..HEAD

@GWeale GWeale assigned GWeale and unassigned rohityan Jun 15, 2026
@he-yufeng
he-yufeng force-pushed the fix/workflow-empty-state branch 2 times, most recently from 2b97d62 to de4db1e Compare June 18, 2026 06:16
@he-yufeng
he-yufeng force-pushed the fix/workflow-empty-state branch from de4db1e to 1bc442a Compare June 18, 2026 10:14
@wdai-chwy

Copy link
Copy Markdown

+1. we see same error when we need to add eval for graph workflow in adk2. please prioritize this PR

@i-yliu i-yliu self-assigned this Jul 29, 2026
@kaligautier

Copy link
Copy Markdown
Contributor

Is there a temporary workaround?

copybara-service Bot pushed a commit that referenced this pull request Aug 7, 2026
Merge #6001

## Summary

Fixes #5995.

`create_empty_state()` only walked `sub_agents`, so graph-based `Workflow` roots crashed when the dev server tried to add the current session to an eval set. Workflow children live in `workflow.graph.nodes`, not `sub_agents`.

This updates the state traversal to:

- keep the existing `sub_agents` walk for normal agents
- also walk `graph.nodes` for Workflow-style roots and nested graph nodes
- track visited objects so shared graph/agent nodes are not processed repeatedly

## Testing

```
python -m pytest tests\unittests\cli\utils\test_state.py -q
python -m py_compile src\google\adk\cli\utils\state.py tests\unittests\cli\utils\test_state.py
python -m pyink --check src\google\adk\cli\utils\state.py tests\unittests\cli\utils\test_state.py
git diff --check
```

Co-authored-by: Yi Liu <yiliuly@google.com>
COPYBARA_INTEGRATE_REVIEW=#6001 from he-yufeng:fix/workflow-empty-state 1bc442a
PiperOrigin-RevId: 960722840
@adk-bot

adk-bot commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Thank you @he-yufeng for your contribution! 🎉

Your changes have been successfully imported and merged via Copybara in commit 568b4f6.

Closing this PR as the changes are now in the main branch.

@adk-bot adk-bot added the merged [Status] This PR is merged label Aug 7, 2026
@adk-bot adk-bot closed this Aug 7, 2026
FrigaZzz pushed a commit to FrigaZzz/adk-python that referenced this pull request Aug 11, 2026
Merge google#6001

## Summary

Fixes google#5995.

`create_empty_state()` only walked `sub_agents`, so graph-based `Workflow` roots crashed when the dev server tried to add the current session to an eval set. Workflow children live in `workflow.graph.nodes`, not `sub_agents`.

This updates the state traversal to:

- keep the existing `sub_agents` walk for normal agents
- also walk `graph.nodes` for Workflow-style roots and nested graph nodes
- track visited objects so shared graph/agent nodes are not processed repeatedly

## Testing

```
python -m pytest tests\unittests\cli\utils\test_state.py -q
python -m py_compile src\google\adk\cli\utils\state.py tests\unittests\cli\utils\test_state.py
python -m pyink --check src\google\adk\cli\utils\state.py tests\unittests\cli\utils\test_state.py
git diff --check
```

Co-authored-by: Yi Liu <yiliuly@google.com>
COPYBARA_INTEGRATE_REVIEW=google#6001 from he-yufeng:fix/workflow-empty-state 1bc442a
PiperOrigin-RevId: 960722840
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

eval [Component] This issue is related to evaluation merged [Status] This PR is merged request clarification [Status] The maintainer need clarification or more information from the author

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AttributeError in create_empty_state prevents adding current session to eval sets in ADK UI for Workflow-based agents

7 participants