You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[v1 backport] LocalEvalService still crashes on failed inference with inferences=None #6553
The current v1 branch (v1.37.0, commit 8b2feb5) still crashes in LocalEvalService._evaluate_single_inference_result() when inference fails before producing invocations.
This is the same bug reported in #5876 and fixed on main by #5878 / commit 9a6cf60.
The fix has not been backported to v1.
As on main, a failed inference without invocations should produce an EvalCaseResult with final_eval_status=EvalStatus.FAILED, without aborting the rest of the evaluation run.
Observed Behavior:
The entire evaluation fails while processing len(None), masking the original inference error.
Environment Details:
ADK Library Version: 1.37.0 (v1 branch)
Desktop OS: macOS
Python Version: 3.13.12
Model Information: N/A; the failure path is model-independent
🟡 Optional Information
Proposed Fix / Backport:
Please cherry-pick the existing fix from main:
9a6cf60fa8d54523e95943ebdb49d4f35341aed0
fix(eval): handle failed inference results without invocations
I verified that the complete commit applies cleanly to the current v1 branch (8b2feb52) without conflicts.
I also ran the focused tests after applying it to v1:
4 passed, 15 deselected
Command:
pytest tests/unittests/evaluation/test_local_eval_service.py -k "failed_without_inferences or evaluate_single_inference_result" -q
Regression History:
A previous fix for #2729 was added by #3805 / commit 7d4326c, but the evaluation changes and regression tests were removed in the immediately following commit 81eaeb5.
The current main fix in 9a6cf60 is a smaller backport and already matches the desired v2 behavior.
@ftnext — confirmed the same crash on our end against v1.37.0, and it's exactly the len(None) gap left over from #5876.
Your cherry-pick of 9a6cf60 lines up with what we'd suggest too, and since you've already verified it applies cleanly with the focused tests passing, go ahead and open the PR against v1 whenever you're ready.
Activity
surajksharma07 commentedon Aug 3, 2026
@ftnext — confirmed the same crash on our end against v1.37.0, and it's exactly the len(None) gap left over from #5876.
Your cherry-pick of 9a6cf60 lines up with what we'd suggest too, and since you've already verified it applies cleanly with the focused tests passing, go ahead and open the PR against v1 whenever you're ready.
Team will review it once it's up!
ftnext commentedon Aug 3, 2026
@surajksharma07 Thanks! I've opened PR #6565 to backport the fix to v1. I appreciate your review.