Repository navigation
fix: Add retry with exponential backoff for 429 errors and batch append support in VertexAiSessionService - #5720
Closed
mahasarabesh wants to merge 3 commits into
Closed
mahasarabesh wants to merge 3 commits into
mahasarabesh wants to merge 3 commits into
Conversation
…nd support - Add _append_with_retry() to VertexAiSessionService with exponential backoff and jitter on 429 RESOURCE_EXHAUSTED (up to 5 attempts) - Add append_events_batch() to BaseSessionService (default sequential) - Override append_events_batch() in VertexAiSessionService with asyncio.Semaphore(5) concurrency control - Add 5 new tests covering retry, max retries, non-429, and batch Fixes google#5714
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
rohityan
added
the
request clarification
Collaborator
|
Hi @mahasarabesh , Thank you for your contribution! We appreciate you taking the time to submit this pull request. Can you please fix the pre commit check errors and failing unit test. |
rohityan
added
the
services
copybara-service Bot
pushed a commit
that referenced
this pull request
Sep 3, 2026
Merge #5720 VertexAiSessionService built its API client with no retry configuration, so a 429 RESOURCE_EXHAUSTED failed on the first response. It now retries 429 once. A call that keeps getting 429 therefore issues one extra request and takes about a second longer before it raises. This does not resolve #5714, which asks for batched or windowed event replay. That replay loop is in the vertexai SDK's _init_session, not here, and the issue notes each retry consumes more of the same per-minute quota, which is why the retry is bounded to a single attempt. PiperOrigin-RevId: 975844908
Collaborator
|
Thank you @mahasarabesh for your contribution! 🎉 Your changes have been successfully imported and merged via Copybara in commit cc5985a. Closing this PR as the changes are now in the main branch. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5714
Problem
When deployed on Agent Engine,
_init_session()replays all prior session events by callingsession_service.append_event()individually for each event. For sessions with many events, this triggers429 RESOURCE_EXHAUSTEDerrors from the Vertex AI API due to rate limiting, causing sessions to fail on initialization.The root cause is in the external
vertexaiSDK (outside this repo), butadk-pythoncan defend against this on its own side.Changes
1. Retry with exponential backoff on
append_event()(core fix)VertexAiSessionServicenow retries API calls that fail with HTTP 429 using exponential backoff with jitter:2.
append_events_batch()methodAdded
append_events_batch()toBaseSessionServicewith a default sequential implementation.VertexAiSessionServiceoverrides it withasyncio.Semaphore(5)concurrency control, allowing callers that need to replay many events to do so with bounded parallelism instead of sequential one-by-one calls.Files changed
sessions/base_session_service.pyappend_events_batch()default implementationsessions/vertex_ai_session_service.py_append_with_retry(), overriddenappend_events_batch()with semaphoretests/.../test_vertex_ai_session_service.pytests/.../test_session_service.pyTesting
I'm new to contributing to this project and happy to make changes based on your review comments. Looking forward to your feedback!