Repository navigation
Conversation
Adds PgVectorMemoryService under google.adk.integrations.pgvector, a BaseMemoryService backed by PostgreSQL with the pgvector extension. Events are embedded (via the google-genai client by default, or an injectable embedder) and stored with their vectors in a table the user owns; search_memory returns the nearest memories by cosine distance using a pgvector HNSW index. Unlike VertexAiMemoryBankService and VertexAiRagMemoryService it needs no Google Cloud, and unlike InMemoryMemoryService it persists across restarts and ranks by semantic similarity rather than keyword overlap. This fills the self-hosted, non-GCP semantic-memory gap raised in google#6254 and complements the exact-match SQLite memory service in google#4116. The service follows the existing integrations/redis pattern: a pydantic config plus an injectable connection pool and embedder, which keeps it unit-testable without a live database. Ships as an optional extra, google-adk[pgvector]. Includes unit tests that exercise ingestion, idempotent upserts, semantic ranking, per-(app_name, user_id) scoping, the distance threshold, and serialization round-trips against an in-process fake pool and a deterministic embedder, so they run without psycopg, pgvector, or any network access. Resolves google#7273
96f7abb to
a04727e
Compare
|
Hi @DeanChensj — friendly ping on this one whenever you have a chance. It's green across CI, unit-tested with the optional driver both present and absent, and I verified it end-to-end against a live PostgreSQL + pgvector instance (ranking, persistence, scoping, idempotent ingest, and the created schema). Happy to adjust anything — including aligning the table layout with the SQLite memory service in #4116 if you'd prefer a shared schema. Thanks! |
|
Gentle second follow-up — this has been green and ready for ~2 weeks. Is there anything I can do to help move the review along, or a better-suited reviewer I should loop in? Happy to rebase or adjust the schema to match the SQLite memory service if that helps. Thanks! |
|
Friendly ping — this is still green and rebased-clean, and I'm happy to make any changes needed to get it reviewed. Is there anything I can do to help move it forward, or another maintainer better placed to take a look? Thanks! |
…y-service # Conflicts: # pyproject.toml
|
Hi @GWeale — would you be able to take a look here, or point me to the right reviewer? This adds a |
|
Hi @Nanduu24, thank you for this contribution and for your patience. Sorry it took us so long to reply. The PR is clearly written, well tested with and without the optional driver, and verified end to end against a live database. We appreciate the care you put into it. After a careful look, we've decided not to merge Maintenance burden
Design points we'd want addressed for any version of this
Alternatives
This isn't a "never". If we see clear community demand for a first-party Postgres/pgvector memory service, we'll gladly re-evaluate. Reactions and use cases on #7273 are the best way to show that. Thanks again for the thoughtful work, and we hope to see more contributions from you! |
|
Thanks @DeanChensj — I really appreciate the thorough and candid reply, and the reasoning makes complete sense. The long-term maintenance commitment for a first-party DB backend is real, and the design points (re-embedding on each add, batching writes and embed calls, runtime |
Link to Issue or Description of Change
Problem:
ADK's memory services do not cover persistent, semantic memory on
infrastructure the user controls.
InMemoryMemoryServiceis keyword-only and"for prototyping purpose only" (loses data on restart), while
VertexAiMemoryBankServiceandVertexAiRagMemoryServicerequire Google Cloud.Teams running on-premise, air-gapped, or under data-residency constraints have
no built-in option, and the pending SQLite memory service (#4116) provides
persistence but exact-match retrieval rather than semantic search.
Solution:
Add
PgVectorMemoryServiceundergoogle.adk.integrations.pgvector, aBaseMemoryServicebacked by PostgreSQL +pgvector:add_session_to_memory/add_events_to_memoryembed each event's text andupsert it (idempotent per event) into a table scoped by
(app_name, user_id).add_memorysupports direct writes of explicitMemoryEntryitems.search_memoryembeds the query and returns the nearest memories by cosinedistance using a pgvector HNSW index, with an optional
distance_threshold.google-genaiclient by default; an injectableembeddermakes the service provider-agnostic.
pip install "google-adk[pgvector]".It follows the existing
integrations/redispattern (pydantic config plus aninjectable connection pool and embedder), so the store runs entirely on a
database the user owns and stays unit-testable without a live database. The
table,
vectorextension, and indexes are created on first use. Happy to alignthe table layout with #4116 if a shared schema is preferred.
Testing Plan
Unit Tests:
New tests in
tests/unittests/integrations/pgvector/run against an in-processfake connection pool and a deterministic bag-of-words embedder, so they exercise
ingestion, idempotent upserts, semantic ranking, per-
(app_name, user_id)scoping, the distance threshold, empty-query handling, skipping text-less
events, the direct-write path, serialization round-trips, and the
missing-driver / missing-dsn errors — all without
psycopg,pgvector, or anynetwork access.
The suite passes both with the optional driver installed and with it absent
(matching CI). Lint/format on the changed files:
pyinkandisortclean,pylint10.00/10.Manual End-to-End (E2E) Tests:
Verified against a live PostgreSQL 17 + pgvector 0.8.6 database using the
real
psycopgasync driver (a deterministic local embedder stood in for theembedding API). Confirmed: session ingestion, semantic ranking (a "billing"
query returns the billing turn; a "basketball" query returns the basketball
turn), cross-instance persistence (a fresh service reads the stored rows back),
(app_name, user_id)scoping, idempotent re-ingestion (row count stays at 3after a double ingest), and the created schema (a
vector(64)column and anHNSW index). To reproduce:
Checklist
Additional context
Design mirrors the
integrations/redissession service and thegoogle-genaiembedding usage in
tools/spannerandintegrations/mongodb. The new optionalextra
pgvectorpinspsycopg[binary],psycopg-pool, andpgvector, and isincluded in the
allextra.