Skip to content

Security: RCE Mitigation via HMAC-SHA256 Signatures, Hybrid JSON Refactor, and CLI Source Validation - #5333

Closed
JoshuaProvoste wants to merge 12 commits into
google:mainfrom
JoshuaProvoste:security/fix-adk-pickle-sinks
Closed

JoshuaProvoste wants to merge 12 commits into
google:mainfrom
JoshuaProvoste:security/fix-adk-pickle-sinks

Conversation

@JoshuaProvoste

Copy link
Copy Markdown

Compliance with CONTRIBUTING.md:

Consistently with the project guidelines, this PR includes:

  • Security Hardening: Implemented a "Defense in Depth" architecture combining Cryptographic HMAC-SHA256 signatures and a "Secure by Default" hybrid JSON refactor to eliminate multiple RCE vectors.
  • Verified Fix: Successfully validated the neutralization of RCE sinks in the migration loop and shared state storage (MySQL/Spanner) through 9 dedicated security tests.
  • Code Hygiene: Enforced Google-specific code style using isort and pyink across all modified files.
  • Vulnerability Remediation: Implemented network-level validation in the CLI to prevent "Remote Pivot" attacks via SMB/UNC and external IP redirection.

Description

This PR addresses critical security vulnerabilities in the google-adk framework involving insecure pickle deserialization. The previous implementation allowed for Remote Code Execution (RCE) by loading unauthenticated binary blobs from session databases or remote migration sources.

To remediate this, I have transitioned the framework to a "Secure by Default" architecture. All serialization operations are now protected by HMAC-SHA256 signatures. Furthermore, I have refactored the legacy schema architecture to prioritize JSON over binary formats, providing a safe and transparent migration path for verified data while strictly blocking malicious payloads.

Key Benefits:

  1. Criptographic Integrity: HMAC-SHA256 signatures ensure that only data generated by the SDK can be deserialized.
  2. Architectural Modernization: Passive JSON-based parsing replaces execution-capable formats like pickle for new and migrated data.
  3. Command-Line Safety: New source validation logic prevents the SDK from inadvertently connecting to malicious remote shares or external databases without explicit user authorization.

Technical Implementation Details

  • src/google/adk/utils/serialization_utils.py:
    • Created a core security utility for secure_dumps and secure_loads with HMAC-SHA256 verification.
  • src/google/adk/sessions/schemas/shared.py:
    • Implemented JsonEncodedType with a Hybrid Resolver that supports JSON for new data and secure HMAC-verified pickles for legacy artifacts.
  • src/google/adk/sessions/schemas/v0.py:
    • Refactored the StorageEvent table to use the new JsonEncodedType, eliminating the unauthenticated PickleType.
  • src/google/adk/sessions/migration/migration_runner.py:
    • Introduced a URL validation layer (_is_trusted_url) to detect and block remote IPs and Windows UNC paths.
    • Implemented a SecurityError to enforce safety boundaries during the migration process.
  • src/google/adk/sessions/migration/migrate_from_sqlalchemy_pickle.py:
    • Refactored the _row_to_event function to use secure_loads, sanitizing the primary deserialization sink in the migration loop.
  • src/google/adk/cli/cli_tools_click.py:
    • Added the --force-untrusted-source flag to the adk migrate session command, establishing a manual security gate for untrusted network operations.

Verification Performed

  • Reproduced Vulnerabilities:
    • Confirmed that unauthenticated pickle payloads could trigger code execution in the migration loop.
    • Confirmed that CLI tools would previously attempt to connect to remote SMB shares without warning.
  • Validated Fix:
    • Verified that the new gate rejects unsigned or tampered payloads with a SecurityError (5/5 unit tests passed).
    • Verified that the CLI successfully blocks external IP sources and UNC paths unless the force flag is used (4/4 integration tests passed).
    • Confirmed that data integrity is preserved during the hybrid JSON/Pickle transition.
  • Code Quality: Successfully ran isort and pyink on all modified files to comply with Google Style standards.

Checklist

  • Followed project guidelines for code hygiene.
  • Includes security-focused refactoring of critical pickle sinks.
  • Neutralizes RCE vectors in migration and persistent storage.
  • Verified backward compatibility for signed legacy data via hybrid decoding.

@rohityan rohityan self-assigned this Apr 16, 2026
@rohityan

Copy link
Copy Markdown
Collaborator

Hi @JoshuaProvoste , Thank you for your contribution! We appreciate you taking the time to submit this pull request.
Can you please fix the failing tests.

@rohityan rohityan added request clarification [Status] The maintainer need clarification or more information from the author core [Component] This issue is related to the core interface and implementation labels Apr 20, 2026
@JoshuaProvoste

JoshuaProvoste commented Apr 20, 2026 •

Copy link
Copy Markdown
Author

Hi @rohityan! Thank you for the feedback. I have resolved the failing tests and CI checks in this latest update.

Here is a summary of the fixes:

  • Resolved Test Failures: Fixed a NameError in v0.py caused by a missing import after refactoring, which was causing multiple session-related tests to fail.
  • Formatting & Linting: Ran isort and pyink across all modified files to comply with the project's style guide and resolved mypy type errors in cli_tools_click.py.
  • Infrastructure & CI: Fixed assertion failures in test_fast_api.py regarding cross-platform line endings and added the required from future import annotations to utilities.
  • Sync with Main: Synchronized the branch with the latest changes from upstream/main.

The security implementation for verified pickle deserialization is now fully validated by the test suite. Ready for another review!

@JoshuaProvoste

Copy link
Copy Markdown
Author

Hi @rohityan, I have resolved the merge conflicts in the GEPA samples (contributing/samples/gepa/experiment.py and contributing/samples/gepa/run_experiment.py) by synchronizing the branch with the latest changes from upstream/main. The branch is now up to date and ready for further processing. Thanks!

@GWeale

GWeale commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

the pickle rce is closed on main by a different design, _restricted_pickle refuses any global outside an allowlist derived from the EventActions model tree, so a hostile events.actions blob raises instead of executing. verified on 2.8.0, and the same payload still runs on 2.7.1. #5634 is closed and this branch conflicts with the rewritten schemas/v0.py. the one idea not carried over is the migration source-url check, and a small pr for just that would be welcome.

@GWeale GWeale closed this Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core [Component] This issue is related to the core interface and implementation request clarification [Status] The maintainer need clarification or more information from the author

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants