Repository navigation
fix(sessions): recover writer authority and verify publication joins - #4388
Conversation
|
ScriptedAlchemy
left a comment
There was a problem hiding this comment.
Pushed 6b9c4e2d96cea73b6d553b4e1fbd347396805755 to address the two concrete recovery blockers.
Both persisted-checkpoint reads use the async exact-SQL path. A refused physical attachment is drained, closed, and joined on the existing blocking executor before retry; close failures propagate.
For graph-first stores, a live graph publication's immutable binding cannot be replaced safely while owners or operation leases retain it. Recovery now retains the persisted epoch floor and fails once with typed WriterAuthorityBehindPersisted, preserving existing leases. It does not perform the same doomed physical open twice. After canonical owner/client release and graph close, reopening allocates a shared binding above the checkpoint and can commit. That follows #4372's explicit “adopts the persisted epoch, or fails once with a typed error” contract.
Added a real GraphDbRegistry owner/client + SQLite-checkpoint regression covering the refusal, continued validity of old leases, release/close, and successful reopen/write. Existing reopen coverage also checks native-worker wait telemetry. Pinned rustfmt and diff checks pass; native tests remain for CI.
The PR description now names //crates/tracedecay-cli:core_cli_suite for the isolated daemon journey. The previous blocked command selected a binary target, so it was not evidence that this journey had run.
There was a problem hiding this comment.
Devin Review found 2 new potential issues.
3 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)
Pull request was converted to draft
# Conflicts: # crates/tracedecay/tests/runtime_acceptance_suite/host_event_fixture_test.rs
Losing the daemon authority record could reopen a store with a writer epoch below its durable checkpoint, leaving observation admission retrying generic infrastructure failures. Read the exact shard/incarnation checkpoint before publication, retain its epoch floor, and retire the refused physical attachment on the blocking executor. Retry once when no live graph owner retains the binding; otherwise return typed
WriterAuthorityBehindPersistedwhile preserving live leases. Ledger fences remain typed and rejected writes roll back.Named Hermes inline transcript callbacks join their own pinned source frontier. Success requires an active temporal generation, an applied relation receipt, and its expected graph watermark. Pending resets, retired generations, and future frontiers cannot supply publication proof. Another session's failure cannot reject a proven publication; the requested session's failed or unproven projection remains unavailable. Source sweeps and unproven provider identities retain the aggregate publication contract.
Pending joins finish advancing bounded discovery pages before waiting for derived-summary admission or the next historical window. This fixes the complete-receipt/zero-backlog batch-join stall without changing deadlines or work budgets. Preserve the exact registry idle-completion assertions through rekey and retirement.
Validation on integration
c990a339252ee6c3e132b0e43cc713f85d332e97:ca677ea030, retaining all peer changes. The sole manual conflict was in the host fixture: retain master's stricter Pi condition, the PR's baseline assertion diagnostic, and canonical typed Kiro/Pi zero-finding validation. There was no production-code conflict.528fe953-8f70-49e0-9464-1dc40a86b7a6). The root Bazel formatting check also passed.cae2bfa8-51af-4e44-a432-3469b825631e). This includes waiting-join refusal for the superseded candidate and successful new joins after automatic reset recovery.Historical evidence:
Tested merge
0fd2e7d8c662a3ccb42f23252d279c84d38adb65integrates masterd7e9afee0eed9f2a199aed2c4cf9ec930c620033, preserving reset rediscovery alongside per-session publication proof and failure isolation. All 61 focused tests passed uncached remotely on this tree: scheduler 48 (c89517d4-a154-4829-87d5-0034f910dcf5), activation/reset 7 (ad811eaa-b12c-4c3d-982b-59eacff40954), and MCP live-transcript joins 6 (8531fe87-9743-45f9-b6b0-f2a8fe366360). Nonzero libtest counts were confirmed; Rustc/TestRunner were remote with local fallback disabled and priority 10. Formatting and whitespace passed.Earlier integration also passed the real daemon writer-authority reopen and host fixtures; those runs preceded this master merge. Subsequent integration
c43f845c83770f2c3125180df488cc62052cfd37added only eight.githubfiles for fleet routing and Tailscale network-environment preservation; Rust source is unchanged from the 61-test tree. The draft CI run 39853962118 skipped Linux tests, Clippy, feature gates, and the shipped CLI, so its green rollup is not full validation. This PR is now ready for review; actual Linux execution and aggregate, Clippy, shipped CLI, and applicable Windows coverage must pass before merge.Previous head
182ac69041f702222e5442e1840b75617336dfe7also preserves peer test commite3a679a806: the reset journey gates historical work, verifies waiting named and aggregate joins refuse superseded publication, and verifies new joins observe the automatically rebuilt publication. Production source is unchanged. That strengthened reset journey has now passed on the newer integration above. Earlier CI run 39864396638 does not establish readiness of the updated head.Fixes #4372.
Resource-refusal diagnostics at
33a9bde3418fec56d2987460e148fe6a1acb0e3f:dd84675530adds only synchronization to the profile wake-coalescing test. Fleet run 39898392822 passed 169 of 170 targets; its sole failure was the real Codex PostCompact journey receivingtranscript_background_resource_unavailable. The specific resource was erased by the hook error mapper.dd84675530, the exact journey passed (1 test, 3.31s; invocation3de4c5c8-c471-42da-87d1-4df409be4a51) and the full transcript-ingest target passed all 207 tests (59.44s; invocationcc545d8e-92ed-4188-8fe5-106fe1057325). The original refusal was not reproduced; its resource and CI pressure state remain unknown.