Skip to content

[Brook] Ended reads of a StreamRef with the owning run. - #337

Closed
moetemp wants to merge 3 commits into
moe/brook/v3/p14-memory-pluginfrom
moe/brook/v3/p15-ref-owner
Closed

moetemp wants to merge 3 commits into
moe/brook/v3/p14-memory-pluginfrom
moe/brook/v3/p15-ref-owner

Conversation

@moetemp

@moetemp moetemp commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

What changed?

This PR adds end_with_run to streams.for_ref(client, ref). StreamRef now carries its owning Workflow, as workflow_id and first_execution_run_id (S30, crate s8 813f9d12, proto fields 2 and 3, also in the WIT copy and the bindings). workflow_stream(), create_workflow_stream() and for_handle() set them. create_stream() and Activity refs leave them empty. for_ref(end_with_run=True) builds the Workflow handle from them and ends the read as for_handle does, also during a wait for the create. It raises ValueError for a ref with no owner. Old refs with only id still decode. On the payload converter, StreamRef uses the proto JSON names (id, workflowId, firstExecutionRunId), so a ref that Python wrote reads in TypeScript and .NET. Snake case keys still decode for one release. The plugin also carries the owner on every ref that it sends to the crate.

A read that ends with the run now polls in steps of 2 s and checks the run in between. Before, it checked the run only after a long poll of up to 5 minutes with no Records. The idle time still counts from the last Record, across the steps.

Why?

The harness found that a client with only a StreamRef could not end its read when the owning run ended.

How did you test it?

pytest tests/contrib/streams on the memory Store and on Redis. The new tests cover:

  • a read of an open Stream that ends with the run;
  • a wait for a Stream that the finished run never created;
  • an idle end across the short polls;
  • the ValueError for a ref with no owner;
  • an old ref with only id, and an old ref with snake case keys;
  • the ref's JSON against the proto JSON of temporal.api.streamservice.v1.StreamRef;
  • the owner on every ref in the plugin's BufferWorkflowCalls requests.

Full poe lint and the full suite pass at this layer, except test_envconfig, which reads a config file on this host.

  • Unit Tests
  • Staging
  • End to End Tests

A client that holds only a StreamRef could not end its read when the run ended. The ref now carries its owning Workflow (crate s8), and for_ref(end_with_run=True) uses it as for_handle does. A read that ends with the run now polls in short steps and counts idle time across them.
The crate gets Workflow refs from lang only, and the plugin dropped the owner when it encoded a call or built a run-end close.
A ref that Python wrote must read in TypeScript and .NET, so its JSON keys are the proto JSON names. Snake case keys still decode for one release.
@moetemp

moetemp commented Oct 25, 2026

Copy link
Copy Markdown
Owner Author

Closing as stale. This layer is folded into the v5 tip, #365, and the upstream review uses a fresh series instead. Plan: https://www.notion.so/4048fc567738814990b7c796ac28fe3f

@moetemp moetemp closed this Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant