Skip to content

[Brook] Added Stream definitions and the Stream handle form. - #342

Closed
moetemp wants to merge 10 commits into
moe/brook/v4/p19-crate-pinfrom
moe/brook/v4/p20-definitions-and-handles
Closed

moetemp wants to merge 10 commits into
moe/brook/v4/p19-crate-pinfrom
moe/brook/v4/p20-definitions-and-handles

Conversation

@moetemp

@moetemp moetemp commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

What changed?

This PR adds Stream definitions and the shared form of the Stream handles.

@streams.defn(name="events") marks a class whose streams.topic(Turn) attributes are its Topics. The name defaults to the class name, and a Topic name defaults to the attribute name. StreamHandle[E] is the base of the handle of each context. It carries the Stream id, the owning Workflow, the first run id, the backend name and the definition, and _topic_spec gives each Topic its name and type: a Topic of the definition keeps its type, and a name outside the definition needs type=.

A handle serializes to the proto JSON of StreamRef (id, workflowId, firstExecutionRunId, backend). On decode, the parameter's type hint picks the handle class and the definition, also for a generic alias such as ActivityStream[Events]. StreamHandle.from_handle gives the same Stream as another handle class, so a Workflow gives an Activity's input the type that the Activity declares.

A hint with Any, such as ActivityStream[Any], decodes as a handle with no definition. From Python 3.11 Any is a class, so it needs its own check.

A handle compares, hashes and prints from its stored fields, so a client handle whose id needs a Describe call still works in a set or a log line.

A first commit lets the JSON payload converter convert transfer type values inside other values, such as a dataclass field. Before, only a top-level value took its transfer type.

Why?

S66 and S67: a definition gives each Topic its type once, and each context has its own handle type, while a handle stays data that passes between Workflows, Activities and clients.

How did you test it?

pytest tests/contrib/streams/test_handles.py tests/test_converter.py and the rest of tests/contrib/streams. The tests cover the definition rules, the proto JSON form, decode by hint with and without a definition, a handle in a dataclass field and in a pydantic model, and nested transfer types in the converter. pyright, mypy, basedpyright, pydocstyle, ruff.

  • Unit Tests
  • Staging
  • End to End Tests

@moetemp

moetemp commented Oct 25, 2026

Copy link
Copy Markdown
Owner Author

Closing as stale. This layer is folded into the v5 tip, #365, and the upstream review uses a fresh series instead. Plan: https://www.notion.so/4048fc567738814990b7c796ac28fe3f

@moetemp moetemp closed this Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant