Skip to content

[Brook S2] Generated the Stream service and writer bindings with nexgen. - #372

Open
moetemp wants to merge 1 commit into
moe/brook/review/s1-stream-protosfrom
moe/brook/review/s2-stream-bindings
Open

moetemp wants to merge 1 commit into
moe/brook/review/s1-stream-protosfrom
moe/brook/review/s2-stream-bindings

Conversation

@moetemp

@moetemp moetemp commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

What was changed

This PR vendors the Stream service and writer WIT and generates their Python bindings with nexgen, into temporalio/common/_streams/system/ and writer_system/.

  • Both packages get nexgen's client target, a client extension per service. The service also gets the Workflow target. writer_system/ gets only the client target, since only Activities and clients make writer calls. S4 implements _create_nexus_client.
  • read-records has @nexus.poll. The service's iter_read_records and the stream handle's iter_read take the next request from next_read_request in temporalio.nexus.system._support. A read with no wait ends at an empty page that didn't move.
  • The WIT files are K10's byte for byte. The stream handle's methods are publish, read, close and describe, and each request's stream is required. A read item is the StreamItem union of StreamItemData, StreamItemSuperseded and StreamItemClose.
  • poe gen-protos runs the generator, and check-protos installs the pinned nexgen, so drift fails CI. The Workflow service bindings keep nexgen 0.2.8.
  • Proto enums are generated IntEnum classes, as on nexgen main (temporalio/nexgen#262), with A1's names (StreamProducerKind, StreamCloseStatus, StreamReadStart, ExecutionType). StreamRef.owner is an Execution model. The hand-written code in S3, S5, S7 and S8 passes them.
  • Fork-only pin. nexgen is built from moetemp/nexgen at d70f5f17, the fork chain below on top of temporalio/nexgen#308, #310, #311, #297, #296 and #312. Upstream installs a nexgen release by --version once the chain merges.

Depends on S1 (#371), K10 (moetemp/sdk-rust#220), temporalio/nexgen#308, #310, #311, #297, #296 and #312, and moetemp/nexgen#54, moetemp/nexgen#55, moetemp/nexgen#71, moetemp/nexgen#60, moetemp/nexgen#69, moetemp/nexgen#73 and moetemp/nexgen#75. No changelog fragment, since S9 has the Streams entry.
Part of the Brook review series.

Why?

The Streams API calls the Stream service through generated bindings, as the SDK calls the Workflow service System Nexus operations. They're generated, so they get their own PR, before any hand-written Stream code.

Checklist

  1. How was this tested:
    • gen_stream_service.py --install-nexgen and poe gen-protos-docker leave no diff. poe lint, poe gen-docs and the binding tests pass.
    • Fork CI passes, check-protos included, except the arm jobs, since the fork has no arm runners.

@moetemp moetemp added the skip-changelog Changelog entry rides another PR label Oct 25, 2026
@moetemp
moetemp force-pushed the moe/brook/review/s2-stream-bindings branch 3 times, most recently from 3925df2 to f2a5ec4 Compare October 26, 2026 08:44
@moetemp
moetemp force-pushed the moe/brook/review/s1-stream-protos branch from 235fde0 to 3b5246f Compare October 26, 2026 08:44
@moetemp
moetemp force-pushed the moe/brook/review/s2-stream-bindings branch from f2a5ec4 to ba649e0 Compare October 27, 2026 09:07
@moetemp
moetemp force-pushed the moe/brook/review/s1-stream-protos branch from 3b5246f to 92725de Compare October 27, 2026 09:07
@moetemp
moetemp force-pushed the moe/brook/review/s2-stream-bindings branch from ba649e0 to e1864c4 Compare October 27, 2026 09:17
No nexgen release has the long-poll, client and client-only features, so
the generator installs nexgen at a fork revision. `poe gen-protos` runs
it, so `check-protos` checks the bindings for drift.
@moetemp
moetemp force-pushed the moe/brook/review/s1-stream-protos branch from 92725de to 7afb312 Compare October 28, 2026 01:33
@moetemp
moetemp force-pushed the moe/brook/review/s2-stream-bindings branch from e1864c4 to 50537f7 Compare October 28, 2026 01:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog Changelog entry rides another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant