Repository navigation
Conversation
…k/v4/p21-workflow-module
…k/v4/p21-workflow-module
…k/v4/p21-workflow-module
…k/v4/p21-workflow-module
…k/v4/p21-workflow-module
…k/v4/p21-workflow-module
…k/v4/p21-workflow-module
…k/v4/p21-workflow-module
Owner
Author
|
Closing as stale. This layer is folded into the v5 tip, #365, and the upstream review uses a fresh series instead. Plan: https://www.notion.so/4048fc567738814990b7c796ac28fe3f |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed?
This PR adds the Workflow module
temporalio.contrib.streams.workflow, withcreate_stream,stream_handle,WorkflowStreamandWorkflowTopic.await create_stream(Events, *, name=None, retention=None, auto_close=None, close_with_run=True, backend=None)creates a Stream with the id<workflow id>:<first run id>:<name>.stream_handle(Events, *, name=None)gives the same Stream again with no call.await topic.publish(*values)puts the Records in the run's buffer and returns None.await stream.close(result=, failure=)goes in the buffer too, andasync withcloses on exit with the exception as the failure, with the existing cancel andBaseExceptionrules.WorkflowTopic.publish_nowait(*values)andWorkflowStream.close_nowait(*, result=None, failure=None)do the same with no await, for sync Workflow code such as@workflow.initand sync signal and update handlers, asasyncio.Queue.put_nowaitdoes forput(S80). The awaited calls use them.create_stream_nowaittakes the options ofcreate_streamwith no await, for a create in@workflow.init(S83), andcreate_streamcalls it.Workflow code only gets errors that replay reproduces: a Stream that this run closed, a Topic outside the definition, and read-only code. A Worker with no Stream plugin, or without the backend that the Stream names, raises a
RuntimeError, so the Workflow Task fails and a Worker with the right config takes the run on.close_with_run=Falseon a Stream that already closes with the chain raisesValueError.close_with_runcloses the Stream with the result or failure of the run chain's last run, through the existing close-with-run machinery. Continue-as-new passes the ids of the open Streams to close with the run chain to the next run in a header, so the last run closes them even when it does not use them. Each plugin takesname=for its backend (redisandmemoryby default) anddefault=to mark the client's default backend. It gives the name to the crate's config and the Workflow interceptor. The end of a retried chain now counts fromworkflow.Info.execution_expiration_time, a new field fromInitializeWorkflow, since the execution timeout covers the whole chain.Why?
S65, S68, S69, S72, S73, S76 and S77: Workflow code creates, writes and closes named Streams with awaited calls, and there is no default Stream.
How did you test it?
Live tests in
tests/contrib/streams/test_workflow_module.pyon the memory Store and on a scratchredis-server, against the stock dev server: create, publish of several values and close with the run's result,async withwith and without a failure, continue-as-new with the handle in the input, the errors in Workflow code and in a query, Worker config errors that fail the Workflow Task, a failed run,auto_close, a cron chain, sync init, signal and update handlers that publish and close with no await, and a create and publish in@workflow.init.test_chaincovers the expiry of a later run of a chain. The fixtures moved toconftest.py.pyright,mypy,basedpyright,pydocstyle,ruff.