[ParidelPooya]

Merge dependency: merge aws/aws-durable-execution-sdk-js#1088 first. This PR removes the worker's handling of an oversized body, so every SDK must reject an oversized input before it sends it.

Summary

microvm and vm.invoke reject a job input whose JSON is over 6 MiB minus 8 KiB, with a ValueError, before any durable operation starts. The worker then no longer needs to read and drop an oversized body, so that code is removed.

Why

1. The worker refuses a job body over 6 MiB.
2. The operation did not check the size.
3. So an oversized job launched a MicroVM first. The worker then answered 400.
4. To make the client read that 400, and not get a TCP reset, the worker handled Expect: 100-continue and read and dropped up to 24 MiB for 10 seconds.
5. A reset makes the request step retry in both retry tiers, for minutes, while the MicroVM is billed.
6. So the operation now checks the size at the call. The SDK never sends an oversized body, so the worker keeps only the plain 400.

This is item D5 of a review of the MicroVM safeguards. The JS SDK gets the same rule and message in aws/aws-durable-execution-sdk-js#1088.

Changes

• Extras:
  • MAX_JOB_INPUT_BYTES is 6 MiB minus 8 KiB, exported next to MAX_RUN_HOOK_PAYLOAD_LENGTH.
  • _validate_input encodes the input once with dumps, the encoder of the job body, and checks its UTF-8 length.
  • The message: MicroVM "<name>": the job input is <n> bytes as JSON. The limit is <limit> bytes, because the worker accepts a job body of at most 6 MiB. Pass larger data by reference, for example as an S3 key.
• Worker:
  • Removed handle_expect_100, discard_unread_body, and their constants. worker.py loses 95 lines net.
  • A body over the 64 KiB hook limit or the 6 MiB job limit still gets 400 without being read, with Connection: close.
  • A sender outside the SDK that keeps writing an oversized body can now get a reset instead of the 400. With Expect: 100-continue, it gets 100 Continue, then the 400. No path hangs, and no path runs the job.
• READMEs: extras "Defaults and limits", "Errors" and Best practices "Job inputs"; worker "What the worker does".

Upgrade note

An execution that started on an earlier build, with an input in the last 8 KiB below the old limit, fails at its next replay. The packages are unpublished, and the best-practices docs say to let executions from an earlier build end before deploying a new one.

Testing

• The fixed fields of the body were measured against JobRequest.to_json(). Within the API models (a MicroVM ID of at most 256 characters, a callback ID of at most 1,024 base64 characters), they take at most 2,692 bytes. With every character of both IDs escaped, they take 7,812 bytes, still under 8 KiB.
• Extras: inputs of 1-, 2-, 3- and 4-byte characters, and a lone surrogate, are accepted at the limit and rejected one byte over, with no durable operation. vm.invoke over the limit creates no job operation. microvm at the limit runs through the real worker.
• Worker: an oversized body, with and without Expect, gets a 400 or a reset, never a hang or a 202.
• hatch run dev-extras:cov: 386 tests, 100 %. hatch run dev-microvm-worker:cov: 449 tests, worker.py 100 %. Typecheck and hatch fmt --check pass. A stress run with -n 8 passed 3 times on macOS and 3 times on Linux.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[yaythomas]

This change assumes that every supported SDK rejects an oversized input before sending it. JS #1088 does not yet guarantee that. It validates the input as a root JSON value but sends it as a nested value. A key-sensitive toJSON can therefore pass that check and still produce an oversized request. Can this retain the existing oversized-request handling until #1088 covers that case, or record #1088 as a merge dependency and wait for the corrected change to merge first?

[ParidelPooya]

Recorded aws/aws-durable-execution-sdk-js#1088 as a merge dependency in the description. #1088 now measures the nested form (90122f96) and covers the key-sensitive toJSON case. The Python check needs no change: JobRequest.to_json() embeds the same dumps text that _validate_input measures, and dumps has no key-dependent hook. I will keep this PR open until #1088 merges.

[yaythomas]