fix: Send request queue write fields under the names the API declares - #1026
Merged
Conversation
`RequestDraft` declared only id/unique_key/url/method and leaned on `extra='allow'` for the rest, but `alias_generator=to_camel` never touches extras. So `user_data`, `no_retry`, `headers`, `payload`, `retry_count`, and `handled_at` reached the API snake_cased on `add_request` and `batch_add_requests`, which the API ignores silently. `update_request` was unaffected - it uses `Request`, which declares the full shape. The spec declares those request bodies as `RequestBase` and uses `RequestDraft` only for `unprocessedRequests` in responses, so the codegen postprocessor now reparents `RequestDraft` onto `RequestBase`. `RequestDraftDict.unique_key` and `.url` are no longer statically required: PEP 589 forbids a TypedDict subclass from redeclaring a key of its base. Pydantic still enforces both at runtime, before any HTTP call.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1026 +/- ##
==========================================
+ Coverage 94.97% 95.21% +0.24%
==========================================
Files 58 58
Lines 5450 5436 -14
==========================================
Hits 5176 5176
+ Misses 274 260 -14
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
vdusek
added a commit
to apify/apify-docs
that referenced
this pull request
Aug 24, 2026
…bundling (#2774) `RequestWithoutId` was a bare `$ref` with sibling keys, so the bundler inlined it: the published `openapi.json` has no such component, the add-request / batch-add-requests / update-request bodies point straight at `RequestBase`, and `required: [uniqueKey, url]` disappears. Generated clients therefore get no schema for those bodies — `apify-client-python` fell back to a response shape and sent `userData` and friends snake_cased, which the API rejects with HTTP 400 (apify/apify-client-python#1026). Wrapping `RequestWithoutId` in a single-member `allOf` keeps it a named component through bundling. `required: [uniqueKey, url]` moves to `RequestBase`, and `Request` drops the now-redundant entries — net constraints unchanged. `redocly lint` passes, and the bundle keeps the component with all 11 properties and the required constraint. *✍️ Drafted by Claude Code*
…g hack apify-docs#2774 fixed the OpenAPI bundler to preserve RequestWithoutId as its own component with required unique_key/url, instead of collapsing add-request and batch-add-requests bodies onto RequestBase. The properly-modeled schema is now generated correctly on its own, so the BASE_CLASS_FIXES reparenting of RequestDraft is no longer needed - RequestDraft reverts to its natural shape, used only for the unprocessedRequests response.
vdusek
marked this pull request as ready for review
August 24, 2026 11:31
Contributor
Author
|
Btw. this is not an issue in the current SDK, because it uses the Crawlee's |
Pijukatel
approved these changes
Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
add_requestandbatch_add_requestssent request fields snake_cased. The API declares its write bodies withadditionalProperties: false, so it rejected the whole write with HTTP 400 — a request carryinguser_data,no_retry,retry_count,loaded_url,error_messages, orhandled_atcould not be added at all. The identical dict passed toupdate_requestworked:The client was using
RequestDraft(the spec's response-only schema forunprocessedRequests, which declares onlyid/unique_key/url/methodand leaves the rest toextra='allow') as its input model. The spec actually declares both add-request bodies asRequestWithoutId, now correctly generated as its own model since apify-docs#2774 fixed the spec bundler dropping its identity — no custom postprocessing needed. Upstream spec fix (apify-docs#2774) is closed.Two things to know:
retry_count='abc'or a naivehandled_atnow raiseValidationErrorinstead of a 400 from the API.update_request's timestamp format changes (fa77d88):mode='json'emits ISO 8601 where python mode emitted2019-06-16 10:23:31.607000+00:00. The API accepts both, so this is spec fidelity, not a fix — its own commit, revertible alone.Integration tests round-trip every field through both add paths against the live API; they fail on master with
InvalidRequestError.✍️ Drafted by Claude Code