Skip to content

feat: Add HTTPX-based HTTP client - #1004

Open
vdusek wants to merge 9 commits into
masterfrom
add-httpx-client
Open

feat: Add HTTPX-based HTTP client#1004
vdusek wants to merge 9 commits into
masterfrom
add-httpx-client

Conversation

@vdusek

@vdusek vdusek commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Adds HttpxHttpClient / HttpxHttpClientAsync as built-in alternatives to the default Impit transport, behind an optional extra: pip install "apify-client[httpx]". Impit stays the default.

The transport is HTTPX2, Pydantic's maintained continuation of HTTPX (stable HTTPX has been parked at 0.28.1 since Dec 2024). The module imports it as httpx, so the client keeps the plain Httpx* naming. It verifies TLS via the OS trust store (truststore) rather than certifi.

The adapters plug into the shared request pipeline from #1022 and implement only the transport hooks (send_request, is_retryable_transport_error, is_timeout_error, close() / aclose()), applying the same retry policy as Impit: every error from the library's own hierarchy is transient except the permanently-failing types. A response event hook keeps the shared cookie jar from leaking server cookies into later API requests. Unit tests run the built-in client matrix (Impit + HTTPX) through the parametrized fixtures.

Note that HTTPX applies timeouts per socket operation, not to the request as a whole, so a slowly-arriving body can outlast the requested timeout. Impit enforces a whole-request deadline. Both class docstrings record this.

Stack: #1022 (merged) → #1004 (this)#1013 (docs — its apify-client[httpx] install lines now match; only the conda-forge::httpx line still points at the stale package instead of httpx2).

✍️ Drafted by Claude Code

@vdusek vdusek self-assigned this Aug 11, 2026
@vdusek vdusek added adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. labels Aug 11, 2026
@github-actions github-actions Bot added this to the 147th sprint - Tooling team milestone Aug 11, 2026
@github-actions github-actions Bot added the tested Temporary label used only programatically for some analytics. label Aug 11, 2026
@codecov

codecov Bot commented Aug 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.43750% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 95.25%. Comparing base (514848f) to head (c8b3628).

Files with missing lines Patch % Lines
src/apify_client/http_clients/__init__.py 87.50% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1004      +/-   ##
==========================================
+ Coverage   95.23%   95.25%   +0.01%     
==========================================
  Files          58       59       +1     
  Lines        5436     5499      +63     
==========================================
+ Hits         5177     5238      +61     
- Misses        259      261       +2     
Flag Coverage Δ
integration 92.19% <90.62%> (+0.12%) ⬆️
unit 87.34% <98.43%> (+0.12%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vdusek vdusek changed the title feat: add httpx http client [test] feat: Add HTTPX-based HTTP client Aug 19, 2026
@vdusek
vdusek changed the base branch from master to unify-http-request-pipeline August 19, 2026 06:56
@vdusek
vdusek marked this pull request as ready for review August 19, 2026 07:07
@vdusek
vdusek marked this pull request as draft August 19, 2026 07:46
Base automatically changed from unify-http-request-pipeline to master August 19, 2026 12:43
@vdusek vdusek changed the title feat: Add HTTPX-based HTTP client feat: Add HTTPX2-based HTTP client Aug 21, 2026
@vdusek vdusek changed the title feat: Add HTTPX2-based HTTP client feat: Add HTTPX-based HTTP client Aug 21, 2026
@vdusek
vdusek requested a review from Pijukatel August 25, 2026 13:53
@vdusek
vdusek marked this pull request as ready for review August 25, 2026 13:53

@Pijukatel Pijukatel left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two minor comments

) as _httpx_import:
from apify_client.http_clients._httpx import HttpxHttpClient, HttpxHttpClientAsync

if _httpx_import.available:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe share the default and only extend. Less duplication and it is clear from first look what is the optional

__all__ = ...

if _httpx_import.available:
    __all__.extend(...)

def apify_client(api_token: str) -> ApifyClient:
"""Sync Apify client instance."""
api_url = os.getenv(API_URL_ENV_VAR) or DEFAULT_API_URL
return ApifyClient(api_token, api_url=api_url)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small concern. The default most common constructor will no longer be well covered by the integration tests.

ApifyClient(api_token, api_url=api_url)
ApifyClientAsync(api_token, api_url=api_url)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants