obsidian/wiki/shared-patterns/httpx-async-client.md at main

Vadym/obsidian

Fork 0

Vadym Samoilenko f51f38b858 vault backup: 2026-05-15 10:37:25

2026-05-15 10:37:25 +01:00

5.9 KiB

Raw Permalink Blame History

title

aliases

Key Takeaways

async with httpx.AsyncClient() as client: is the dominant pattern across all Oliver projects — a new client is created per call, not shared at module level.
resp.raise_for_status() is called unconditionally after every request; error handling is done at the caller.
timeout= is passed per-call when the endpoint is known to be slow (Mailgun: 30 s, Adobe DAM OAuth: 15 s, Adobe DAM search: 15 s); internal/fast APIs omit it entirely.
follow_redirects=True is used explicitly only when required (ADO attachment downloads); omitted everywhere else.
Auth is built into headers at client construction time, not passed to each call site.

Standard Pattern

Pattern seen in cc-dashboard ADO client and DevOps_Click_UP_sync ADO client (identical structure):

import httpx

class SomeServiceClient:
    def __init__(self):
        # Build auth headers once; reuse per request
        self.headers = {
            "Authorization": f"Bearer {settings.api_token}",
            "Content-Type": "application/json",
        }

    async def get_resource(self, resource_id: str) -> dict:
        url = f"{BASE_URL}/resources/{resource_id}"
        async with httpx.AsyncClient() as client:
            resp = await client.get(url, headers=self.headers)
            resp.raise_for_status()
            return resp.json()

    async def post_resource(self, body: dict) -> dict:
        url = f"{BASE_URL}/resources"
        async with httpx.AsyncClient() as client:
            resp = await client.post(url, json=body, headers=self.headers)
            resp.raise_for_status()
            return resp.json()

    async def patch_resource(self, resource_id: str, patch: list[dict]) -> dict:
        url = f"{BASE_URL}/resources/{resource_id}"
        # Override Content-Type for patch+json (ADO pattern)
        headers = {**self.headers, "Content-Type": "application/json-patch+json"}
        async with httpx.AsyncClient() as client:
            resp = await client.patch(url, json=patch, headers=headers)
            resp.raise_for_status()
            return resp.json()

Binary upload variant (from ADO attachment upload):

async def upload_bytes(self, filename: str, content: bytes) -> dict:
    url = f"{BASE_URL}/attachments?fileName={filename}"
    headers = {
        "Authorization": self.headers["Authorization"],
        "Content-Type": "application/octet-stream",
    }
    async with httpx.AsyncClient() as client:
        resp = await client.post(url, content=content, headers=headers)
        resp.raise_for_status()
        return resp.json()

Timeout Configuration

Real values found in Oliver projects:

Use case	Timeout	Source
Mailgun email send	`timeout=30`	`cc-dashboard/src/services/mailgun.py`
Adobe DAM OAuth token	`timeout=15`	`Barclays-banner-builder/backend/app/services/adobe_dam_client.py`
Adobe DAM search / get asset	`timeout=15`	same
ADO API calls	none set	`cc-dashboard`, `DevOps_Click_UP_sync`
ADO attachment download	none set	`DevOps_Click_UP_sync/src/clients/ado.py`

No projects define a module-level default timeout. For new integrations: use timeout=15 for third-party APIs, omit for internal services.

Error Handling

All projects rely on raise_for_status() to propagate httpx.HTTPStatusError. Callers catch at the route/task level. No retry logic is implemented in any current Oliver project.

# What raise_for_status() raises:
import httpx

try:
    async with httpx.AsyncClient() as client:
        resp = await client.get(url, headers=headers)
        resp.raise_for_status()
        return resp.json()
except httpx.HTTPStatusError as e:
    # e.response.status_code, e.response.text available
    raise
except httpx.TimeoutException:
    # connect / read / write / pool timeout
    raise
except httpx.RequestError:
    # network-level: DNS, connection refused, etc.
    raise

mailgun.py is the only place that catches the response status manually (resp.status_code == 200) instead of raise_for_status() — this is the Mailgun exception because a non-200 is a soft failure there.

Gotchas

1. New client per call — no connection pooling. Every async with httpx.AsyncClient() opens a new TCP connection. For high-throughput services (e.g., batch ADO fetches) this is a latency penalty. The fix is to instantiate httpx.AsyncClient at the class level and close it in a __aenter__/__aexit__ or lifespan hook. No Oliver project does this yet.

2. Content-Type must be overridden for JSON-patch. ADO's PATCH endpoints require application/json-patch+json. Using json= kwarg sets Content-Type: application/json automatically — you must spread and override the header dict manually ({**self.headers, "Content-Type": "application/json-patch+json"}). Forgetting this returns a 400 from ADO with no useful error body.

3. data= vs json= for form-encoded payloads. Mailgun's API expects multipart/form-data / application/x-www-form-urlencoded, so it uses data= not json=. Using json= sends JSON body and Mailgun silently ignores it, returning 200 with no email sent.

4. follow_redirects defaults to False. ADO attachment download URLs redirect. Without httpx.AsyncClient(follow_redirects=True) the client returns a 302 and .content is empty. Only set this flag when you know the endpoint redirects.

5.9 KiB Raw Permalink Blame History