Timeouts and Deadlines in Production: Time Budgets, Propagation Across Services and Retries That Don't Multiply Load

A practical guide to making sure no request waits forever: why a per-socket timeout is not a total time limit, the difference between a timeout and a deadline, how to propagate a time budget across services (asyncio, httpx, FastAPI, Go and gRPC), how to split it across layers, how retries fit in without multiplying load, PostgreSQL and load balancer timeouts, common mistakes and a production checklist. With code in Python, Go, SQL and YAML. Expanded edition of October 10, 2026: Little's law applied to pools and timeouts with a script that computes when a pool saturates, structured cancellation with asyncio.timeout_at, TaskGroup and except*, fan-out with degradation of optional dependencies, hedged requests with a budget, HTTP client defaults and AbortSignal in Node, gRPC deadlines, codes and retry configuration, application servers (gunicorn, Go, Node) and which timeouts do not cancel the work, the full map of timers in nginx, Envoy, Istio and ALB, visibility timeouts and heartbeats in queues, PostgreSQL 17 transaction_timeout and budget-derived limits, admission control, a data-driven method for choosing the numbers, Toxiproxy tests, a worked case, retry budgets, traces with remaining time, FAQ and glossary.

42 minutes read

Josue Puig

2026-10-10

1 views

Verificando acceso...

Loading comments...

Related Resources

Guía

PREMIUM

API Versioning and Contract Testing: Safe Changes, OpenAPI in CI, Pact, and Retiring Endpoints with Sunset

A practical guide to changing your APIs without breaking the people who consume them: why compatibility rules invert between the request and the response —and why widening a returned enum breaks clients even though you are "only adding"—, the tolerant reader pattern in Pydantic with an escape hatch and a metric, OpenAPI generated from the code and diffed on every pull request with oasdiff (including the git diff --exit-code step without which the whole check is theatre), consumer-driven contract testing with Pact: type matchers instead of literal values, well-designed provider states, version selection with deployed_or_released, and the gate that actually makes it safe, can-i-deploy paired with record-deployment. It also covers rolling this out without stopping the factory using pending and WIP pacts, bi-directional contracts when the provider is a third party, the three versioning strategies with their real operational costs, why versioning the whole API for a single endpoint guarantees nobody migrates, retiring versions with the Deprecation (RFC 9745) and Sunset (RFC 8594) headers plus the migration Link, the per-consumer metric without which no sunset date is ever met, brownouts returning 410 Gone before the final shutdown, and the BACKWARD, FORWARD and FULL compatibility modes for event schemas. With production-ready code in Python, YAML, Bash and PromQL, eight recurring mistakes, a production checklist, FAQ and glossary. It also extends the contract beyond the happy path: errors with RFC 9457 (problem+json), cursor pagination and defaults as part of the contract, the expand/contract pattern for renaming a field across database and API with no maintenance window, governance with Spectral, buf breaking for gRPC and Protobuf, GraphQL schema evolution with @deprecated and real per-field usage, and semantic versioning of generated SDKs. It also covers the contracts that never show up in the schema and break just as hard: outbound webhooks with the version pinned on the subscription, HMAC signing with a timestamp window and key rotation, Idempotency-Key for safe POST retries, the RateLimit and RateLimit-Policy headers —and why lowering a limit is a breaking change—, and OAuth scopes, the blind spot no OpenAPI diff will ever catch.

Guía

PREMIUM

Architecture Decision Records (ADRs): Document the Why Behind Your Technical Decisions

A practical guide to recording architecture decisions with ADRs: what they are and are not, when to write one, a template covering context, alternatives and consequences, a lifecycle with statuses and supersession, and automation with a creation script and a CI validator. With Bash, Python and YAML code, common mistakes and a checklist. Expanded edition of October 2, 2026: writing context with evidence, alternatives without straw men and a weighted decision matrix with sensitivity analysis, Nygard, MADR 4 and Y-statement templates, the advice process with roles, CODEOWNERS and a PR template, the boundary between ADRs, RFCs, design docs and runbooks, ADRs as executable architecture with import-linter and fitness tests, generated index and supersession graph, scheduled reviews with Review-by, two complete annotated ADRs, ADRs at scale in monorepos and multi-repo setups, adoption through retroactive ADRs, ADRs as context for coding assistants, health metrics, advanced mistakes, FAQ and glossary.

Guía

PREMIUM

asyncio in Production: Never Block the Event Loop — TaskGroups, Cancellation and Bounded Concurrency

A practical asyncio guide for Python services in production: why blocking the event loop degrades the whole process without raising a single exception, how to catch it by measuring loop lag and with Python 3.14 introspection, structured concurrency with TaskGroup and handling ExceptionGroup via except*, the task the garbage collector makes vanish, timeouts with a deadline budget propagated across services, correct cancellation with cleanup and shield, bounded concurrency with semaphores and backpressured queues, synchronization primitives, and what changes with eager tasks, python -m asyncio pstree and free-threading. With production-ready code and a deployment checklist.