Distributed Locks and Leader Election: Fencing Tokens, Kubernetes Leases, PostgreSQL Advisory Locks, and Why Redlock Isn't Enough for Correctness

A practical guide to the mechanism almost everyone gets wrong: why every TTL-based lock has a window where two processes both believe they own it (GC pauses, cgroup throttling, network partitions), the question to answer before picking a tool —are you locking for efficiency or for correctness?—, a Redis lock done properly with SET NX PX and a Lua compare-and-delete release, the Redlock controversy and what it actually buys you, PostgreSQL transaction-level advisory locks (and why session-level ones break under PgBouncer transaction pooling), queues with FOR UPDATE SKIP LOCKED, leader election with Kubernetes Leases and etcd —including the fact that client-go does not fence and the leaseTransitions field that does work as a token—, end-to-end fencing with UPDATE ... WHERE fence_token < $1, and the most valuable chapter: the five designs that remove the need for a lock entirely. With production-ready code in Python, SQL, Go, Lua and YAML, eight recurring mistakes, Prometheus observability, a checklist, FAQ and glossary. Expanded on 18 September 2026: the clocks that decide whether your TTL is a promise or a fiction (CLOCK_MONOTONIC vs CLOCK_BOOTTIME, why you subtract the whole RTT, and the point of no return before every write), the watchdog that renews the lease and why renewal does not close the window unless it is a CAS on the token, the full arithmetic of Kubernetes leader election (LeaseDuration, RenewDeadline and RetryPeriod, the mandatory os.Exit(1) in OnStoppedLeading, and the leaseTransitions field that hands you a fencing token for free), a Redis failover told second by second in which a lock is lost with nobody at fault and what WAIT and min-replicas-to-write actually fix, two more backends that work (DynamoDB conditional writes with the rvn trick that avoids comparing clocks, and Consul sessions with lock-delay and ModifyIndex), scheduled jobs with the concurrencyPolicy Forbid that is not mutual exclusion and the hundred missed schedules that stop the controller for good, granularity and cost with the three metrics that make contention visible and the convoy that takes the system down, and three tests that prove your lock is broken before production does (SIGSTOP, Toxiproxy and the assertion on the fence).