Designing for failure
[Placeholder] Why every network call is a question, not a statement.
Everything fails
[Placeholder] In a distributed system, partial failure is the normal state. A request can succeed, fail, or — most annoyingly — do neither and simply never answer.
[Placeholder] The question isn’t whether a dependency will fail, it’s what your system does when it does.
Timeouts are a feature
[Placeholder] Every outbound call needs a deadline. Without one, a slow dependency quietly becomes your outage.
ctx, cancel := context.WithTimeout(ctx, 300*time.Millisecond)
defer cancel()
resp, err := client.Do(req.WithContext(ctx))
Retries need budgets
[Placeholder] Retries without backoff and jitter turn a blip into a thundering herd.
[Placeholder] A retry is a bet that the next attempt will go better than the last one. Place it carefully.
About me
I’m from Bangladesh and have lived in Seattle long enough to call it home. I’ve run two half marathons and I’m training for a full one now. I write here about reliability, distributed systems, and running, which have more in common than you’d think.
I’m always happy to talk about AI/ML infrastructure, platform engineering, and the cool things you’re building:
Let's talk