The Incident That Wasn't: False Positives and Alert Fatigue at Scale
How a Prometheus misconfiguration burned two on-call engineers for 11 hours chasing a ghost. What the postmortem missed.
Kubernetes changelogs. Terraform war stories. Incident postmortems nobody else writes up. Lands in your inbox every Tuesday at 6 AM ET.
Issues Published
147
Readers
24,800+
Avg. Read Time
11 min
Every Tuesday
6 AM ET
Every vendor wants to tell you their platform is the answer. Every conference talk ends with a slide that says "and then we adopted GitOps and everything was fine." Neither of those things is true, and you know it, because you're the one who gets paged when it isn't.
Deploy started because there was a gap — a real one — between the practitioner's lived experience and what was being written about it. The gap between a Kubernetes changelog and understanding which of those 47 bullet points actually matters to your 200-node cluster. Between "we improved our DORA metrics" and the six months of organizational friction that actually moved the number.
This is a newsletter written by people who have been on-call, who have argued about internal developer portals in quarterly reviews, who have read the Terraform provider changelog so you don't have to — and who know which parts to flag and which parts to skip. It lands every Tuesday at 6 AM ET. It assumes you already know what a control plane is.
"Written for the people who keep it running — not the people who sell it."
147 issues in. Still no sponsored content. Still no "10 reasons to migrate to service mesh." Just signal, delivered on a schedule you can set your incident rotation by.
§ Archive
How a Prometheus misconfiguration burned two on-call engineers for 11 hours chasing a ghost. What the postmortem missed.
Beyond the release notes. Three patterns that break in existing codebases and the one feature worth adopting immediately.
Your internal developer portal has no SLA, no roadmap slide, and three stakeholders who think it should be Backstage. Here's how to walk in prepared.
When your deployment frequency stops improving, it's rarely a process issue. It's usually a queue somewhere in your pipeline.
Sidecar containers are stable. Dynamic resource allocation is beta. Here's which of the 40 changelog items you need to actually read.
A senior SRE at a fintech shares the full timeline, the contributing factors, and what the public RCA left out.
Not a feature comparison. A decision framework based on team topology, existing tooling, and the one question most evaluations skip.
Platform teams obsess over golden paths. They rarely measure whether engineers actually follow them — or why they don't.
We interviewed six platform teams who made the switch. Three are happy. Two are neutral. One wants to talk to a lawyer.
Postmortems that go deeper than the five whys. Alert fatigue, on-call burnout, blameless culture in practice — not in theory. The gap between what your runbook says and what actually happens at 3 AM.
Internal developer portals, golden paths, and the organizational friction of building infrastructure for engineers. Why most platform teams fail at adoption — and the ones that don't.
Kubernetes releases, Terraform changelogs, Helm chart archaeology. The week's most important infrastructure tool updates, filtered for signal and annotated for what actually matters to production systems.
The only newsletter I read the same morning it arrives. The Terraform coverage alone saves me two hours of changelog archaeology every week.
Marcus Holloway
Principal SRE · Relay Financial
I use the postmortem coverage in our own incident reviews. The framing is better than anything my team writes in the heat of the moment.
Priya Venkataraman
Platform Engineering Lead · Meridian Health Systems
Forwarded the DORA plateau issue to my VP as context for why we need a dedicated platform team. It did more in 12 minutes than six months of my slides.
Daniel Osei-Mensah
Engineering Manager · Arco Commerce
"The Incident That Wasn't: False Positives and Alert Fatigue at Scale" — how a Prometheus misconfiguration burned two engineers for 11 hours chasing a ghost.
24,800+
Readers
147
Issues
4 yrs
Publishing
0
Sponsored Issues
Get Tuesday's Dispatch
Platform engineering signal · Every Tuesday 6 AM ET