From LinkedIn · · 2 min
5 signs your production system needs an audit
An audit is not for a system that is broken. It is for a system that still works, for reasons nobody can fully state. Five signs you have crossed that line.
By Olha Shevchenko. Audits production systems on AWS and Node.js.
1 / 8
Nobody schedules an audit the week everything is on fire. They schedule it the week they realize they cannot say when the fire starts.
Most production systems that fail were not broken. They were running on what the team believed was true, not on what the team could still prove.
Here is how you know you have crossed that line.
You can explain how the system is supposed to work, but nobody can show you how it actually works. The design lives in memory and old diagrams. Production drifted off it months ago.
Every real answer about the system ends with a person's name, not a link. One person holding the whole map is not seniority. It is a single point of failure with a payroll number.
There is one part of the system nobody will touch. You route around it instead of changing it. It is load-bearing and unmapped, and nobody can draw what a single deploy there would reach.
The parts you worry about least are the parts nobody is watching. Calm is not the same as safe. A subsystem with no alerts is not healthy, it is quiet. Quiet is where the next incident is already forming.
The last incident ended, and nobody can say exactly why it stopped. If you did not decide how it recovered, you do not have a recovery path. You have a system that came back on its own, and a clock running until the time it does not.
If two or more of these land, you do not need convincing that something is off. You already feel it. An audit turns that feeling into severity, evidence, and a roadmap.
Most systems do not get dangerous when they break. They get dangerous when they keep working and nobody can say why.