> Had Knight implemented an automated deployment system – complete with configuration, deployment and test automation – the error that cause the Knightmare would have been avoided.
Would it have been avoided though? Configuration, deployment and test automation mean nothing if they don't do what they are supposed to do. Regardless of how many tests you have, if you don't test for the right stuff it's all useless.
The specific part is configuration as code. So the config change (flag activation) and code change (flag calling) would have been synchronized.
And there wouldn't have been one server of 8 with a different build for a meaningful time and also if it did fail to deploy on that one server it would have been obvious.
That's based on the assumption that someone would have thought about testing that particular flag for that particular scenario.
In my view this would only have been caught by a deployment to an identical copy of production, with running, simulated transactions, and high level funtional testing. Testing for each individual config value and scenario of where it may be used is playing whack a mole. Basically, I'd make a clone of prod, simulate everything that happends externaly (APIs, etc) and observe transaction KPIs and other high level business indicators. Testing for tech is insuring that the tech works, and sometimes that means testing that it's broken.
Would it have been avoided though? Configuration, deployment and test automation mean nothing if they don't do what they are supposed to do. Regardless of how many tests you have, if you don't test for the right stuff it's all useless.