Survivorship bias is a tricky phenomenon.
During World War II, the Allies studied Nazi damage to their airplanes. Their study resulted in this dotted illustration:
The analysis of these damages resulted in the idea that their airplanes should be reinforced in the areas of the airplane showing the densest clusters of red dots. A statistician by the name Abraham Wald didn’t agree. Instead, he suggested that the airplanes should be reinforced where there were no indications of damage.
Wald was right, of course. By studying planes that got back home somehow, the red areas indicated non-fatal damage areas. If it had been possible to study the plane that were shot down and destroyed by the Nazis, they would be likely to have found an inverse pattern.
The Fallacy of Survivorship Bias
This is a great example of a very specific fallacy — survivorship bias.
During World War I, the military forces increased the issuing of helmets for soldiers. The number of wounded soldiers skyrocketed, but this wasn’t because helmets somehow made it easier for more people to get hurt. The helmets simply rather made sure that less people died and instead showed up in the column for “wounded” soldiers. Progress of a kind, no doubt.
In both popular culture and business, we’re fascinated by successful people and successful brands. We want to know how they think and how they act. But few think to study those who tried — but never turned out to be successful.
It bears repeating again and again; correlation does not equal causation.
Actually inverting your data sets be many times more causative and provide much more powerful clues to your analysis.
Cover photo by Jerry Silfwer