Here's an excerpt from the article I linked yesterday. It's a summary of the most important findings from Philip Tetlock's 20-year study of prediction accuracy.
Tetlock’s experiment had 284 experts make 82,361 predictions about political and economic trends. Over time those predictions were checked for accuracy, and Tetlock examined his data for trends. Among the trends Tetlock uncovered in his study:
* simple extrapolation using mathematical models typically does better than human predictors
* education and popularity increase the predictors’ confidence but not their accuracy
* prognosticators overpredict change and underpredict the status quo
* extremists predict worse than moderates
* some people predict better than others, and their superiority will not be confined to their area of expertise
What's the difference between more accurate and less accurate predictors?
Tetlock ultimately concluded that cognitive style was the most important influence on prediction accuracy. Using the framework derived from Isaiah Berlin’s essay The Hedgehog and the Fox [Tetlock found that] hedgehogs “know one big thing” and “apply that one thing everywhere,” express “supreme confidence in their forecasts, dismiss opposing views and are drawn to top-down arguments deduced from that Big Idea”; they “seek certainty and closure, dismiss information that undercuts their preconceptions and embrace evidence that reinforces them” (Begley, 45). Foxes “consider competing views, make bottom-up inductive arguments from an array of facts, doubt the power of Big Ideas” and “are cognitively flexible, modest and open to self-criticism” (Begley, 45). Ultimately, “what experts think matters far less than how they think: their cognitive style” (Begley, 45). Tetlock found that foxes outperform hedgehogs in prediction accuracy in virtually all fields, across all time periods, and across the various levels of expertise.
Media audiences like Hedgehogs--big idea people who are inflexible, imodest and unencumbered by doubt. The big idea frames everything and there is no room for questions.
In the media ratings chase, Hedgehogs will win most of the time, if not every time. Foxes are right more often, but audiences tend to find them boring.
Photo via Shroedinger's Oreo.