Russ Roberts interviewed psychologist Philip Tetlock about his research on forecasting future events using groups of amateurs. Here's a sample:
Russ: Going back to the earlier work that you did and before the fall of the Soviet Union, what were some of the main empirical takeaways from that work?
PT: Well, one big takeaway was that--liberals and conservatives had very different policy prescriptions, and they had very different conditional forecasts about what would happen if you went down one policy path or another. And that nobody really came close to predicting the Gorbachev phenomenon. Nobody, for that matter, came really close to predicting the disintegration of the Soviet Union later on. But everyone after the fact seemed to have an explanation that either appropriated credit or deflected blame.
Russ: And it was consistent with their worldview, I'm sure.
PT: And meshed perfectly with their prior worldview. So, it was as though we were in an outcome-irrelevant learning situation. It didn't really matter what happened--people would be in an excellent position to interpret what happened as consistent with their prior views. And the idea of forecasting tournaments was to make it easier for people to remember their past states of ignorance.
PT: This is an aside of sorts, but it's just a wonderful insight into human nature, and it's a theme here at EconTalk. Which is: When you went back and asked people to give what they remember as their probability of, say, the Soviet Union falling, what did they say?
Guest: Well, they certainly thought they assigned a higher probability to the dissolution of the Soviet Union than they did. And there were a few people who assigned really low probabilities who remember assigning higher than a 50% probability. So, people really pumped up those probabilities retrospectively. So, the psychologists call that 'hindsight bias' or the 'I knew it all along' effect. And we saw that in spades in the Soviet Forecasting Tournament.
Russ: Yeah. I think that's an incredibly important thing that we all tend to do. We tend to think we had much more vision than we actually had. And we usually don't write those things down. You happen to have written some of them down. So that was awkward, that they actually had their original forecasts. But most of us, the I-knew-it-all-along problem is a bigger problem for most of us because we don't write it down.
PT: Well, we truly remember it differently. Even if you think the person on the other side of the table knows what the correct answer is, you still tend to misremember it.
Econtalk has the hour-long podcast and a transcript of highlights from the interview.