Just recently read a brief blog post on Freakonomics (see here) about the rather poor quality of expert judgments. Tetlock, a social scientist I otherwise strongly dislike because of his, rather anti-science, stance against the use of implicit attitudes (see here for a good smackdown of his objections), has also published an extensive and incredibly ambitious project on the fallibility of expert political judgments, primarily from a foreign policy standpoint (see here). Basically, Tetlock finds that no one predicted the end of the Cold War and pundits are only slightly better and often worse at prediction than dart throwing monkeys. Dubner is basically arguing that coaches and GM’s aren’t too good at predicting pro-success. I’m not sure though how bad we (as social scientists, etc) really are at prediction or whether we allow non-relevant considerations (ideology, jersey sales, Tim Tebow) to cloud that judgment. I mean, we’re pretty good at predicting vote choice, turnout, and a lot of other social phenomenon. Then again, most social scientists aren’t pundits or consultants (BDM and the Freakonomics guys notwithstanding). Also, it makes you wonder about the “wisdom of markets” and the “wisdom of crowds” if experts are so bad at prediction. Provides a whole new spin on the economic crisis if even hedge fund managers, presumably experts, could be just as bad as chance at predicting what will go up and down. Anyway, just an interesting area of research and an interesting aspect of social science to think about, especially considering how important we seem to think political knowledge is (Zaller 1992, 1996, Converse 1964, etc).
