I’m going to start showing this article to anyone who questions the usefulness of social science, it’s status as an actual science, or what use my research (especially the implicit stuff) might be. For those who aren’t interested in clicking the link, some social scientists are arguing that they can get a handle on a player’s future success in the NFL by doing textual analysis on college press conferences, etc. These responses can then be used to map intangible psychological characteristics. Basically, they argue that it can help measure things like “leadership”, “winning attitude,” etc.
Honestly, I think it makes a lot of sense. I’ve done some research with implicit attitudes, and I like implicit research, textual analysis, etc. because it’s a quieter way of measuring attitudes and behavior. People construct answers to explicit things like survey questions, etc. Methods like this textual analysis (and the IAT, and the Misattribution Test, Go-No Go, etc.) are sneaky and don’t illicit the same sort of interviewer, framing, question wording, comparability, etc. problems of standard surveys. And apparently NFL Combine tests.
That said, there are obviously issues. John Kerry, to the best of my knowledge, is not a terrorist and yet their analysis identified him as one. Well, that’s not really a problem, but just an aspect of dealing in probabilities not certainties, but yeah, kind of a big mistake there. And, the 4-quadrant graph that accompanied the bit about Aaron Rogers (who is awesome, by the way) and Aleks Smith is a little messy. The relationships aren’t immediately clear (plus, on a visualization note, it tries to show too much. I’d have done 80+ and >80 on QB ratings).
However, I think these issues have more to do with needing to refine the measures and theorize about which psychological aspects are most important to future success than any underlying problem with the concept. I mean, just from a pro-sports perspective, once you get to the level of the NFL (or the NBA, MLB, NHL, etc.) you are so far to the right of the bell-curve on pure physical ability that distinguishing, physically, between Vince Young and Aaron Rogers (or Drew Brees, etc.) is probably next to impossible and likely useless if trying to predict future NFL success. Analysis like that described in the link though, that can help you (probabilistically mind you) get inside the players’ heads and help managers/owners make more informed decision. It may still be something of a crap shoot, but at least you’ll be playing with weighted dice.