Why David Adesnik is really wrong

When I started reading David Adesnik’s “jeremiad” against political science while he was guest-blogging at the Volokh Conspiracy, I started to cringe. Then I got mad. David is a very smart guy, but there’s a lot in this post that Adesnik either distorts or gets flat-out wrong. Chris Lawrence has already taken him to task — though Laura of Apt. 11D sympathizes with Adesnik — but there’s so much that’s wrong with his post that I’m going to have to indulge in a quasi-fisking here. Adesnik’s post is in italics and indented:

The secret to success in America’s political science departments is to invent statistics. If you can talk about regressions and r-squared and chi-squared and probit and logit, then you can persuade your colleagues that your work is as rigorous as that of a chemist, a physicist, or (at worst) an economist.

There’s a very big difference between creating new data and using new statistical techniques to analyze old data. I strongly suspect Adesnik’s source of irritation is the latter. The former is way too rare in the discipline, especially in international relations. Mostly that’s because building new data sets takes a lot of time and the rewards in terms of professional advancement are not great, whereas relying on old data has no fixed costs. This is one reason why Pape’s article is worthy of note — he actually collected new data, which leads to results that Adesnik himself admits are “surprising.” David mistakenly conflates creating new data with the use of fancy statistical techniques when they’re not necessary. The latter can be a occupational hazard — though I’d argue that the greater danger is the proliferation of sophisticated regression analysis software like STATA to people who don’t have the faintest friggin’ clue whether their econometric model corresponds to their theoretical model.

And unsurprisingly, in the rush to invent statistics, political scientists have made whatever assumptions they needed to justify their work. As historian John Gaddis has shown, political scientists actually have a very poor understanding of what science is. As a result, their work has suffered considerably.

Sigh. Of all the social sciences — including economics — I’ll bet that political scientists actually spend the most time discussing what constitutes proper scientific work. This is partly due to insecurity, but it’s also due to a refreshing humility about the difficulty of the enterprise. For good examples of this sort of debate, click here for one example, and here’s another. And, for good measure, click here, here, and here. Note that some of these works disagree with each other — and I certainly disagree with some of them. [So, has any good come from these books?–ed. Sometimes I think this has generated a healthy debate within the discipline, and other times I think it’s just navel-gazing.]

While I haven’t read Pape’s APSR study, the points he makes in the NYT are pretty much nonsense. For example in support of the assertion that suicide attacks are not connected to religion, Pape points out that… [quote here about the Tamil Tigers being responsible for the most suicide terrorist attacks–DD] The [Tamil] Tigers’ behavior only reinforces the belief that suicide bombing is a product of ideological extremism. But since you can’t put a number on extremism, political scientists have hard time studying it. (Which is why there are historians and anthropologists.)

I have no doubt that historians can, through closely argued scholarship, identify which groups are extremist — ex post. The key is to find descriptive characteristics that can be identified ex ante. Without ex ante markers to identify proper explanatory variables, theories degenerate into tautologies. Islamic affiliation is a descriptive category that can be identified ex ante, and Pape’s discovery that it’s not correlated with suicide attacks is a relevant and counterintuitive finding.

To be fair, Pape has some good points. As his study shows, democracies are the almost exclusive targets of suicide attacks, because liberal political systems are vulnerable to terror. Moreover, he is probably right that there is an element of rational calculation behind such attacks, since even extremists have an interest in success. Still, it is absolutely impossible to explain the tactics of Al Qaeda or Hamas without reference to their perverse ideologies.

This is a nice summary of Pape’s value-added. On the “perverse ideologies” question, I don’t think Pape would disagree. Without the ideology, it’s impossible to delineate these groups’ substantive preferences.

The real problem is that Pape, like so many political scientists, abandons all nuance in deriving policy programs from his work. As I see it, the cause of this unsubtle approach is political scientists’ obsession with statistics, a pursuit that dulls their sensitivity to the compexity of real-world political events. If numbers are your thing, you’re going to have a hard time explaining why Israelis and Palestinians have spent five decades fighting over narrow tracts of land.

I agree with Adesnik that one can draw different conclusions from Pape’s findings than he does — and this is a weakness in the paper. However, to attribute this to Pape’s obsession with statistics is amusing on a number of levels, many of which Chris Lawrence explained. Let’s just say that Bob Pape would not be considered welcome at a meeting of the large-N brotherhood at APSA. Indeed, Pape fully supports the Perestroika movement that I’ve discussed previously.

So then, what is to be done? As you might of heard, many political science programs require training in statistics but not foreign languages. That trend has to be sharply reversed. Learning foreign languages promotes immersion in foreign cultures and ideas, which in turn make it hard to ignore the role of those cultures and ideas in the realm of politics. Given that politics is an art rather than a science, there is no substitute for getting inside the minds of those we study.

I’m perfectly happy to see more cultural immersion, but the notion that such training will automatically induce greater understanding is horses@&t. Witness the self-criticisms — or rather, the lack thereof — within the Middle Eastern Studies community in the wake of 9/11. These people are deeply immersed in the culture and language of the Arab peoples. Is Adesnik really suggesting that people like Edward Said can enlighten us about the region? In conclusion, politics is an art and a science, a simple fact that many people within and without political science seem incapable of understanding. And for Pete’s sake, read the whole paper before penning a jeremiad like that. UPDATE: Adesnik continues on his jeremiad in this post (though he’s right on Moneyball). He gets it wrong again when he says:

The great flaw of modern political science is its desire to imitate microeconomists (and share in their prestige) by developing theorems that explain and predict the behavior of rational actors. Of course, that is exactly the wrong way to go about things. It is only when political scientists recognize that ideas and values are what drive politicians and voters that they will begin to produce something worthy of the name “science”.

Chris Lawrence explains what’s wrong with this statement. ANOTHER UPDATE: David Adesnik responds in non-jeremiad fashion. See also Josh Chafetz.