Dark Crystal

In early March, California Sen. Dianne Feinstein, who heads the Senate Intelligence Committee, complained that America’s spooks had failed to warn about the risks of uprisings in the Arab world. Instead, they had provided "nothing that we didn’t read in the newspapers," she griped. "Whether it was Yemen, or Bahrain, or Egypt … nothing." James Clapper, the director of national intelligence, had been forced to acknowledge at an earlier hearing, "We are not clairvoyant."

One could ask the same tough questions about the 2011 Failed States Index, which notably ranks Tunisia — where the Arab revolt began — 108th out of 177 countries and Bahrain — where it stalled in a frenzy of sectarian hatred — 129th. Clearly, the index is not a prediction model; it’s an annual portrait of state failure, examining the pathology of weak states through detailed rankings in a dozen categories ranging from education to refugee flows to economic decline. It’s not meant to be a crystal ball.

"For certain countries in the region you could see various indicators sliding over the years," says J.J. Messner, senior associate at the Fund for Peace, which partners with Foreign Policy to produce the index. "But that doesn’t necessarily mean a revolution is going to happen tomorrow, and it does need to be contextualized."

It turns out that looking at rows of numbers is a lousy, or at least insufficient, way to predict an uprising. "People who point to structural conditions almost always do so post hoc," argues University of North Carolina sociologist Charles Kurzman, author of a book about the 1979 Iranian revolution. They tend to produce complicated models that accurately correspond to past events, but may not do so well in predicting the future — social scientists call the phenomenon "hindsight bias." Says Kurzman: "These structural analyses are always playing catch-up."

The Failed States Index has often captured what could be captured in the Middle East, from the permanent teetering chaos of Yemen (13th on this year’s index) to the destabilizing refugee crises of Sudan (3rd) to the rock-solid stability of wealthy Qatar (139th). Lebanon’s fractured politics and history of sudden sectarian bloodshed placed it reasonably at 43, fifth among Arab states. Yet the index rated oil-rich Libya, a hollowed-out shell of a state run by a murderous madman, all the way down at 111th in the year before its civil war.

Why? Well, even the best data can be subtly misleading. Tunisia, where the uprisings began, had become one of the most developed countries in the Arab world, with a vibrant middle class, burgeoning trade with Europe, and one of the best education systems in the region. Egypt on the eve of revolt was certainly much worse off in general, but economic reforms had brought a modicum of growth in recent years. Bahrain was perceived to be a liberal oasis in the Persian Gulf, with a robust banking sector and a per capita income to rival Europe’s.

Aggregate figures often mask a more complex reality, failing to adequately account for potentially explosive factors like popular resentment of police brutality, rampant economic inequality, or the unmet expectations of the Middle East’s biggest-ever generation of college graduates. And they certainly can’t anticipate the powerful "demonstration effect" that events in one place — i.e., Tunisia — can have elsewhere. After all, plenty of countries perform poorly on the index each year, and very few experience revolutions.

"Revolutions are inherently unpredictable," says Kurzman. "They involve massive disobedience, huge numbers of people breaking from their normal patterns of behavior, highly risky confrontations with security forces — the sorts of activities that people are loath to engage in and usually don’t, until they think that other folks may join them." It’s thousands of individual decisions, not rows of statistics, that add up to political upheaval.

Even George Mason University’s Jack A. Goldstone, recently touted by Foreign Affairs as "one of the world’s leading experts on the subject" of revolutions and co-author of an article in the American Journal of Political Science called "A Global Model for Forecasting Political Instability," failed to see Tunisia and Egypt coming.

That’s no particular failing of Goldstone’s. As Philip E. Tetlock, professor of psychology at the University of California, Berkeley, found in a 2005 book evaluating "expert political judgment," the experts, for all their superior knowledge, rarely do better than chance in predicting future events. As Goldstone acknowledged, "The degree of a sultan’s weakness is often visible only in retrospect."