The Full Measure of Freedom

Is the United States becoming less democratic? It depends on how you look at it. For many progressives, modern America is a place where politicians, lobbyists, and big corporations collude to constrain the political participation of ordinary citizens. For conservatives, freedom is eroded by an overweening federal government that intrudes into even the most miniscule details of everyday life.

Both positions seem self-evident to their supporters, and both sides can cite plenty of numbers to bolster their arguments. But what do those numbers ultimately mean? Calculating the share of gross domestic product comprised by federal spending would seem to offer a rock-solid indicator of "big government," for example. Yet does that mean that Somalia, where central government barely exists, is more democratic than the United States? One can offer a similar response to the liberal argument about the corrupt intertwining of private and public interests. There are no lobbyists in North Korea — yet few would argue that Kim Jong Un’s kingdom is a bastion of freedom.

Many people around the world aspire to "democracy." You’d think that such a desirable good would be easy to measure. But it isn’t — precisely because the concept is a notoriously slippery one. A healthy democracy has many potential components. Notions of popular rule differ widely according to country and culture. And even in places with long traditions of democracy, how one defines the term is inextricably bound up with complex value judgments, and the criteria by which those judgments are made change constantly.

But we shouldn’t give up too quickly. There are, after all, many facets of politics that we routinely quantify. Opinion polls are a widely accepted feature of electoral life. Finance and demography offer rich data sets with wide political relevance. Campaign strategists mine mountains of data in order to understand voting patterns.

Applying comparable techniques to the measurement of democracy could be enormously useful. Today, democracy promotion is no longer the exclusive preserve of the United States and a handful of other rich countries. The number of democratic societies around the world has increased dramatically over the past few decades, and many of these newcomers are increasingly offering money and know-how to nations aspiring to emerge from dictatorship or dysfunction. But it’s hard to understand whether you’re having an effect unless you can measure the progress of the societies you’re trying to help.

"In macroeconomics, we invest tens of millions of dollars in measurement," says John Gerring, a political science professor at Boston University. "But we have nothing like that in politics" — and especially when it comes to the international realm. The International Monetary Fund, the World Bank, and the World Trade Organization all compile reams of statistics on comparative economic indicators. The World Health Organization and other public health organizations track a wide array of data on global health. Scholars of democracy would like to follow suit. But so far, Gerring notes, "we don’t have the tools to understand these phenomena in a nuanced way."

In the United States, surveys by Freedom House, a venerable Washington-based non-profit largely funded by the U.S. government, are widely used as a basis for assessing the state of liberal values around the world. A few weeks ago, Freedom House released one of its annual reports, The Worst of the Worst. This particular survey aims to establish scores for the "world’s most repressive societies." It should come as no surprise that brutish North Korea — the gold standard of tyranny — ends up at the top.

But there are other cases where the rationale seems less obvious. Syria and Saudi Arabia, two countries that have recently unleashed their security forces on protestors, both earn average ratings of seven (based on a 1 to 7 scale, with 1 representing the most free and 7 the least free). Yet Bahrain, which is guilty of equally egregious behavior, doesn’t even appear in the group of most abhorrent authoritarian states. It’s not immediately clear why that should be the case.

In academia, many political scientists rely on another ranking system called Polity IV, which some regard as methodologically more sophisticated than the Freedom House studies. (See also Vanhanen’s Index of Democracy, which covers the years 1810 to 2000, the Democracy Index of the Economist Intelligence Unit, and Democracy Barometer). Each one of these systems comes at the problem from its own distinct perspective. Polity IV, for example, concentrates "only on the institutions of the central government and on political groups acting, or reacting, within the scope of that authority" (excluding separatist groups or rebels, for example), while the EIU Democracy Index tries to capture a broader sweep of criteria that includes not only electoral process and government functioning but also civil liberties, political participation, and political culture.

Some critics nonetheless regard such indices as blunt instruments. One problem, they contend, is the idea that you can reduce the myriad factors that shape democracy to a single "score." (Imagine doctors giving a person a ranking of "overall health" on a scale of 1 to 10. That might give you some idea of the patient’s life expectancy, but it won’t be very helpful if you’re trying to cure a particular disease.)

Experts are now taking up the challenge. Consider the Variety of Democracies Project, whose team includes Gerring. "Democracy is such a complex concept," he says. "There are so many dimensions to it that it’s a lot to ask of an index to try to boil everything down to a single score. So our approach is to try to disaggregate that score." His team maps out seven categories of measurement (electoral, liberal, participatory, majoritarian, consensual, deliberative, and egalitarian) and awards scores in each.

Of course, all of these models might find themselves forced to evolve as they confront the advent of "big data," the tsunamis of data now rolling in thanks to modern information technology. Cyber-guru Esther Dyson recently posited the pending arrival of what she calls the "Quantified Community," in which communities constantly measure

the state, health, and activities of their people and institutions, thereby improving them. Just consider: each town has its own schools, libraries, police, roads and bridges, businesses, and, of course, people. All of them potentially generate a lot of data, most of it uncollected and unanalyzed. That is about to change.

I suspect she’s right — and this could soon apply even to relatively undeveloped societies thanks to the ubiquity of Internet-capable mobile phones. Of course, such technologies will increase opportunities for both participation and surveillance as well as for measurement, radically transforming the very stuff of democracy even as they enable us to track its ups and downs. On top of that, we’ll also be able to conduct surveys in something close to real time rather than at year-long intervals (as is mostly the case right now). I can’t decide whether I’m thrilled or aghast at the prospect; probably a little bit of both. But there’s no question that we will learn a lot more about ourselves along the way.