How Big Data can find the big story.
- By Kalev LeetaruKalev H. Leetaru is the Yahoo! Fellow in Residence at the Institute for the Study of Diplomacy in the Edmund A. Walsh School of Foreign Service at Georgetown University. His work centers on the application of "big data" towards understanding global human society in new ways and he is the creator of the GDELT Project.
The escalating tension between Ukraine and Russia in Crimea has captivated the world’s headlines the past few weeks, invoking imagery of Russian occupation not seen since the fall of the Soviet Union. As the world’s media outlets run round-the-clock coverage of masked soldiers facing off against besieged Ukrainian military outposts, the rest of the world has largely been drowned out. Few, for example, have likely followed the events in Nigeria, where Boko Haram has executed 59 children in an attack on a boarding school and killed more than 150 over the past two weeks.
While both Nigerian attacks were reported by international outlets like the BBC and The Guardian, the token attention they received was almost immediately lost in the enormity of articles, blog posts, and social media updates that the unraveling situation in Ukraine has generated. Can the promise of "big data" be leveraged to sift through the world’s news coverage over the past year and create a map of the evolving unrest in Nigeria?
Using the Global Database of Events, Language, and Tone (GDELT) Project, which monitors global media each day, and Google’s BigQuery system, one can identify the location and intensity (as proxied by media volume) of protests (indicated by the pink dots) and violence against civilians (red circles) in Nigeria from April 1, 2013, to the present. In all, international media outlets produced nearly 3.5 million news stories on events in Nigeria during this period that were monitored by the database, providing a rich portrait of a nation in turmoil.
In general, Africa tends to be underrepresented in mainstream U.S. media compared with coverage of European nations or American hotspots like the Middle East, skewing the American public’s understanding of world events.
Nigeria presents a particularly interesting case study as a geographically-divided country in that the nation’s stability is being increasingly challenged through the rise of Boko Haram in the country’s northeast, domestic unrest in the southwest, intensifying ethnic strife in the center, and the growing influence of radical groups on the continent at large.
Most striking is that protest activity is largely centralized in the country’s south, while violence far outpaces peaceful protests in its north. President Goodluck Jonathan’s concerns in January 2012 that the Boko Harem violence in northeast Nigeria had become "even worse than the [1967-1970] civil war" are seen in stark relief today with the visible violence there. These range from an attack that killed 10 in a remote village in Adamawa last April to the more than 150 killed in Borno State in clashes between Boko Haram and government soldiers. Kano saw gunmen open fire on a primary school and a raid on a major Boko Haram bomb-making factory. In Zamfara State a five-hour attack killed 48 people and included hilltop snipers.
In Nigeria’s "Middle Belt," 30 people were killed in Plateau State in January 2014 and over 100 houses were burned to the ground two months later, in early March. In May of last year, nearly 70 people were killed or injured in Taraba State in a clash between the Jukun and the Hausa and Fulani members. In Benue State, 205 Christians were killed in the last half of 2013.
Violence in the southern half of the country is more interspersed with protests. In September 2013 in Abuja in central Nigeria, thousands of electrical workers threatened to disrupt national power supplies in protest over privatization plans of the nation’s major power company. In the southwest, 200 Christian students — protesting the wearing of hijab by their Muslim peers in the classroom — blocked major roads in Osogbo. Ogun State, in particular, has been a flashpoint for protests since university instructors staged a walkout in July 2013 over salary disputes with the government.
Big data now gives us the ability to collect the vast number of micro-level stories from the ground and aggregate them together to give a satellite-like view of a country, letting us peer down onto earth through the collective voices of the world’s news media. Placing the shared chronology of millions of news articles into a single map makes it possible to see how strongly Nigeria’s unrest is clustered into specific geographic regions with unique profiles and creates a tool that can be used to communicate these trends more concretely with policymakers.
For Nigeria, big data paints a portrait of a strongly divided nation in sharp relief. Yet, what new insight can be gained by transforming millions of news articles into a single visual map? Perhaps most critically is a cohesive view of Nigeria’s conflict, turning casual anecdote into a geographic atlas. This atlas clarifies how deeply entrenched Boko Haram has become and its rapid spread through the northern half of the country. It also illustrates how protest activity is far more common in the south, while the cultural divisions that gave the Middle Belt its name are indeed a center point of ethnic conflict.
However, perhaps most critically, big data is a powerful tool to rise above the deluge of information available today, to watch emerging trends of conflict from the most remote regions of the world — even when a breaking story in Eastern Europe seems to wash all else from view. Perhaps this is big data’s greatest potential at the moment: not as a crystal ball seeing into the future, but as a mapmaker that transforms chaos into cartography.
Isaac Stone Fish is associate editor at Foreign Policy. Previously a Beijing correspondent for Newsweek, he wrote stories on such subjects as the Dalai Lama’s effect on international trade, China’s love affair with rogue states, and crystal meth in North Korea. His articles have also appeared in the International Herald Tribune, the Economist, and the Los Angeles Times.| Passport |
Marc Lynch is associate professor of political science and international affairs at George Washington University, where he is the director of the Institute for Middle East Studies and of the Project on Middle East Political Science. He is also a non-resident senior fellow at the Center for a New American Security. He is the author of The Arab Uprising (March 2012, PublicAffairs).
He publishes frequently on the politics of the Middle East, with a particular focus on the Arab media and information technology, Iraq, Jordan, Egypt, and Islamist movements.| Marc Lynch |