Nate Silver vs. Kim Jong Un

The presidential election is finally over and Nate Silver won. It was a "revenge of the nerds" moment: The data geek bested many of the cool kid pundits and their time-honored tradition of predicting election results based on gut feel, a favorite poll, and a few conversations with unnamed campaign aides and undecided voters in Ohio diners. Silver’s 538 model proved what we should have known for years: good election forecasting is all about math, not anecdotes. It’s the analysis of big data that produces big insight.

If only improving analysis of American national security threats were that easy. But of course it’s not. Because assessing foreign policy challenges is less about crunching numbers and more about understanding the touchy-feely world of intentions — how allies and adversaries think, what they want, how they feel. Intentions are still the tough stuff of most of our hardest security challenges — whether China will become a responsible stakeholder or a revisionist power, whether Iran will cross the nuclear redline, whether North Korea will come in from the cold.

North Korea’s twenty-something leader, Kim Jong Un, is the poster child of this intentions problem. Even though the DPRK is one of the most closed societies on earth, we know an awful lot about Kim’s military. We can spot his country on a map, identify his Navy ships at sea, recognize his army’s uniforms across the Demilitarized Zone, and even watch video of the Supreme Leader visiting amusement parks and kindergartens. We also have a pretty darn good idea of his nation’s military capabilities, including the approximate number and types of nuclear weapons. Using commercial satellites, Google Earth, and other open sources, my colleague Sig Hecker, along with Frank Pabian, have even pinpointed the likely location of North Korea’s third underground nuclear test tunnel, which they believe could be used with two weeks’ notice.

The one thing that intelligence analysts inside the government and academic experts outside the government don’t have a clue about is what this inexperienced leader with the dour haircut and Mao suit intends to do next. Information about Kim was so tightly controlled that as late as 2009, CIA officials were compiling their dossier of the heir apparent based on secondhand reports from a sushi chef. Given North Korea’s penchant to play the reckless rogue, its growing nuclear arsenal, and its close proximity to 28,500 U.S. troops stationed in South Korea, getting a better read on Kim Jong Un’s policy preferences and personality is no small matter.

So what does Kim have in mind? Experts are all over the map. Some see Kim’s recent moves — including public statements about ending "belt tightening" and rumored plans to start decollectivizing farms — as a sign that North Korea might be moving toward Chinese-style economic reforms. A few, like Jay Ulfelder, think that political liberalization may not be so far-fetched. Victor Cha is far more skeptical, warning that Kim is no reformer, despite his love of pro basketball and pizza. About the only points of agreement among Korea watchers are that Kim likes Disney characters and that he’s got a nose for throwback fashion styles. But even on this last point, it’s still unclear whether Kim’s dress is for show or for real — is it a deliberate ploy to make himself look like his popular grandfather? Or is that just the Hermit Kingdom strongman uniform?

In 2002, Defense Secretary Don Rumsfeld got sneers and jeers for rambling about "known unknowns." But Rumsfeld was onto something. He was trying to make a crucial distinction originally developed by Sherman Kent, the legendary leader of the CIA’s analytic branch. Kent argued that there are three types of intelligence:

1. Information that is knowable in the world and known by our government;
2. Information that is knowable in the world but not known by our government;
3. Information that is not knowable to anyone at all.

The first, "knowable and known" category includes things like weapons that can be counted in theory and have been counted by U.S. officials in practice. The number of Chinese aircraft carriers, for example, is well known (they currently have one, a refurbished Ukrainian ship). The second category of intelligence includes things that can be discovered in theory by our intelligence agencies but have not been discovered in practice. We know about the existence of China’s carrier, but unless you have been on board or have a spy who has, it’s hard to know how the newly retrofitted ship operates under various weather and battle conditions. Carrier performance is known by some in the Chinese navy but probably not by us. The third intelligence category is the doozy: Things that are not knowable to anyone at all. This is the world of intentions.

Assessing even our own intentions turns out to be harder than you might think. Imagine that I asked you to write on a piece of paper where you intend to spend your summer vacation next year. Eleven months from now, you retrieve your piece of paper and compare what you predicted to what you actually did. There is a pretty good chance that your prediction turns out to be wrong, for a gazillion possible reasons. Maybe you took up snorkeling, gave up smoking, bought a house, lost a job, had a baby. Or maybe the change in plan had nothing to do with you — the hotel closed for renovation, the travel deal expired, the security situation there deteriorated. Life has a way of intervening, making even the best laid plans go awry.

This is the easiest case: It’s you predicting your own behavior. Now try predicting the behavior of someone else. Add volumes of information, some of it correct, some incomplete, most of it probably wrong. Extend the time horizon. And then consider that this person is actively trying to deceive you.

Welcome to the world of intelligence analysis.

This is why intelligence agencies have such a hard time understanding whether China’s rise will be peaceful, whether Iran is going nuclear, whether Burma’s political opening will continue, whether the Muslim Brotherhood is serious about democratization, or what in the world Kim Jong Un will do next. Even Kim may not know the answer to that one. But maybe Joe Scarborough and Nate Silver can bet on it.