One person’s dispassionate expert is another person’s lobbyist

Last week I received the following news release from the National Research Council:

A new report from the National Research Council recommends that the U.S. intelligence community adopt methods, theories, and findings from the behavioral and social sciences as a way to improve its analyses.  To that end, the Director of National Intelligence (DNI) should lead a new initiative to make these approaches part of the intelligence community’s analytical work, hiring and training, and collaborations.

The report, which was requested by the Office of the Director of National Intelligence, urges the intelligence community to routinely evaluate the performance of its analytical methods.  One important step in that direction is to attach whenever possible numeric probabilities with uncertainty estimates for the events that analysts assess and forecast.  Without explicit quantifiers, analysts cannot communicate their conclusions clearly or evaluate the accuracy of their analyses over time. Policymakers need to know how confident analysts are and how well they understand the limits to their knowledge, the report emphasizes.  It recommends many specific steps that DNI can implement as part of analysts’ everyday work. 

"The social and behavioral sciences have long studied topics central to analysts’ work, such as how people evaluate evidence and collaborate on difficult tasks,” said Baruch Fischhoff, chair of the committee that wrote the report and professor of social and decision sciences and of engineering and public policy at Carnegie Mellon University, Pittsburgh. “That research has had some impact on that work. Our report shows how the community can take full advantage of that research – and of its dedicated analysts – by adopting an evidence-based approach to its own analytical methods.  We envision a community engaged in continual learning, both absorbing scientific research into the analytical process and evaluating its own performance."

Now, this all sounds good to this social scientist’s ears, but there’s one little thing nagging at me.  A quick glance at the "Committee on Behavioral and Social Science Research to Improve Intelligence Analysis for National Security" reveals the following membership:  four psychology professors, three political scientists, three business professors, and a public policy professor. 

Now, no offense, but is it really shocking that a group of social and behavioral scientists conclude that their work should be embraced more by the intelligence community? 

None of this is to say that I disagree with the report’s findings.  I concur that the intelligence community should “ensure that the intelligence community (IC) applies the principles, evidentiary standards, and findings of the behavioral and social sciences.”  And I wholeheartedly agree that there should be more "exchange of expertise between the IC and academic research environments."  (Based on this survey, by the way, a fair percentage of IR scholars already do paid or unpaid work for the government). 

The tools of social sciences are not magic bullets, but they’re actually quite useful, and I want analysts to rely on every tool in their cognitive arsenal.  To use a baseball metaphor, think of this report as suggesting that sabermetrics would be a useful complement to traditional scouting as a way of analyzing talent.. 

The thing is, as much as I might want to be viewed as a thoroughly detached and dispassionate expert on these questions, I fear that the rest of the world will view this an exercise in interest group lobbying.  The report would have been more persuasive if more "old-school" intelligence analysts had signed off on the report (though Thomas Fingar was one of the signatories).