Irony Alert: Pentagon Now Sees Big Data as ‘National Security Threat’

The data divers at the Defense Department know better than most how to track down someone just by looking at his phone records. Now they want to know if America’s enemies could cause a fiscal meltdown or a massive cyber attack by combing through Netflix queues, Uber accounts, and Twitter feeds.

The doomsday thinkers over at DARPA are looking for researchers to "investigate the national security threat posed by public data available either for purchase or through open sources." The question is, could a determined data miner use only publicly available information — culled from Web pages and social media or from a consumer data broker — to cause "nation-state type effects." Forget identify theft. DARPA appears to be talking about outing undercover intelligence officers; revealing military war plans; giving hackers a playbook for taking down a bank; or creating maps of sensitive government facilities.

The irony is delicious. At the time government officials are assuring Americans they have nothing to fear from the National Security Agency poring through their personal records, the military is worried that Russia or al Qaeda is going to wreak nationwide havoc after combing through people’s personal records.

As timely as this new DARPA project is, it wasn’t NSA snooping that piqued the agency’s interest. It was Brokeback Mountain. In 2009, Netflix sponsored a contest to improve its movie recommendation algorithm. Things went off the rails when a pair of researchers used supposedly anonymous information provided by the company to identify Netflix customers, by comparing their film reviews with reviews posted on the Internet Movie Database. A closeted lesbian who had watched the award-winning gay cowboy flick sued Netflix, alleging her privacy was violated because the company had made it possible for her to be outed.

DARPA’s requests for research proposals points to the Netflix debacle, and the lawsuit, as a cautionary tale. Part of the research is aimed at identifying which potentially dangerous databases and computing tools are out there.

And in a second bit of irony, DARPA suggests a few, including "low-cost big data analytic capabilities" like Amazon’s cloud service. That’s the service that the CIA wants to use to build a $600-million cloud for the intelligence community. Could a tool meant to serve the spies’ computing needs end up being used against them? Researchers who think they have the answer may submit their proposals starting Aug. 26.