Twitter mining vs deep viral mining

Twitter-disinformation aside, there still remains an important question of whether we could actually use the Internet to spot new epidemics. I haven’t yet formed a firm opinion; perhaps, once data from mobiles is well-integrated into our tools this would be possible — but for now, we are, probably, still are quite far fromfiguring out how to predict epidemics with the Web tools alone (the point being that epidemics usually break out in places with limited internet access). 

Here are two interesting perspectives, which may help us stimulate some debate about this issue, for, ultimately, WHO and national governments would need to rethink their own toolkit for predicting epidemics.

First comes a piece by Mark Honigsbaum in Prospect, where he finds faults with Google’s Flu Trends and argues that, instead of Google, we need better ways of doing "deep viral mining: 

But for all their ingenuity, the worry is that these amount to little more than technological tricks. One concern is that they can end up fueling the very anxieties they are supposed to alleviate. Google call this the “Angelina Jolie” effect—“If Angelina has diarrhoea you see a sudden spike in people searching for diarrhoea,” admits Mark Smolinski, the head of Google’s predict and prevent initiative. Equally, if people fear an outbreak of swine flu, and search for it out of worry, the tool becomes useless. But much more importantly, clever internet sites and search algorithms would have done little to spot a swine flu in Mexico, or many of the other less developed countries where viruses often begin, because few people use the internet. (The current best estimate suggests that the swine flu virus had been circulating in Mexico for a couple of weeks prior to being recognised, at least in part because the Mexican health system is poorly set up to spot emerging pandemics.) The problem with sites like Flu Trends, and similar technology-driven approaches, is that they are necessarily reactive.

What is needed is a fundamentally different approach, in which rather than waiting for the viruses to come to us, we go and find them first. This is the theory behind “deep viral mining”—essentially traveling to the jungles of Africa and Asia and gathering data on animal viruses before they leap the species barrier to humans. These techniques are the most reliable way to spot emerging threats. Such an approach in the early 1960s could have spotted the presumed precursor virus to HIV circulating in Africa, and in turn prevented the eruption of Aids in 1981. Similarly, if in the early 1990s the WHO had spent its money putting field epidemiologists on the ground in Guandong, the Chinese province where the SARS coronavirus likely first leaped from civet cats to humans, we may have be able to avoid the 2003 SARS epidemic.

Second is a piece by Erik Hersman of Ushahidi, where he argues that there are still a lot of things that can be done with online data about epidemics; we simply need more integration between different existing tools, which don’t yet work together that well (plus, platforms like Ushahidi might help to open this process to mobile phone users, which is a much bigger pool than internet users):

 What we have is the beginnings of an ecosystem for emergency and disaster information. The projects are disjointed and unconnected, and there’s little hope of making them one cohesive unit (nor should the necessarily be).

What I do hope to see in the future is that the protocols, tools and processes for gathering, making sense of, and then disseminating crisis information becomes more open and standardized. There’s no reason that Ushahidi shouldn’t plug and play well with Evolve, which then feeds into Threatwatch on Twitter and is all part of a mapping and visualization scheme by larger publishers.

It would be very interesting to get some of the minds behind Twitter, Ushahidi, InSTEDD, Facebook, Wikia, Google and others together to better figure out how we can each continue to build independently, yet at the same time work towards a better ecosystem for emergency information.

Provided we can control for misinformation and data accuracy, I think both approaches should be pursued, especially if the data generated by social media/mobile phones is cheap to aggregate and analyze.