“Big data is the latest casualty of overcooked promises made in pursuit of a good story.” This is the claim made by Nick Heath in Big Data: neither snake oil or silver bullet” in Tech Republic looking at the hype surrounding Google’s Flu Trends.
The example of Google Flu Trends is useful to look at the pitfalls and limitations of harnessing Big Data for Development. GFT is a service that predicts flu infection rates worldwide based on the search terms people are using, parsing a vast number of searches across 29 countries. It was designed as an early warning system for looming epidemics by analysing internet search terms for signs that people were coming down with the bug. Google Flu Trends’ early successes led to articles celebrating the triumph of big data and hailing the ability to resolve information from noise through correlation in large datasets. From searching through social media’s data exhaust in real time, the tech giant was able to map accurate estimates of flu prevalence two weeks earlier than the CDC’s traditional data sources.
And then, GFT failed—and failed spectacularly—missing at the peak of the 2013 flu season by 140 percent. GFT went quickly from the poster child of big data to the poster child of the foibles of big data – of big data hubris.
So what went wrong? In a Financial Times article, Big Data: are we making a big mistake?, the author comments:
“The problem was that Google did not know – could not begin to know – what linked the search terms with the spread of flu. Google’s engineers weren’t trying to figure out what caused what. They were merely finding statistical patterns in the data. They cared about correlation rather than causation. This is common in big data analysis. Figuring out what causes what is hard (impossible, some say). Figuring out what is correlated with what is much cheaper and easier. That is why, according to Viktor Mayer-Schönberger and Kenneth Cukier’s book, Big Data, “causality won’t be discarded, but it is being knocked off its pedestal as the primary fountain of meaning”.
And when Google Flu Trends was used in Latin American the results were even worse, correlating (unsurprisingly) to the degree of internet penetration, as well as different cultural practices. In Bolivia where the results were worst, speculation for its failure was based around firstly the low level of internet usage, and the cultural tendency to seek out traditional medicine and practitioners rather than go online to search using flu-related keywords.
There is a growing realization that big data analytics is still plagued by the same sample errors and biases and methodological hurdles of traditional information gathering, and works best when it is “ground-truthed” with these networks.
To some extent this is unfair. As the FT article acknowledged, the Google Data Scientists were not making exclusive claims for Big Data:
“This system is not designed to be a replacement for traditional surveillance networks or supplant the need for laboratory-based diagnosis,” they wrote in Nature magazine. “The data are most useful as a means to spur further investigation and collection of direct measures of disease activity.”
The question then for Big Data for Development is whether the additional cost and burden to harness big data AS WELL AS traditional surveillance networks is justified, or adds enough benefits to cover the costs to often under-resourced health services in the Global South?
As Tim Harford concluded in his article:
“Big data” has arrived, but big insights have not. The challenge now is to solve new problems and gain new answers – without making the same old statistical mistakes on a grander scale than ever.
Despite the methodological failings the belief in the potential remains strong. Google Flu Trends is one of many internet biosurveillance tools, which use population-level trends in Google and other internet search-engine queries about infectious diseases such as dengue, pertussis, influenza or norovirus to detect and predict epidemics.
Mobile phone data, ATM withdrawal data as well as sentiment trawls are being used to map in real time resilience, population movements after disasters, workplace gender discrimination and a vast host of other applications. The possibilities are endless and dizzying. But the question remains; can the Global South afford to learn these lessons through trial and error, and what choices have to be made to scale up on Big Data potential from pilot explorations to national development solutions?