Tag Archives: Big data

They’re Talking. Are We Listening?

Development agencies and programs around the world are staffed with skilled Monitoring & Evaluation professionals, responsible for tracking a program’s progress against its goals through quantifiable results – think back to the days of creating SMART (specific, measurable, actionable, time-based) goals for your professional evaluations or science class hypotheses. The program I work on, funded by the U.S. Agency for International Development, measures our progress against twelve indicators, which include standard measurements such as number of people trained, number of transactions recorded to track tuna catch, and millions of dollars leveraged from the private sector. While all worthwhile measurements that seek to quantify the program’s impact, does this really capture the full program’s full accomplishments (or failures) in way that reflect beneficiaries personal gains and experiences?

With developments in technology, data collection, and increased use of social media in everyday life, the amount of data is ever increasing, along with professions to analyze and assess this data. How can development tap into this potential?

Development agencies and non-profits are increasingly pushing out information using social media. But, should we be spending more time listening?

In response to America’s current political landscape and reduced funding of government agencies, a recent article in the Boston Business Journal poses how urban social listening can help to determine the impact of aid and social programs. In the study, researchers, “…began by listening to the ‘digital crumbs’ generated by collective searches and postings to social media like Twitter.” Urban social listening offers a, “systematic, rigorous collection and analysis of [social media] ‘crumbs…’ offering useful insights into understanding the government’s role in addressing urban problems” (Hollander and Renski, The Boston Business Journal).

It has been shown that there is an evidenced correlation between positive sentiment and health indicators. In a 2015 study, Twitter sentiment was shown to be a better predictor of cardiovascular disease than any conventional indicator like smoking or socioeconomics. Hollander, author of the Journal article, applies urban listening techniques to U.S. Department of Housing and Urban Development’s Community Development Block Grants (CDBG). While he wasn’t able to find significant improved sentiment in cities with CDBG funding, it seems that the community scale being analyzed is too large to yield highly correlated results.

In development, non-profits and government agencies alike are increasingly using social media to communicate with stakeholders and beneficiaries, but listening is a newer trend. Payal Arora and Nimmi Rangaswamy echo this in Digital Leisure for Development: Reframing New Media Practice in the Global South. “…Research is driven by development agendas with a strong historical bias towards towards the socio-economic focus. Data that does not directly address project-based outcomes is side-lined.”

Tracking sentiment through social media is not a new concept, used for years in consumer confidence and political opinion. While certainly more people are tweeting on popular political opinion topics, how can we the development world harness and tailor these listening practices to hear our beneficiaries, and through listening show that their voice counts.

Arora, Payal and Nimmi Rangaswamy. Digital Leisure for Development: Reframing New Media Practice in the Global South. Sage Journals. October 10, 2013.

Hollander, Justin. Viewpoint: As Ben Carson Hearings Get Under Way, Can Social Media Help Identify Hud Impact? The Boston Business Journal, 2 March 2017. http://www.bizjournals.com/boston/news/2017/03/02/viewpoint-as-ben-carson-hearings-get-under-way-can.html

Eichstaedt, Johannes, et al. Psychological Language on Twitter Predicts County-Level Heart Disease Mortality. Sage Journals. Volume: 26 issue. Pages 159-169. Published January 20, 2015. http://journals.sagepub.com/doi/abs/10.1177/0956797614557867

Big Data Catches: Furthering Development, not the Divide

Women in the port of Bitung, Indonesia prepare the day’s catch for market. 

The global fishing industry is a multi-billion dollar industry, with a recent report released by Stratistics MRC valuing it at $239.8 billion in 2015 — and projecting it to reach $320 billion by 2022. Southeast Asia is one of the largest contributors to this market, accounting for more than 50% of the world catch and with the largest concentration of fishing vessels in the world– over 3 million. More than just an economic powerhouse and one of the globe’s biggest sources of protein, Southeast Asia’s fisheries employ 93% of all fishery and aquaculture employees worldwide, and 10% of the world’s total working population.

The industry, while growing and a profitable employer of hundreds of millions, is also a large developmental concern. Its exponential growth and increased demands that it enjoys negatively impact the health of valuable ecosystems, marine biodiversity, and threaten already dwindling fish stocks. In addition to the environmental impacts, the region is rife with illegal, unreported, and unregulated (IUU) fishing – an issue that is gaining traction in development both because of its environmental impacts, but also because of its serious human welfare implications.

“Data has become increasingly important to the way we think and talk about conflict and our humanitarian responses to it” (Róisín Read, Bertrand Taithe and Roger Mac Ginty). In a world where technology is often looked to to address human problems, how do these issues relate to data and development? A number of non-governmental, governmental, private sector, and industry players are increasingly looking to data to gather information on an industry that, despite its wealth, is incredibly challenged by tracking and sourcing its products, far behind other food sector’s traceability capabilities.

With new U.S. regulations that will require a set of standard data to be submitted with every seafood import to enhance transparency (released in December 2016 and going into effect in January 2018) also serving as a catalyst, fisherfolk and industry must quickly get aboard supply chain data collection efforts, and figure out how to implement these systems in their operations. Data collection is challenged by limited connectivity at sea, hesitancy to invest, unwillingness to share proprietary data (such as location of catch), and complex supply chains.

So, how can development organizations address this challenge to increase sustainability, protect finite marine resources, and address serious human welfare concerns that include limited labor rights, forced labor, and slavery? While more data can equal more intelligent decision making, transparency, and the development of effective policy and regulation, it can also leave beneficiaries behind – often the most critical beneficiaries, those that have sole-source incomes, limited access to technology, and depend on the sector for their livelihood.

Not only do large scale commercial operations need to be addressed, but small-scale fishers – those that often catch only a few kilograms of fish per day and is their sole source of income. How can large scale operators and artisanal fishers, living in small coastal fishing villages comply with regulations and uniformly collect data in a way that doesn’t push small, independent fishers out of the supply chain? With the proliferation of big data, how can development take care to further development goals with technology, and not further segment society into groups that are more or less likely to adopt data technology? “…The way information technology [has] operated in the sector [is] equivalent to ‘buying a state of the art car, driving it into the desert and leaving it there’. More and more money is invested in developing these technologies but their use is often limited, driven not by a clear sense of what is needed to improve response, but by what the advances in technology enable.”



Stratistics MRC. Commercial Fishing Industry – Global Market Outlook (2016-2022). January 2017. http://www.satprnews.com/2017/03/03/commercial-fishing-industry-market-size-share-analysis-report-and-forecast-to-2022/

Global Implications of Illegal, Unreported, and Unregulated (IUU) Fishing. National Intelligence Council. 19 September 2016. http://www.iuufishing.noaa.gov/RecommendationsandActions/RECOMMENDATION1415/FinalRuleTraceability.aspx
Third World Quarterly, 2016  Data hubris? Humanitarian information systems and the mirage of technology Róisín Read, Bertrand Taithe and Roger Mac Ginty Humanitarian and Conflict Response Institute, University of Manchester, UK. http://dx.doi.org/10.1080/01436597.2015.1136208

Big Data in Crises: Predicting the Future

Little ‘Data’, aNto on Flickr CC by A

Gathering, processing and interpreting data sets is what runs the modern global economy. Everything from your weekly online supermarket grocery shop, to how the shelves get stacked with goods delivered by cargo ships from all across the world. Teams of statisticians make their daily bread from finding more and more sophisticated ways of predicting human behaviour.

Complex mathematical modelling is also being used in humanitarian emergency situations to prevent further loss of life. I’m not just referring to how aid is delivered through more efficient supply chains, but to how the mapping of crises and outbreaks of diseases is used to instigate the correct response and predict how the spread is evolving.

Dr Sebastian Funk at the London School of Hygiene and Tropical medicine is a leader in the field and was at the centre of important work to map the Ebola outbreak in West Africa. By processing data collected from the affected area Dr Funk and peers across the world were able to look almost 6 weeks into the future, with 95% confidence of the first week.

Finding a ‘magic bullet’ or key to suppressing an outbreak is time sensitive – one must collect enough quality data to make sure that the models can be accurate, but when people’s lives are at stake conclusions need be drawn quickly.

It was found that ‘most people infected with the deadly virus became ill through contact with a small number of so-called ‘superspreaders’ and ‘if superspreading had been under control, about two-thirds of Ebola cases could have been avoided’.

Here he is speaking to RFI’s English service

Dr Funk’s work was reliant on effective feedback on the ground. He knew that whilst cases might be dropping off, there are unreported areas and if the superspreaders had been identified earlier, lives could have been saved.

The unprecedented rise of smartphone and social media has changed the data landscape. Agencies can have access to crowd-sourced information, which taken together can be highly accurate.

This was already happening during the Ebola outbreak – ‘When epidemiological data are scarce, social media and Internet reports can be reliable tools for forecasting infectious disease outbreaks’, from findings published in the Journal of Infectious Diseases.

Patrick Meier advocates the use of geographically mapping social media posts –  in Digital Humanitarians: how BIG DATA is changing the face of humanitarian response he describes crisis mapping of the 2010 Haitian earthquake and Libyan political turmoil in 2011 known as the Arab Spring. This approach allows aid workers to have a more complete live overview of the situation that’s constantly updated.

Libya Crisis Map Deployment 2011 Report

The crowd-sourced reports are collated by a volunteer team that work 24hrs a day, corroborating information from social media sources. It led to a significant change in the response, something which was praised by UNOCHA, if not without some reservations.

In this case, the role of the human as an arbiter of trustworthiness remains a significant undertaking. Even with pleas like: ‘we just need to make sure you’re not Gaddafi!…we are not Facebook!’ (Meier, p.125) for people to declare their background information, the digital gathering of sources, even on a big scale, still has to have an element of journalistic rigour.

The future will be to use artificial intelligence to perform checks that people simply don’t have the capacity to do, given the volume of information coming in. Systems are being developed that analyse qualitative rather than quantitative qualities of posts, allowing computers to detect false rumours and unrelated background noise.

With accurate models, the prediction capabilities in future outbreaks of disease or disaster will certainly be enhanced, leading to an untold impact on lives saved.

Leave your comments or tweet @data_bigly if you want to join the conversation.

References. Links in text plus:

Mier, Patrick. Digital Humanitarians: How BIG DATA Is Changing the Face of Humanitarian Response CRC Press, London. 2015