Statistics is an essential piece of the policy-making puzzle. From polls to censuses to studies, data is one of politics’ few truths, an impartial measure of the approval or success of any action. In the fight against poverty, the positive results of a policy are often used to justify its implementation in another country, often with mixed results. Although statistics can be incredibly useful for suggesting and tracking anti-poverty efforts, these numbers on poverty carry a series of caveats and assumptions that one cannot overlook.

The Difficulties Behind Statistics

A clear example of this is an experiment run in 1987 by Sally Grantham-McGregor in Jamaica. Looking for a correlation between malnutrition, cognitive development and income, she found that supplementing the meals of malnourished children raised their income over 20 years by 25% compared to the average. Although seemingly indisputable from her evidence from Jamaica, her experiments saw dramatically less success when implemented on a larger scale in Colombia and Peru, with Grantham-McGregor saying that “there is no way to know what caused the impact.”

Much of this uncertainty has to do with the way researchers define terms and the way seemingly unimportant assumptions end up affecting results. A study from 1970 to 2000 focusing on the connection between economic growth and inequality across 26 U.S. states found no correlation, but the correlation between economic growth and inequality of opportunity was extremely strong. Another study by Patrizio Piraino in 2015 found that race-based inequality accounted for around 70% of inequality in South Africa, a figure that is not visible when using the traditional measure of income, GDP per capita, as representative of individuals in a society with a high degree of race-based inequality.

This problem is particularly prevalent in studies on low-income populations. Because questions regarding salary and living situation are sensitive, asking about these topics in questionnaires often yields incorrect data. Low-income groups tend to face underrepresentation in studies, meaning policies meant to alleviate poverty often work with a flawed understanding of the overall improvement objective.

Researchers also present another key difficulty in using statistics in the study of poverty. A study by Lyberg and Kasprzyk in 1991 found that interviewers stood as a large source of error when conducting studies. Language barriers as well as “socioeconomic and demographic characteristics” inhibit accurate responses and interpretations, increasing the margin of error in those studies.

The Benefits of Randomized Studies

However, data science is improving and researchers are beginning to apply the lessons of these studies to improve the accuracy of statistics on poverty. Randomization, or randomly choosing a group as a test for a particular policy, has become an increasingly popular method of policy analysis. A successful example of this process occurred in Indonesia where researchers wanted to test whether identification documents (IDs) carrying information on “program eligibility and entitlements” improved access to a social assistance program. Indeed, researchers found that these social assistance ID cards increased access to Raskin, Indonesia’s rice subsidy initiative that began in 2012. Leaning on the results of this research, the government of Indonesia decided to scale up the distribution of these IDs, reaching more than 15 million poor households in 2013.

Many academics in the field of poverty research remain opposed to randomized studies, but there is an increasingly vocal group advocating gradual implementation based on the past successes of randomization. Particularly in education research, experts have criticized the high costs, overly general results and unorthodoxy of randomization, but policy research has shown it can be effective on a large scale.

The PROGRESA Pilot Program

In Mexico, the social conditional cash transfer program PROGRESA, launched in 1997, is an example of a successful randomized evaluation that informed future policy. PROGRESA first began as a pilot program tested across 506 communities in Mexico. The evaluation showed promising outcomes for children from households receiving conditional cash transfers. As such, the Mexican government decided to implement PROGRESA on a wider scale. Other countries conducted similar randomized studies to evaluate the benefit of “PROGRESA-like programs on their populations before scaling up.” Poverty Action Lab indicates that by the year 2014, 52 nations “had implemented PROGRESA-like programs.”

Each statistical method comes with its own advantages and drawbacks, but researchers are increasingly aware of how to balance the usefulness of statistics with its limitations. By improving the quality of these statistics on poverty, statistics can become an essential part of the reduction in global poverty.

– Samuel Bowles
Photo: Flickr