, , , , , , ,

Nobel Laureates Use Randomized Control Testing

Randomized Control Testing
“It can often seem like the problems of global poverty are intractable, but over the course of my lifetime and career, the fraction of the world’s people living in poverty has dropped dramatically.” – Dr. Michael Kremer

In October 2019, Michael Kremer of Harvard and Esther Duflo and Abhijit Banerjee of MIT won the Nobel Prize in Economics for their extensive, randomized control testing-based research in tackling global poverty. At 46 years old, Duflo is the youngest economics laureate ever and only the second woman to receive the prize over its 50-year history.

Incorporating Scientific Studies

The trio set out to establish a more scientific approach to studying the effects of investment projects in the developing world. One of the ways they discovered that they could accomplish this is through randomized control testing. Commonly used in the medical field and made legitimate in the social sciences by the trio, this type of testing involves randomly selecting communities as beneficiaries of experimental projects. Randomly selecting the beneficiaries removes selection bias, providing more accurate and legitimate results.

Randomized Control Testing in India and Kenya

Duflo and Banerjee used randomized control testing experiments in schools in India in an effort to improve the quality of education. The authors discovered that simply getting students to school was not sufficient in improving test scores. Previous research also noted that additional resources, even additional teachers, had minimal impact on students’ performance.

The laureates discovered instead that providing support for an interventionist to work with students behind on their educational skills and making computer-assisted learning available so that all students could have additional math practice improved their scores. In the first year, the average test scores increased by 0.14 standard deviations and in the second year, they increased by 0.28 standard deviations. In the second year, the children initially in the bottom third improved by over 0.4 standard deviations. Those sent for remedial education with the interventionist saw 0.6 standard deviations increase and the computer-assisted learning improved math scores by 0.35 standard deviations in the first year and 0.47 in the second year for all students equally. These results provide clear and definite numbers on the success of the program and show that those who experienced the most benefits were the students in the greatest need of assistance.

Kremer completed a similar study in Kenya. Again, the research found that additional resources did little to improve the learning abilities of the weaker students and that much of the school policies and practices were helpful to the advancement of the already high achieving students. Another of Kremer’s studies in Kenya further showed the impact small interventions can have on student retention. His research found that by bringing deworming medication directly into the classroom, school absenteeism rates decreased by 25 percent, leading to higher secondary school attendance, higher wages and a higher standard of living.

Impact vs. Performance Evaluations

The key to Kremer, Duflo and Banerjee’s success was not the result of pumping out positive statistics. Their success, and reason for winning the Nobel Prize, came from the rigorous scientific approach they took with their studies by using randomized control testing that led to not only positive results but also to meaningful impact where they were working and beyond. For instance, after the success in Kenya with the deworming, the U.S. Agency for International Development (USAID) agreed to finance Kenyan scientists to travel to India to help expand the program. Soon, 150 million children were receiving treatments of deworming medication each year.

This example shows the lasting impact of the work of the laureates. When the fields of economics and politics use more rigorous and randomized studies, it becomes clearer what programs work and which do not, creating greater efficiency and enabling successful projects to expand. The work of the three professors has already led to the leaders of USAID to question the utility of performance evaluations over impact evaluations. In other words, the agency has started to see a shift from success defined as the generated output of the programs to success as the net gain or impact as a direct result of the programs.

Altogether, the work of Kremer, Duflo and Banerjee has raised the bar for economic and social research in the future. Their work has set new expectations that will force researchers to create more detailed and accurate studies that will continue to guide policy.

– Scott Boyce
Photo: Flickr