One of the hardest parts of instituting public health policies that will benefit the poor is knowing beforehand whether or not they will work. After all, donors don’t want to waste money on ineffective programs, and those in the developing world don’t want to be troubled by policies that don’t positively impact their lives.

What if there was a way to determine the effectiveness of a policy before executing it? The answer might just be a technique called randomized controlled trials (RCTs).

RCTs are economics techniques that have been around since the 1960s, used to test the effectiveness of different social programs and economic policies.

The premise is simple enough: institute a scaled-down version of your policy over a random area, and compare it to another randomly assigned control group that does not benefit from the policy. While not a novel idea, randomized controlled trials only gained notoriety relatively recently in 1997 in Mexico.

In the midst of an economic crisis, Mexican President Zedillo tasked economist Santiago Levy with devising a strategy to alleviate the burden on the poor. Traditionally, the Mexican government had subsidized the cost of food or basic services like electricity.

Instead, Levy introduced a system of cash incentives to families who met requirements such as ensuring school attendance and visiting health clinics. Because the program rolled out in stages, Levy was able to collect data on regions who had the program, and those who hadn’t yet implemented it, providing him with control groups.

The program, called PROGRESA, was highly successful, with visitation to health clinics 60 percent higher for impacted populations versus control groups. Levy was also able to measure improved health outcomes for children, with a rate of illness decreasing by 23 percent, and an 18 percent reduction in anemia.

His experiment began an era of testing and collecting data on social programs where previously policy-makers could only guess at their effectiveness.

Since then, RCTs have been used to test the effectiveness of a wide range of policies. They have become a favorite tool of development professionals but have perhaps seen the most use for public health programs. In 2000, there were only about 10 public health studies published using RCTs as the primary method of analysis, while by 2012 there were over 200.

Ann Mei Chang, executive director of the Global Development Lab for the United States Agency for International Development (USAID), commented that RCTs are “front and center on a lot of people’s agenda.”

Explaining why RCTs have become so important to aid donors, she asks, “where do we get the biggest bang for our buck?” RCTs are designed precisely to answer that question.

In Haryana, India, RCTs are being used to test the effectiveness of an incentive-based health program. 70 local health clinics were authorized to distribute sugar and cooking oil to families whose children start and finish a round of immunizations.

The goal is to determine whether or not incentives can increase immunization rates in the poorest districts. If incentives are proven to work through the randomized controlled trials, the government can begin to scale up the program to cover the entire state, if necessary.

However, not everyone wants to board the RCT train. Critics of the technique say that focusing on small metrics of success, such as whether or not incentives in-kind work to improve public health outcomes, detracts from the larger macroeconomic picture.

For example, while the results of the Haryana study seem to indicate that incentives improve vaccination rates, variables such as availability of medical personnel and supplies, geographic disease burdens, and income inequality would tend to confound the results.

Supposedly the fact that the trials are randomly assigned would control for these variables, but macroeconomists tend to disagree, preferring “Big Push” solutions to development issues.

As Princeton economist Angus Deaton puts it, “development is ultimately about politics,” meaning that the RCTs can only evaluate individual programs, rather than wider trends such as corruption, trade, infrastructure, etc.

RCTs aren’t the only evaluative technique for public health programs; randomization isn’t always desirable or necessary, and in a health crisis, preliminary testing takes too much time.

However, RCTs do represent a valuable tool for developing countries and aid donors alike to measure the efficacy of incentive-based programs and health policies. When it comes to public health, every dollar counts, and knowing policies work before scaling them up ensures that aid resources are used as effectively as possible.

– Derek Marion

Sources: Nature, The Economist
Photo: Wikimedia