The latest in faith-based development: Randomized control trials?

Lant Pritchett’s fondest 10th birthday wishes to the Poverty Action Lab:

The delightfully quirky aspect of the success of the randomista movement is that it was, and remains, entirely faith-based.

…The claim that attracted resources and support from development organizations and attention from the press was the claim that “rigorous” evidence from these RCTs could, should, and would produce better development projects and policies and, hence, ultimately better outcomes for human beings.

…The randomistas were not proposing new methods or techniques but rather broader adoption into the field of development methods that already had a long history. There was a big fad toward the use of experiments in a variety of social policy domains in the USA in the 1970s.

…Strangely, whether or not decades of social policy RCTs actually did have impact on policies and outcomes in the USA just kind of never came up in arguing that they would in developing countries.

In blogger camp they say “never feed the trolls”, but since Lant is my favorite development troll, I cannot resist. Even if I should be reading someone’s dissertation at the moment.

This week I’m teaching about the politics of foreign aid. Lant’s timing is excellent. I’m having students watch his lecture on how aid is like bad ballet recitals.

His point is this: you need a lot of really, really bad ballet to produce one prima donna. The pinnacle of the art isn’t possible without tens of thousands of 10-year olds doing crappy pirouettes.

As with ballet is development: you need lots of mediocre ideas for the great ones to emerge–the ones that change the world. His example: India in 1992, in the midst of a financial crisis, tips its macroeconomic policy in a liberal direction, helping set the country on a path of unprecedented growth. That one move was, he imagines, partly the result of years of practice, of theory, of trial and error. This one move reduced so much poverty that even if the only thing every development project in history did was make that one moment happen, it was worth it.

Now, the only problem with this story is that Great Moments in Centralized Decision Making Based On Economic Ideologies also brought us hits such as “The Great Leap Forward”. So we have to amend the lesson a little: if all the development projects and research in history brought us slightly closer to tipping the less disastrous way in moments of crisis, then it was all worthwhile.

I can get behind that. And I have to say that some of projects and evidence that make me hopeful about the future include randomized control trials. I don’t expect policymakers to quote the latest trials and suddenly convert to evidence based decisions. But I do expect the evidence to infiltrate the way economics is taught, the way policy is talked about, and the received wisdom. This might take years or decades. Sometimes it will happen quickly, the way that the wind has fallen out of the sails of micro-finance and filled those of cash transfers.

Yes, this is a faith-based belief. So is any prediction of the future. Most of the most interesting trials haven’t been run yet. And it’s always faith based to suppose politicians will put hard facts above, well, politics. But there’s some basis to my hope.

Take this paper by David Weil. He looks at decades of evidence on health interventions and development. The slow accumulation of evidence from many trials suggests that making people healthier might make nations poorer in the very short run (since there’s the same amount of wealth but more people). But in the longer run there are small, positive returns to wages and productivity. These gains are larger than some economists would argue (such as Daron Acemoglu and James Robinson) but smaller than others would have us believe (such as Jeff Sachs and the World Health Organization). Terrific. This is important stuff. And I think it will infiltrate thinking in the field.

When the randomistas run their revivals, some start speaking in tongues and juggling snakes. I think, for the most part, they do this deliberately–because they’re skillful politicians and know that nuanced messages don’t make for policy change. And if a little populist evangelism will get more evidence-based thinking in the world, and tip us marginally further from Great Leaps Forward, I have one thing to say: Hallelujah.

9 thoughts on “The latest in faith-based development: Randomized control trials?

  1. Now, the only problem with this story

    The other problem with that story is that it doesn’t even nearly fit the data. The structural break in Indian GDP per capita growth happens in the mid 1980s, not in 1991. It probably is associated with a degree of liberalisation and tariff reduction, and one can argue either way as to whether the 1990s deregulations were necessary to sustain the momentum of the late 80s. But it’s really troubling that people like Lant Pritchett want to have the morality tale Sunday-school perfect and all the credit allocated to the government they like the most.

  2. Like I do for all expanding faiths, I appreciate the intentions of the leaders and the insight it represents, and their distinction between what they preach and what they privately believe, but I remain somewhat terrified of the zeal of the newly converted who may take snake handling at face value. That’s why it’s helpful to have folks like Lant Pritchett pointing out some of the shortcomings.

  3. Having been a professional ballet dancer and now a development economist, I can’t help but comment.

    The major difference between ballet and development, is that you really don’t learn much from a lot of bad pirouett-ers. You improve by comparing yourself to the very very best (which only can happen if you’re already at the very very top).

    In development, you can at least build on the findings of others no matter where you stand.

  4. A separate argument for health interventions is that improvement in health outcomes can be achieved much much faster than economic growth. I often hear economists say that the way to solve AIDS/malaria/diarrhea/etc. is through growth. But India’s life expectancy today is 70, which the US achieved only in 1980. India today is much poorer than the US was in 1980, but things like newer vaccines more than make up the difference.

  5. As far as this ballet thing: Sure one way to succeed is to just try tons of things and see what works. But it may be possible to be far more efficient and find what really works far sooner, and at far less cost, by using intelligent analysis to cull out the vast majority of bad ideas before you waste a lot of time and resources trying them, or to end the bad tries a lot more quickly.

    Massive trial and error with little thought beforehand is not the only way to find the gems. You can often apply more intelligence beforehand and see what’s very likely to fail and not be worth the cost of trying.

  6. Random controlled trials can be great, but they don’t take the place of intelligent analysis. Like with models, they’re only as good as their interpretation. You have to really think of the details, the specifics, the flaws in the data, and the whole, and of course, the why’s behind the results.