The mistakes made by most development reformers

Dani Rodrik is interviewed:

Suppose you’re in a setting where the rule of law and contract enforcement are really weak. And you realize that they don’t change overnight. Are you better off promoting the set of policies that presume that rule of law and contract enforcement will take care of themselves, or are you better off recommending a strategy that optimizes against the background of a weak rule of law? And I say that the evidence is that you do much better when you do the second.

The best example is China. Its growth experience is full of these second-best strategies, which take into account that they have, in many areas, weak institutions and a weak judicial system, and therefore they couldn’t move directly to the kinds of property rights we have in Europe and the United States. And yet they’ve managed to provide incentives and generate export-orientation in ways that are very different from how we would have said they ought to have done it, which would have been to simply open up their economy or privatize their enterprises. There, second-best strategies have been very effective. The same can be said of Vietnam, say, or farther afield, a country like Mauritius.

The full interview is interesting. Hat tip to the Development Impact blog.

Understand the world you live in. Think about the politics of reform. These are good points. It’s kind of amazing they need to be said out loud as news, but people do need reminders.

One trouble I have is that I think even very smart and experienced people are profoundly bad at knowing what the problems are in the economy, where the political winds are blowing, and what will work. This needs to be said out loud as well.

To take an example from a smaller scale: I spend a lot of time studying local labor markets in Africa, especially when people opt for crime or mercenary work rather than farming and business. I try to figure out what holds back legal work and test programs that deliver those things: skills, capital, socialization, and so on. And I get it wrong almost every time.

What I mean is that the experiments never end like I expect them to. Even (maybe especially) when they work out. I was blindsided by how frequently the poorest young men in slums of Nairobi have a home robbery or theft, meaning it’s almost impossible to accumulate capital. I was amazed that, yes, with a little skills and capital that a young woman can become the 183rd tailor in her community and turn a good profit.

This isn’t a defeatist point of view. I’d make a different point: the way I’ve learned how things operate is to work with a government or organization to try out a policy and succeed or fail. This kind of trial and error seems crucial to me. Karl Popper called this the piecemeal social engineer. Deng Xiaoping called it crossing the river by feeling each stone.

This sounds like a good way to figure out the way your world works (your model), and then to reform. A lot of people would say this is China’s secret to success: informal experimentation on a grand scale. The problem, as I see it, is that most governments and aid organizations I’ve worked with are really, really bad at this. They don’t use the lessons from past failures to try again a different, better way. They don’t throw out bad programs.

I think Dani would agree. He pushes policymakers to do more diagnosis before acting, to figure out the right model. But as the LSE economist John Sutton once said (in a truly terrific lecture), implementation trumps diagnosis. I think the best diagnostic is probably trial and error.

In any case, to me the important question is not “what is the right policy?”, but “what is the process for generating good policies over time?”, and more importantly “how to get governments and aid organizations to adapt to the good and throw out the bad?”.

I don’t know a good answer. To me, this is what makes most development aid and planning not just fruitless but downright dangerous.

113 Responses

  1. I agree with Metatone – There are serious limits to the process of adaptation or “muddling through.” For example, consider the US strategy for the Viet Nam war. Also, in some sense all of our policy strategies are adaptive (just depends on the time scale). One mistake after another does not necessarily lead to success. Especially if the situation is evolving faster than the policy. So – when it comes to developing better policy, “never give up.”

    We know that looking for the facts is necessary, but it is not sufficient for developing an effective policy. A new approach to analysis is focused on using the logic-structure of the policy model. Sort of the Yin to the data’s Yang. “Reality” cannot be found by looking only at data. We live in a systemic universe (social systems, ecosystems, etc.). So, what we need are policy models that reflect the systemic nature of our world.

    Essentially, we can measure “how systemic” a policy model is. This new approach can be used to predict the potential effectiveness or usefulness of a policy model (assuming the model is implemented according to plan – a bit “if” but that is another story).

    For more info, see our white papers at: http://scipolicy.org/principles-of-ipa—white-papers.html

    Thanks,

    Steve
    swallis@meaningfulevidence.com

  2. This is an interesting premise from my point of view not so much because trial and error, or experimentation, or most strategies don’t survive their initial contact with reality, but because I think it indicates a substantial paradigm shift in the world. I grant you what I am about to write is not well thought out because it is a fairly new paradigm I have been working with but please hear me out.

    The European experience is based on reducing the uncertainties of life for the vast majority of the population by establishing rules (politics, economics, laws, international relations, and so on) based on productive property and the returns you can get on it. Never mind who owns it or even who gets the benefits of that productivity; it is still property-based. However, in the last century or so, with the exception of a few pockets of very localized conflict, most of the non-European world has the caught up with basic standards of living in the West without creating systems based on property. Even though we, in the West, do not wish to adapt our worldview to this reality it seems to be a fact that needs to be reckoned with and which has significant consequences for us moving forward.

    Possibly, this is why policies and programs do not produce the expected results?

  4. I think there is a bit of defeatism in your analysis and, if you’ll forgive the challenge, a bit of arrogance. Trial and error is always useful and important, esp. in complex systems – however, I’m always wary of those who want to rely on it exclusively.

    The challenge I’d put to you is: can you develop better diagnosis?
    I’d suggest it’s a bit defeatist (giving up too easy) to say no.
    The arrogance of saying you can’t develop better diagnosis is in the idea that PoliSci and Int. Affairs are a good summation of the knowledge and techniques you could use for diagnosis. This is blatantly not the case – there are whole arenas of useful knowledge (from the histories of the Industrial Revolution in Europe, through to ethnographies and cultural studies of Kenya) that could have helped you in Nairobi… My challenge to you is that while you continue to trial and error, actually spread your horizons and find some of the information that would help you assemble better trials…

  5. Thanks for this post. I hope some funders read this because most of the current funding mechanisms do not allow implementation organizations to apply the trial and error approach. In health, I think it is quite obvious now that striving for the highest quality of care yields nothing that solves the dire needs in the here and now – there are many ‘second best’ options that can provide incremental improvements.

  6. Interesting post. And a very good point. But, I think there are limits to applying second-best strategies. It reminds me of the debate about corruption: does it grease the wheels or sand the wheels of growth in poor institutional settings. Justifying corruption because it greases the wheels is a second-best solution, but a case in which it might be best to aim for the best solution instead.

  7. I think this is 100% right as applied to governments reforming their domestic economies, but it’s not a particularly helpful recommendation for international development organizations. The problem is that the process of experimentation, of trial and error, must carry actual consequences. The reformer has to have real skin in the game. A World Bank official can recommend experimental policies, but from his position it’s academic. He’ll be gone from the country in a few years anyway, and the success or failure of the policy won’t affect his job security in the least. From his perspective, there are more important things to worry about, like making sure tightening budgets at the Bank don’t reduce his breakfast allowance for force him to fly coach.