Do development organizations like to hear contrary findings?

With a global email field experiment using 1,419 micro-finance institutions (MFI) as subjects, we test the effects of scientific findings about microfinance on organizations’ willingness to learn more about MFI effectiveness and pursue an offered partnership to randomly evaluate their programs.

In the positive treatment subjects were randomly assigned to receive a summary of a study by prominent authors finding that microcredit is effective. The negative treatment provided information on research – by the same authors using a very similar design – reporting the ineffectiveness of microcredit. We compare both conditions to a control in which no studies were cited.

The positive treatment elicited twice as many responses as the negative treatment – and significantly more acceptances of our invitation to consider partnering on an evaluation of their program – thus suggesting significant confirmation bias among microfinance institutions. The randomization revolution thus faces real challenges in overcoming development organizations’ apparent aversion to learning.

A new paper by Brigham, Findley, Matthias, Petrey and Nielson.

To get a general sense of magnitude: Take up of positive messages is about 10%, and 5% for negative ones. (This is actually higher than I would have expected, for both treatments.)

I wouldn’t have singled out randomized trials. One thing I have noticed in many NGOs: bad news is buried. The number of consultant evaluations that get toned down or shelved when the news is bad is atrocious. Rigorous trials have a big advantage: they are big and expensive and external enough that they can’t be buried. Does this mean that the evidence from trials is the hardest to bury? I would think so.