All those experimental evaluations of development and social programs? It looks like we could be doing our randomizations poorly, or even wrongly. At the very least, we’re not reporting our methods, and we’re not accounting for randomization techniques properly.
That’s the conclusion reached by the World Bank’s Miriam Bruhn and David McKenzie in a new paper. The paper is a must read for program evaluators.
The paper doesn’t deal with the special challenges of place-based evaluations in detail. I recommend this paper by Howard Bloom on the best way to randomize across villages, school, communities and other groups.
These kinds of papers are a huge public good, and it’s a pity there is not more of them. David also has an excellent experimental study on how to measure the profits of small firms in poor countries.
It’s a pity the Bank, the NBER, and the discipline does not invest more in practical research on measurement error, randomization techniques, and other estimation. It strikes me as the same collective action problem that prevents the development of systematic data in developing countries.