He makes many excellent points on the new World Bank impact evaluation blog.
One point: The sum of individual program impacts never equals the impact of the whole package. (The optimists think the whole is greater than the sum of its parts. I am less certain.)
But a more important point, I think: We’re selective in which programs we evaluate, and ten towards the decentralized, uncoordinated, and less sensitive programs.
I call it the “non-random impact of random impacts”. Tyler Cowen has expressed similar sentiments.
Here’s what worries me about the current path: the advice is heavily biased to the evaluations we’ve done. And big donors listen. The risk: we attack the causes of poverty we can see under the light of the lamp post, and pay less attention to the roots under cover of darkness.
Better than no lamp post at all? Probably. Maybe. But not assuredly. My reasons why will have to wait for another post when I don’t have a deadline looming.