He makes many excellent points on the new World Bank impact evaluation blog.
One point: The sum of individual program impacts never equals the impact of the whole package. (The optimists think the whole is greater than the sum of its parts. I am less certain.)
But a more important point, I think: We’re selective in which programs we evaluate, and ten towards the decentralized, uncoordinated, and less sensitive programs.
I call it the “non-random impact of random impacts”. Tyler Cowen has expressed similar sentiments.
Here’s what worries me about the current path: the advice is heavily biased to the evaluations we’ve done. And big donors listen. The risk: we attack the causes of poverty we can see under the light of the lamp post, and pay less attention to the roots under cover of darkness.
Better than no lamp post at all? Probably. Maybe. But not assuredly. My reasons why will have to wait for another post when I don’t have a deadline looming.
4 Responses
It’s been 60 years since the great empires of the world started coming apart after WWII. If there isn’t adequate money for stats by now, it’s because it’s more profitable not to know what the results are than it is to know.
David McKenzie and I have separate posts in response, both which can be found at: http://blogs.worldbank.org/impactevaluations/
Michael makes an excellent point, I am very impressed with the high amount of statistics (authenticated?) coming from Africa. I am not sure that the currently available statistics are entirely reliable, but if that’s all we have to work on then we don’t have a choice!
Development agencies should definitely put in place a solid framework to obtain better statistical data on developing regions.
It is very difficult to measure the impact of programs without thorough surveying processes and a large enough sample population.
Thanks
This is a tempest in a teapot. Wake me when the IMF or USAID starts funding adequate statistical agencies in the countries we’re aiding. Until that happens, we’re just trying to measure the unmeasurable, in pursuit of talking points to reach the disinterested.