Philanthropy Action has been asking hard questions about the cancelling of public debate around the Millennium Village projects and the contested claims and evaluation approach. They now have an online petition circulating for a more open debate.
I have mixed feelings about this kerfuffle. I will sign the petition. But I wish we could also focus on the development questions that the big players–the bilateral and multilateral agencies) ought to be asking and why they are not doing it.
My hope is that (in addition to holding a high profile project like the MVs accountable to their claims) this becomes a moment for asking how the research community (i.e. academics like me, or think tanks like CGD, or the research group at the World Bank) can be doing a better job at answering questions relevant to interventions like the Millennium Villages.
2 Responses
Chris,
Thanks for your support (and I’ve added your name to the petition).
There’s a reason that most of the external sources I linked to in regard to MVP evaluation were your posts on the subject–I entirely agree that there are lots of interesting things to measure and learn from in the MVP project that are just limited to direct inputs and outputs.
From my perspective the purpose of encouraging a public debate is to open the door for the broader discussion.
The biggest tragedy of all will be if the lack of public discussion undermines our ability to learn anything from the massive efforts of MVP–not necessarily because of the specific evaluation protocol used but because the evaluation reporting is under a cloud of suspicion because the MVP have refused to publicly discuss pros, cons and alternatives.
Chris this is a great point; yes this is an ideal moment for everyone in the development community to have conversations about what it means to do convincing and useful impact evaluation. Gabriel and I wrote the paper precisely to start such conversations.
If anyone reading this post hasn’t seen Chris Blattman’s thoughts on future directions for impact evaluation, what he calls “Impact Evaluation 2.0“, I highly recommend them. They are the perfect starting point for a discussion of what evaluation needs to become.
Alongside that, I do stand by my claim that the overall impact of a project remains, and should remain, a parameter of central interest to any policy-relevant impact evaluation — simultaneously with other, richer parameters. Aid is and always will be scarce, and it’s important to know whether the impacts claimed for it are real and how they compare in magnitude to other uses of the same resources.