We meta-analyze enrollment, attendance and dropout impact and cost-effectiveness estimates from forty-two CCT program evaluations in fifteen developing countries.
Average impacts and cost-effectiveness estimates for all outcomes in primary and secondary schooling are statistically different from zero, with considerable heterogeneity.
CCT programs are, all else constant, most impactful and cost-effective for programs that, in addition to transfers to families, also provide supply-side complements — such as infrastructure or additional teachers. Impacts are also larger in programs with infrequent payments and more stringent schooling conditions, which aligns with previous single program evidence.
Impact and cost-effectiveness estimates from randomized research designs are smaller than those from observational studies.
A new paper from Saavedra and Garcia.
In 2008 I predicted that the randomized revolution was going to disappoint, and that meta-analyses like this one would prove inconclusive, mostly because differences in context and program choices would outweigh any systematic effect of the programs. Looks like I was wrong (at least in the CCT domain).
Some are still skeptical. I’m curious how Sandefur and Pritchett will react.
6 Responses
Hi, great to see your website. I like the content and the research done behind every aspect of your blog. It looks great and very knowledgeable. Keep it up the good work.
The thing that is striking here is there is no mention of learning outcomes. So we pay people to go to school, and they go to school. But if they learn nothing when there, then this might be a sub-optimal policy. Most of the cct’s i’ve seen on are mixed about learning, even after controlling for changes in samples. You get what you incentivise – i think we need to be cleverer about spelling out what we really want from education programs
Paul, this is exactly consistent with the recent 3ie meta-analysis on access versus learning: CCTs are good for enrollment but not mixed for learning. I think there is a difficult balancing act, though. In a Tanzania CCT that I evaluated, and qualitative work from a CCT evaluation in northern Nigeria (Habyarimana & Sabarwal), the sense is that more complicated conditions are much, much harder to communicate. Must be in school: Easy to communicate. Must attend 80% of the time: More difficult to communicate. I suspect much of what we need to do to improve learning will come from the supply side (as both the 3ie and the McEwan review suggest).
Lant,
You wouldn’t believe how difficult it is to get that very point across to audiences interested in this question. I’ve been trying for the past few years – with little success…Even with the benefit of both straightforward theory and lots of recent evidence, I’d say that many people would still argue that there is no consensus.
Chris, I assumed you never expected, in 2008 or any other time, there not be to a consensus on the question “if we give people lots more money to do X will they do more X?”
Patrick McEwan recently put out a meta-analysis of RCTs with student learning outcomes in low and middle income countries with some conclusive results: http://academics.wellesley.edu/Economics/mcewan/PDF/meta.pdf. (Summarized at http://blogs.worldbank.org/impactevaluations/finally-what-works-so-far-improve-learning-developing-countries.)
So I would add that to the (so far) short list of domains with informative meta-analyses.