The difficulty of generalizing RCT results (the RCT result)

The recent wave of randomized trials in development economics has provoked criticisms regarding external validity and the neglect of political economy. We investigate these concerns in a randomized trial designed to assess the prospects for scaling-up a contract teacher intervention in Kenya, previously shown to raise test scores for primary students in Western Kenya and various locations in India. The intervention was implemented in parallel in all eight Kenyan provinces by a non-governmental organization (NGO) and the Kenyan government. Institutional di fferences had large e ffects on contract teacher performance. We nd a signi cant, positive e ffect of 0.19 standard deviations on math and English scores in schools randomly assigned to NGO implementation, and zero e ffect in schools receiving contract teachers from the Ministry of Education. We discuss political economy factors underlying this disparity, and suggest the need for future work on scaling up proven interventions to work within public sector institutions.

A new paper by Bold, Kimenyi, Mwabu, Ng’ang’a and Sandefur.