The authors attempted to implement randomized experiments to evaluate the impact of seven matching grant programs offered in six African countries, but in each case were unable to complete an experimental evaluation. One critique of randomized experiments is publication bias, whereby only those experiments with “interesting” results get published. The hope is to mitigate this bias by learning from the experiments that never happened.
This paper describes the three main proximate reasons for lack of implementation: continued project delays, politicians not willing to allow random assignment, and low program take-up; and then delves into the underlying causes of these occurring.
Not earth-shaking but important to do and unfortunately unusual that the authors took the time to write up these results.
Kudos to them: Francisco Campos, Aidan Coville, Ana Fernandes, Markus Goldstein, and David McKenzie, all at the World Bank.
15 Responses
I just love the title “Learning from Experiments that Didn’t Happen.” Hilariously misleading
RT @SKalyvas: I knew experimental results suffered from publication and feasibility bias; implementation bias is another issue: http://t.co…
RT @TimHarford: Learning from experiments that didn’t happen http://t.co/mlaAkEfRq8
RT @TimHarford: Learning from experiments that didn’t happen http://t.co/mlaAkEfRq8
RT @TimHarford: Learning from experiments that didn’t happen http://t.co/mlaAkEfRq8
Εxperimental results suffer from publication & feasibility bias; Paper highlights implementation bias http://t.co/Jtj41NEEc9 via @SKalyvas
RT @timharford: Learning from experiments that didn’t happen http://t.co/zyIel1mP1U
Excellent share “@TimHarford: Learning from experiments that didn’t happen http://t.co/F8kGt6o6Ku”
Learning from experiments that didn’t happen http://t.co/mlaAkEfRq8
RT @SKalyvas: I knew experimental results suffered from publication and feasibility bias; implementation bias is another issue: http://t.co…
RT @SKalyvas: I knew experimental results suffered from publication and feasibility bias; implementation bias is another issue: http://t.co…
I knew experimental results suffered from publication and feasibility bias; implementation bias is another issue: http://t.co/fr1ZAXTCEy
RT @cblatts: Learning from experiments that didn’t happen http://t.co/MQXjCdqdLw
RT @cblatts: Learning from experiments that didn’t happen http://t.co/MQXjCdqdLw
RT @cblatts: Learning from experiments that didn’t happen http://t.co/MQXjCdqdLw