Chris Blattman

Chris Blattman

Twitter Facebook-f Envelope Rss
Search
Close this search box.

“Risk and Evidence of Bias in Randomized Controlled Trials in Economics”

A new paper by Eble and Boone:

The randomized controlled trial (RCT) has been a heavily utilized research tool in medicine for over 60 years. It has enjoyed recent popularity in the social sciences, where it has been used to evaluate questions of both policy and theory.

The early economics literature on RCTs invokes the medical literature; but seems to ignore a large body of this literature which studies the past mistakes of medical trialists and links poor trial design, conduct, and reporting to exaggerated estimates of treatment effects.

Using a few consensus documents on these issues from the medical literature, we design a tool to evaluate adequacy of reporting and risk of bias in RCT reports. We then use this tool to evaluate all reports of RCTs published in a set of 52 major economics journals between 2001 and 2011, alongside a sample of reports of RCTs published in medical journals over the same time period.

We find that economics RCTs fall far short of the recommendations for reporting and conduct put forth in the medical literature, while medical trials stick fairly close to them, suggesting the risk of exaggerated treatment effects in the economics literature.

The medical sciences are a little too rabid and narrow-minded about a number of questionably important practices, but given that most of these practices are low cost and potentially high benefit, and since some of them are quite important, point taken.

h/t Don Green

20 Responses

Why We Fight - Book Cover
Learn More
Subscribe to Blog

Recent Posts