Peer review, the experiment

We evaluate policies to increase prosocial behavior using a field experiment with 1,500 referees at the Journal of Public Economics. We randomly assign referees to four groups: a control group with a six-week deadline to submit a referee report; a group with a four-week deadline; a cash incentive group rewarded with $100 for meeting the four-week deadline; and a social incentive group in which referees were told that their turnaround times would be publicly posted.

We obtain four sets of results.

First, shorter deadlines reduce the time referees take to submit reports substantially.

Second, cash incentives significantly improve speed, especially in the week before the deadline. Cash payments do not crowd out intrinsic motivation: after the cash treatment ends, referees who received cash incentives are no slower than those in the four-week deadline group.

Third, social incentives have smaller but significant effects on review times and are especially effective among tenured professors, who are less sensitive to deadlines and cash incentives.

Fourth, all the treatments have little or no effect on rates of agreement to review, quality of reports, or review times at other journals.

We conclude that small changes in journals’ policies could substantially expedite peer review at little cost. More generally, price incentives, nudges, and social pressure are effective and complementary methods of increasing prosocial behavior.

A new paper in the JEP by Chetty, Saez and Sandor.

64 Responses

  1. You know, i Think this is all too cool (no kidding, every time I see an IE I’m excited about the simple and rigorous statistics and curious results). But in the end…. Are we in developed countries just having fun with ourselves with all these IEs? This one as an exception happens with (prob) high-income people, but what about the majority? It’s all about the ingenious experiments we can think of in our heads and go and apply in some under developed countries, many times coming to useless and weird conclusions…. Or maybe I’m totally wrong?