That quote is from psychologist Richard Nisbett, on the tragedy of not enough experimental studies of social programs like Head Start. (Hat tip to Paul Lagunes.)

Regular blog readers will not need to be convinced of this point (and will even think, as I do, that the experimental fad is getting carried a little too far). But I mention this quote for two reasons:

1. It is a particularly elegant phrase in the plea for more evidence of any kind
2. I happened to have spent the last couple of weeks reading a bunch of psychology meta-analyses, a field that (unlike the shameful state of economics or political science) actually has enough experiments and evidence to run them. But I have noticed a number of other tragedies of evidence inflicting a high price on society:
   1. The huge number of social experiments with samples of a 100 people or less. The famous Perry preschool study had only 123 pupils! Ridiculous and hardly better than garbage, at least without a meta-analysis.
   2. The huge reliance on just one or two outcomes available in administrative data, rather than going down the expensive, difficult, but rewarding route of tracking people down and convincing them to talk to you.
   3. Few studies come from a repressentative population, with tons of unknown selection into the sample. So it’s unclear how to generalize.

This is all to say we often speed past one tragedy of evidence only to come to a screeching halt at the next one, randomizing doesn’t teach the data to sing in perfect harmony.