Social science is not a real science because it only published positive results?

Joachim Voth points me to a paper that looks for a “hierarchy of science” according to whether or not the discipline publishes null results.

Here’s how a sample of papers perform, by discipline:

Screen Shot 2015-02-04 at 9.46.12 AM

You can quibble with sampling, sample sizes, definitions, etc (and I would) but anyone who has had tremendous difficulty getting a null result published (I have, twice) knows that confirmation and publication bias is alive and well. If they simply got pushed to the good field journals, I could understand, but even there it can be tricky.

Even so, one reason to take the so-called hierarchy of science  with a grain of salt is the following figure:

Screen Shot 2015-02-04 at 9.46.30 AMEverything is driven by “pure” science, meaning (I think) the testing of theories and predictions from very basic science (think theoretical physics). The article is weak on definitions.

So the punchline is that empirical tests of highly theoretical models seldom pan out. Which is basically my experience in economics and political science too.

Even so, the next time you are asked to referee or report on a null result, give it a second chance.

 

 

58 Responses