Joachim Voth points me to a paper that looks for a “hierarchy of science” according to whether or not the discipline publishes null results.
Here’s how a sample of papers perform, by discipline:
You can quibble with sampling, sample sizes, definitions, etc (and I would) but anyone who has had tremendous difficulty getting a null result published (I have, twice) knows that confirmation and publication bias is alive and well. If they simply got pushed to the good field journals, I could understand, but even there it can be tricky.
Even so, one reason to take the so-called hierarchy of science with a grain of salt is the following figure:
Everything is driven by “pure” science, meaning (I think) the testing of theories and predictions from very basic science (think theoretical physics). The article is weak on definitions.
So the punchline is that empirical tests of highly theoretical models seldom pan out. Which is basically my experience in economics and political science too.
Even so, the next time you are asked to referee or report on a null result, give it a second chance.
58 Responses
RT @TimHarford: ‘Publication bias is alive and well’ says @cblatts http://t.co/cd1hTRakT3
Social science is not a real science because it only published positive results? http://t.co/Anv0Q205Vb
RT @BoraZ: Social science is not a real science because it only published positive results? http://t.co/ycoyWEMJPU
Social science is not a real science because it only published positive results? http://t.co/8aNNFTI3Cb
Social science is not a real science because it only published positive results? http://t.co/bUBvYFRb1z
RT @neuroecology: The hierarchy of science via @cblatts http://t.co/HGZEcgNwYs http://t.co/cqJ0inEPus
Social science is not a real science because it only published positive results? http://t.co/X7aL32ATV4
Social science is not a real science because it only published positive results? http://t.co/SOtnWhNbi2
We’re in Plato’s Cave if neg results stay concealed. “@TimHarford:’Publication bias is alive and well’ says @cblatts http://t.co/3pc5VIvOoO“
RT @TimHarford: ‘Publication bias is alive and well’ says @cblatts http://t.co/cd1hTRakT3
@TimHarford Why haven’t you linked to any of the papers which says that publication bias is a thing of the past? @cblatts
RT @TimHarford: ‘Publication bias is alive and well’ says @cblatts http://t.co/cd1hTRakT3
RT @TimHarford: ‘Publication bias is alive and well’ says @cblatts http://t.co/cd1hTRakT3
‘Publication bias is alive and well’ says @cblatts http://t.co/cd1hTRakT3
RT @neuroecology: The hierarchy of science via @cblatts http://t.co/HGZEcgNwYs http://t.co/cqJ0inEPus
Hum? Social sciences rarely publish null results http://t.co/1DEFFjjokm
RT @neuroecology: The hierarchy of science via @cblatts http://t.co/HGZEcgNwYs http://t.co/cqJ0inEPus
RT @neuroecology: The hierarchy of science via @cblatts http://t.co/HGZEcgNwYs http://t.co/cqJ0inEPus
publish null results! (bonus: be a “real” science) http://t.co/IuyvLNBC7z via @cblatts via Joachim Voth via Daniele Fanelli cc @3ieNews
RT @neuroecology: The hierarchy of science via @cblatts http://t.co/HGZEcgNwYs http://t.co/cqJ0inEPus
RT @neuroecology: The hierarchy of science via @cblatts http://t.co/HGZEcgNwYs http://t.co/cqJ0inEPus
RT @neuroecology: The hierarchy of science via @cblatts http://t.co/HGZEcgNwYs http://t.co/cqJ0inEPus
RT @neuroecology: The hierarchy of science via @cblatts http://t.co/HGZEcgNwYs http://t.co/cqJ0inEPus
RT @neuroecology: The hierarchy of science via @cblatts http://t.co/HGZEcgNwYs http://t.co/cqJ0inEPus
RT @neuroecology: The hierarchy of science via @cblatts http://t.co/HGZEcgNwYs http://t.co/cqJ0inEPus
RT @neuroecology: The hierarchy of science via @cblatts http://t.co/HGZEcgNwYs http://t.co/cqJ0inEPus
RT @neuroecology: The hierarchy of science via @cblatts http://t.co/HGZEcgNwYs http://t.co/cqJ0inEPus
The hierarchy of science via @cblatts http://t.co/HGZEcgNwYs http://t.co/cqJ0inEPus
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
I’d say… money is the motive. There’s probably (read: definitely) a relationship between “type of scientific research receiving funding” and “type of scientific research deemed… scientific” lol! Henry Miller would likely agree… https://www.project-syndicate.org/login?redirect=%2fcommentary%2fscientific-research-investment-applications-by-henry-i–miller-2015-01 and that there are more “pure”
RT @DrewHerrick: Social science is not a real science because it only published positive results? http://t.co/0nQlgDPrnb
Social science is not a real science because it only published positive results? http://t.co/0nQlgDPrnb
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @MarkThoma: Social science is not a real science because it only published positive results? – Chris Blattman http://t.co/s3fyCCNn3r
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
Social science is not a real science because it only published positive results? – Chris Blattman http://t.co/wvbiRENIGF
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @MarkThoma: Social science is not a real science because it only published positive results? – Chris Blattman http://t.co/s3fyCCNn3r
Social science is not a real science because it only published positive results? – Chris Blattman http://t.co/s3fyCCNn3r
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9
RT @cblatts: Social science is not a real science because it only published positive results? http://t.co/1dXCbHiDe9