….unfortunately, there have been far too many cases where the quantitative analysis of those numbers has been flawed, causing doubt about the authors’ interpretation and uncertainty about the result. Furthermore, it is not realistic to expect that a technical reviewer, chosen for her or his expertise in the topical subject matter or experimental protocol, will also be an expert in data analysis.

For that reason, with much help from the American Statistical Association, Science has established, effective 1 July 2014, a Statistical Board of Reviewing Editors (SBoRE), consisting of experts in various aspects of statistics and data analysis, to provide better oversight of the interpretation of observational data.

…I have been amazed at how many scientists have never considered that their data might be presented with bias. There are fundamental truths that may be missed when bias is unintentionally overlooked, or worse yet, when data are “massaged.” Especially as we enter an era of “big data,” we should raise the bar ever higher in scrutinizing the analyses that take us from observations to understanding.

This is an important move. I would love to see medical journals do the same, where I think the problems are greater and the consequences for human welfare more immediate.

At the same time, if your research mainly deals with numbers, then I think it *is* time to expect people with substantive expertise to become better statisticians. They need this not only to produce better work, but to be effective users of what their peers produce. This cannot simply be exported to a committee in the top journal.

Raising the refereeing bar is going to get the incentives right, which is a step in the right direction. But something will need to change in graduate admissions requirements and training.

In particular, I think that a 21st century undergraduate degree in social science ought to require fluency in statistics. It’s such a fundamental part of science, medicine, social science, and even reading the newspaper. But even the Columbia’s and Yale’s of the world don’t impress this on their undergraduates, let alone require it. I’m a big supporter of the liberal arts education, but on the margin I’d substitute a couple courses in the humanities for statistics and causal inference.

@cblatts @BrendanNyhan read this as “statistical bear”, got really excited

. @cblatts would substitute a couple courses in the humanities for statistics and causal inference. http://t.co/wsot98ixcB

I believe that many social science programs require introductory stats and a course in research methods. While this may make for better newspaper readers I wouldn’t suggest that it helps those students make sense of academic works employing quantitative methodology. A major assumption in your argument Prof. Blattman is that students will have the math background to be able to progress in stats at the graduate level.

RT @cblatts: Fixing social science’s statistical problem http://t.co/OXbZMbkyum

RT @chrisalbon: “I think it is time to expect people with substantive expertise to become better statisticians.” – @cblatts. THIS. http://t…

Fully agree with the last paragrah. Science Magazine raises its statistical bar. Will we? http://t.co/uHSQKgbY39 via @cblatts

Science Magazine raises its statistical bar. Will we? – Chris Blattman http://t.co/2oIM9sjtxa

I agree with this. One amendment: math and stats qualify as traditional liberal arts subjects in my view

Science Magazine raises its statistical bar. Will we? http://t.co/4L7xuhvdb2 via @cblatts

RT @peteraallen: ‘a C.21st undergrad degree in social science ought to require fluency in statistics’ – great post from @cblatts – http://t…

So very true! “@cblatts: Fixing social science’s statistical problem http://t.co/gAXo1pK9Ot”

RT @PaulGowder @cblatts: Social science & stats – need v wide fluency/ability to engage, not all need hands-on skills http://t.co/rfhn98rgc5

[email protected] partners with ASA to add additional review process specific to data analysis/methodology: http://t.co/5wmWKCxiiZ

RT @cblatts: I support liberal arts ed, but I’d substitute a couple courses in the humanities for stats and causal inference http://t.co/zH…

RT @bill_easterly: “Amazed at how many scientists have never considered that their data might be biased” http://t.co/Mh9WZMyd6k

Some basic level statistics class should be required even in high school, IMO. Every person, even outside of research, is frequently faced with statistics-based arguments, and the need to assess the validity of them.

My own thinking in social science has been improved by advanced calculus, statistics, probability and in general, quantitative methods. Ironically, I got most of that as Kris says above, in high school and then in undergraduate as a chemical engineer. And my brothers were engineers who then became PhDs in Mechanical Engineering and who do advanced mathematical modeling, so it has always helped that the family raises the bar. I am not sure how that would translate to undergraduate teaching though.

RT @clibassi: Science Magazine raises its statistical bar. Will we? http://t.co/h8oMzbVYWS cc: @dynarski

RT @kpelechrinis: “[…] a 21st century undergraduate degree in social science ought to require fluency in statistics” http://t.co/JVsyuJqu…

RT @cblatts: “I’ve been amazed at how many scientists have never considered that their data might be presented with bias” http://t.co/MInun…

Science Magazine raises its statistical bar. Will we? http://t.co/Hvp5c2ugTE via @jetpack

RT @VictorDeMiguel2 @TimHarford: Science Magazine raises its statistical bar. Will we? http://t.co/szdqZMbZfa

http://t.co/npbnnejYGw

It’s an interesting move, but I am not quite sure what the Statistical Reviewers do. In statistics, disagreements pop up early. I do not mean how to deal with various sorts of dependence in panel regression, but relatively simply issues in cross-section OLS. On what basis do the statisticians decide whether an analysis is sound or not?

What I do certainly like is that journals (e.g., PSRM) check whether scripts and data are running and produce the results an author is reporting in the manuscript.

I also agree one statistics and research design/causal inference course is warranted. Starting with this in the Master’s studies is too late.

I completely agree that the undergraduate social science programs need to step up and require more stats. You would think economics would be leading the way, but at Yale, undergrads only have to take one combined prob/stats/econometrics course. Most other top programs are better.

I wrote more about this here:

http://teachbetter.co/blog/2014/03/09/economics-and-data-analysis/

Science Magazine raises its statistical bar with new review board. @cblatts on the lessons others can take from it http://t.co/JFoYYk0qF3

@sciencemagazine takes a laudable step, increasing scrutiny for numbers in an era of #bigdata: @cblatts http://t.co/BYEeliGNTA

‘@sciencemagazine takes a laudable step, increasing scrutiny for numbers in an era of #bigdata: @cblatts http://t.co/BYEeliGNTA

Social science should too. MT @EPoDHarvard: ‘@sciencemagazine increases scrutiny for numbers #bigdata: @cblatts http://t.co/NHk8YqHpC0

RT @EPoDHarvard: @sciencemagazine takes a laudable step, increasing scrutiny for numbers in era of #bigdata http://t.co/LiA22sJMGk

Science Magazine raises its statistical bar. Will we?: From the Editors of Science: ….unfortunately, there hav… http://t.co/2M5jJq7rIO

Preach “…on the margin I’d substitute a couple courses in the humanities for statistics and causal inference” http://t.co/8EXtNEePzQ

‘@sciencemagazine takes a laudable step, increasing scrutiny for numbers in an era of #bigdata: http://t.co/BYEeliGNTA

RT @AnMailleach: ‘…but on the margin I’d substitute a couple courses in the humanities for statistics and causal inference’ http://t.co/7…

Completely agree! “Science Magazine raises its statistical bar. Will we?” @cblatts http://t.co/rMeMDydWoF

All very well and good: bad stats is bad research, so statistical education should start earlier and be more thorough. But when social scientists all have the mathematical and statistical skills of engineers and physicists – or better, if you really want to do the job properly, ‘cos the data you deal with is more complex – then what has been left out of their educational experience to fit in all the maths? We already have economists being pretty much wanna-be mathematicians who have forgotten about the real world, do we want to completely erase any actual understanding of what the numbers might mean from the social sciences as well? It’s a bit of a dilemma, I know, but whatever happened to David Freedman’s advocacy of “low-tech” research that emphazises actually knowing your subject?

Right, there is a limited number of courses students can take. Two or three solid courses on causal inference, research design and methods, and statistics could achieve a lot. For example, it would help if students understand significance testing correctly. Teaching them how to determine the “lowest-tech method” appropriate for a given research question could be part of a course too.