9 Responses

1. amazing

2. Very Nice article
   Thanks for sharing

3. как всегда на высоте

4. I say this all the time, but why can’t disciplines “shake things up” by setting up some sort of practitioner’s nominated best paper (and best paper within a field) award every year with online voting. Screw Nobel and AER/QJE honchos. Let economists elect and vote on what they think is the best paper they’ve read all year. Each PhD holder gets 1 vote an nobody’s vote matters more than another. It’s like SAG awards instead of the Oscars. We can even have subcategories like “best written by a non-tenured prof” or “best written, but maybe not best technique” and “most clever idea, though maybe not well written”. And find a way to prevent people from voting themselves 100 times…I know SSRN is sorta the closest thing, but there can be multiple downloading from different computers, assigning the paper to students to read, etc.

5. The BMJ’s ability to pick up errors in its peer review process is fairly abysmal as the BMJ’s own editor and colleague convincingly demonstrated in a research project on the journals own peer review practice.

   The article’s key finding: BMJ reviewers detected less than one third of ‘major methodological errors’ deliberately inserted into papers sent out for review in the trial.

   That is not so good. What is worse is that it is extraordinarily difficult to get rebuttals published in the journal——even when you can demonstrate ‘major methodological errors’.

   See: http://jrsm.rsmjournals.com/cgi/reprint/101/10/507

   Andrew Mack

   PS. This is the abstract of the article:

   “Objective
   To analyse data from a trial and report the frequencies with which major and minor errors are detected at a general medical journal, the types of errors missed and the impact of training on error detection.

   Design
   607 peer reviewers at the BMJ were randomized to two intervention groups receiving different types of training (face-to-face training or a self-taught package) and a control group. Each reviewer was sent the same three test papers over the study period, each of which had nine major and five minor methodological errors inserted.

   Results
   The number of major errors detected varied over the three papers.The interventions had small effects. At baseline (Paper 1) reviewers found an average of 2.58 of the nine major errors, with no notable difference between the groups.The mean number of errors reported was similar for the second and third papers, 2.71 and 3.0, respectively. Biased randomization was the error detected most frequently in all three papers, with over 60% of reviewers rejecting the papers identifying this error. Reviewers who did not reject the papers found fewer errors and the proportion finding biased randomization was less than 40% for each paper.

   Conclusions
   Editors should not assume that reviewers will detect most major errors, particularly those concerned with the context of study. Short training packages have only a slight impact on improving error detection.”

6. At the momemt, the paper is available here:
   http://www3.interscience.wiley.com/cgi-bin/fulltext/123323213/PDFSTART?CRETRY=1&SRETRY=0

7. I think this is it:
   http://www3.interscience.wiley.com/journal/123323213/abstract

8. Yes, please, give a working link (or a reference ?)

9. That link doesn’t work