The 10 things I learned in the trenches of the Worm Wars

If you have no idea what I’m talking about, either count yourself lucky or see yesterday’s post. The rest of you, carry on.

  1. One of the things I love about the Internet is that it brought a lot of very smart people to a key intellectual problem, the discussion brought out the central assumptions and claims, and they were answered within about a day or two. See Berk Ozler, for example. My conclusion is that the Kenya deworming results are relatively robust. Yay hive mind.
  2. On the other hand, the hive mind has a tendency to be grumpy, rude, shrill and angry. I found the debate more dignified than some, but vicious at times.
  3. I am guilty as well. I was too quick to suspect and insinuate bad faith on the part of the replicators. I can hold suspicions but I shouldn’t publicly insinuate or accuse without grounds. I am sorry for that.
  4. I do find any big, coordinated media push of a scientific finding to be problematic, to say the least. This drove and drives my suspicion of bias, even if accidental.
  5. To me the big failure in this entire business was by the editor of the academic journal. The competing claims on whether or not the results are fragile or not, and why, should never have been allowed to remain ambiguous.
  6. To me, a glitzy media push undermined the credibility and intentions of the journal further. This is a general problem in medicine and hard science that I do not see as much in social science (where the journals could care less about news coverage).
  7. On the journalist side, I can’t blame any of the writers for not following the finer statistical points. I had trouble myself. But almost none of the journalists read the reply by Miguel and Kremer (published by the same journal) and maybe none called Miguel or Kremer on the phone. I am told I was the first. Tell me if I’m wrong, but isn’t this the definition of sloppy journalism?
  8. I think the deworm the world movement has also tended to exaggerate or selectively quote the evidence in order to justify the cause. GiveWell has a much more balanced account: the evidence is not that great, but it’s good enough. Sort of. I thought GiveWell had one of the best posts. Do read it.
  9. To me, the real tragedy is that, 18 years after the Kenya deworming experiment (which was not even a real experiment) we do not have large-scale, randomized, multi-country, long-term evidence on the health, education, and labor market impacts of deworming medicine. This is not some schmuck cause. This is touted as one of the most promising development interventions in human history.
  10. I also fear for the reputation of replications in development economics. I imagine some of the problems could be addressed by getting more clarity into pre-analysis plans for replications. But the incentives to make mountains out of molehills is huge. Maybe everyone should sign a “no glitzy media push” pledge.

Okay, I am DONE with worms.

6 thoughts on “The 10 things I learned in the trenches of the Worm Wars

  1. I love the idea that we would spend any time NOT treating children for fricking WORMS living IN THEIR BODIES.

    I mean . . . yeah.

  2. Thanks Chris for the great and constructive reporting – as always!

    There is one key point that we should not lose track of in this discussion. Of course debating the robustness of the evidence is great and really important. At the same time, we have to remember that most current development policies are based on zero or very meager rigorous evidence. So the alternative of governments in Kenya and India, for example, implementing deworming would not be for them to spend that money on a policy for which there is more or better evidence, but on a policy for which there is less.

    The policy relevant question is this: is there another intervention out there, for which we have good evidence that they will provide stronger benefits on education and future earnings than deworming?

    Ps. Even if so, governments should probably implement both that other policy and deworming, and reduce another policy for which there is weak evidence.