Chris Blattman

Search
Close this search box.

Links I liked

1. The danger in mixing your coms with your orgs

2. Did market reforms kill Russians? The Economist takes down a sloppy statistical study from The Lancet. (Who reviews these things?)

3. Ethnic groups closer to the slave trade have higher levels of modern mistrust

4. Coffee reduces dementia. I knew it all along!

5. What do San Jose, Applebee’s, Wyoming, Chuck E. Cheese, and the Apple Store have in common? They all appear on the first page of a Google search for “Hell on Earth is”

4 Responses

  1. Sadly, no time for deep discussion this week, but The Lancet is infamous for sloppy social science statistics, among other things. There seems to be little appreciation for how to analyze observation data well.

  2. No, they haven’t done enough work and haven’t really included enough of their data work and regressions to let the reader decide for herself.

    You can come up with all sorts of stories to refute the findings. Here are a few (possibly implausible) stories:

    Countries with worse institutions under Soviet control fudged mortality statistics as they were unable to keep track (we already know that Russia does a lot to fudge infant mortality statistics). In the post-Soviet era, these statistics collapsed to their true, lower values. Those with weaker institutions also caved in to external (or internal, read: special interests) pressure for privatization. After 1995, from their graphs it seems that both countries saw the same growth rates in life expectancy, which doesn’t make much sense given their conclusions.

    Another thing they do is hold GDP per capita constant. It’s good to control for things, but it causes some problems with interpreting the effect of privatization: Say life expectancy grows (roughly) with pcgdp. Say privatization leads to high future levels of pcgdp, and thus higher life expectancy. If you control for the level of pcgdp, you could be masking a potential large positive effect of privatization!

    I’m not, by any sense, saying these are viable alternatives, but given the amount of information available in their article (thank you Lancet), you can’t come to the same conclusions.

  3. I agree that there were several non-starter arguments in the economist article. However, I think assertion (2) deserves more consideration. I didn’t find anything in the analysis that really got at pre-existing trends (like including region specific time trends perhaps) and this is certainly a limitation. At the same time, the authors have a much longer working paper version on this topic and they try to jump through more empirical hoops to assess robustness.

    Final analysi: Is the argument for causality convincing? Without the controls for pre-existing trends and policy endogeneity, probably not. But is this really a “sloppy” study? I don’t think so. It’s a difficult and important question and the authors seem to do their best given the data and research area.

  4. Did the Economist really take down the Lancet study? The Lancet is a peer-reviewed medical journal. The Economist is a magazine with articles whose authors are not even named.

    It’s true that the Economist *claimed* to rebut the study. Looking more closely, the Economist’s argument goes as follows, in order:

    1. Some blame for the 1990s disaster must go to the prior decades of Communist rule. And surely some blame must go to human genetics, and some blame must go to Russia’s geographical situation. This line of argument strikes me as a distraction.

    2. The Economist says that the 1990s reflected a simple continuation of a long-term trend in life expectancy (if we ignore other a reversal that occurred during the 1980s). This is pure hand-waving; why should we expect trends in life expectancy to continue?

    3. The Economist says that Lancet got the timing of the reforms wrong. If so, this strikes me as an important part of the response, and not something to be left to the next-to-last paragraph. The Lancet used a particular definition of reforms (basically counting the number of state-owned enterprises that were privatized); what was wrong with their definition? The Economist simply says that by 1994, Sachs had quit and the reforms were bogged down. (As an aside, the Economist claims that in 1994 life expectancy started rising; this is obviously contradicted by the helpful graph accompanying the article.)

    4. The Economist says that the Lancet got the effect of the reforms wrong. Simple assertion.

    5. And in the final paragraph: “Correlation is not causation.” This is certainly a common error, but is this really the error that the Lancet study made? Most of the Economist article is spent saying that there isn’t even a correlation.

    So what we have here is a “rebuttal” that makes virtually no reference to the article it is rebutting; isn’t that rather unconvincing? Am I supposed to just think that the Lancet study authors are stupid idiots who saw a correlation that wasn’t there, and then incorrectly inferred causality from that nonexistent correlation? Am I supposed to be surprised that the Economist performed some glib “analysis” and came to the conclusion that free markets work for the best?

Why We Fight - Book Cover
Subscribe to Blog