My bleg: seminal papers with fatal flaws

I’ve been puzzling through how best to teach a research design and causal inference course to poli sci grad students. I want to catch the big group of dissertation-writers who are using data and statistics but haven’t been trained as statisticians. There’s a lot of important skills and subjects for a political scientist to master, and stats sometimes falls by the wayside until it’s too late.

I’ve settled on the style in which the likes of David Card, Ken Chay, and David Lee taught me Labor Economics. They had this wonderful method of teaching by example: making us read seminal papers, write review reports, and spend seminar time finding the fatal flaws that pushed econometric science forward.

If I were satisfied with papers on training programs, labor supply, and minimum wages, my syllabus would be set. But I’m hoping to fill the syllabus with papers on crime, riots, war, elections, democratization, policy-making, and the like.

My bleg is for papers that fit the following criteria:

  1. Important in its time (and perhaps even now);

  2. Illustrate an econometric technique for understanding causality;

  3. Have a fatal flaw–a weak instrument, an omitted variable, a terrible measurement error problem–that we only recognize in retrospect; and

  4. The flaw isn’t so obvious that a second year grad student would catch it.

I’ll post the final syllabus later this week.

10 thoughts on “My bleg: seminal papers with fatal flaws

  1. I'll suggest Stanley Miller's "A Production of Amino Acids Under Possible Primitive Earth Conditions (Science 117: 528-529. 1953). Certainly qualifies as a seminal paper and in my opinion suffers from the flaw of the assumptions about the composition of the atmosphere of the early earth.

  2. Angrist and Krueger 91, although people have been piling onto that one for years.

    Card’s distance to college paper. (Does he use this in the his class?)

    Heckman’s selection model not really being identified without an exclusion restriction.

    What papers did Card / Lee / Chay use? Is their syllabus up somewhere?

  3. Maybe too econ-ish, but Baumol’s Productivity Growth, Convergence and Welfare (AER) for selection bias.

  4. I would also include the paper by Brambor, Clark and Golder-Understanding Interaction Variables: Improving Empirical Analysis in Political Analysis (2006). They cite various examples of flaws in the use of interaction variables in political science.

  5. Mauro’s paper about corruption and growth (QJE) I use the paper as an example of inadequate IV.

  6. I thought all the literature on cross-country econometrics civil war was fatally flawed in retrospect, except the survey articles that point out the flaws?

    To be serious, how about Miguel Satyanathan etc using rainfall by averaging all the nodes in the country. Surely someday with better/relevant rainfall measures, this paper will seem quaint? Is a grad student creating a Politically Relevant Rainfall Grouping (PPRG) that can be homophone with PREG?