This is how to recognize bad science

pngbase64db8e9525745b448

pngbase64db8e9525745b448

Rather, my message is that this noisy, N = 41, between-person study never had a chance. The researchers presumably thought they were doing solid science, but actually they’re trying to use a bathroom scale to weigh a feather—and the feather is resting loosely in the pouch of a kangaroo that is vigorously jumping up and down.

Andrew Gelman critiquing a Kahneman psychology study. Via one of my favorite blogs.

Health, psychology, and exercise studies are, in my experience, the worst kangaroo-jumping feather-weighers.

One of the most important things you can do as a consumer of science is also the simplest: check the N. If you are reading a peppy New York Times study about how coffee makes you live longer, or exercise doesn’t help you lose weight, more often than not the study had 30 subjects and you can simply ignore the information you just received.

If you want to go a little bit further, you could also say “is this the only outcome I care about?”

53 Responses

  1. . If you are reading a peppy New York Times study about how coffee makes you live longer, or exercise doesn’t help you lose weight, more often than not the study had 30 subjects and you can simply ignore the information you just received.

    golu dolls
    golu dolls

  2. Came here to say the same thing as John. Misleading description to call this a “Kahneman psychology study.”