So you want to be an impact evaluator? A cautionary tale

Time for a little general advice (if only to avoid writing the same inadequate e-mails over and over again).

Several aspiring graduate students have written me about becoming an impact evaluator. ‘What degree to get and where?’ is the most common question.

I think the best advice is: don’t get a PhD to do evaluations. The randomized evaluation is just one tool in the knowledge toolbox. It’s currently the rage, but that means it will probably be old news by the time you finish your PhD.

Yes, the randomized evaluation remains the “gold standard” for important (albeit narrow) questions. Social science, however, has a much bigger toolbox for a much broader (and often more interesting) realm of inquiry. If you want to know the effects of small binary treatments, you are in business. If you find any other question in the world interesting, you have some more work to do. Dani Rodrik has made a similar point here.

Don’t get me wrong: a large number of my projects are randomized control trials. They are eminently worth pursuing. But to be honest, uncovering the causes of effects excites me more than measuring the effects of causes. An evaluation masters the second, but only hints at the first. The hardest and most rewarding work is the theoretical and investigative work that comes with uncovering the underlying rhythms and rules of human behavior.

The best advice I can give to aspiring researchers: apply to PhD programs that will give you the best all-round training in as many different tools and pools of knowledge as possible.

Also: use your schooling time to tech up in formal theory and statistics (plus qualitative and comparative methods if you are a political scientist). Once you are finished, you won’t have time to acquire these skills. From the day you finish your PhD, it is a slow but steady descent into technical obsolescence.

If you’re interested in becoming a professional evaluator, rather than an academic, my advice changes little. It is now possible to be an evaluation consultant in much the same way that you can be an accountant or a lawyer–a highly specialized professional, with interesting and rewarding work. A PhD helps, but I don’t think it’s requisite. Yet I would still make the same case: you will be a better consultant, manager, and professional if you have a broad range of skills and knowledge.

If your goal is to improve the delivery of aid, and truly advance development, many more skills and knowledge are involved than the randomized evaluation. See here for more. But in short: a well-identified causal impact that arrives two years after the program does not performance management make.

For aspiring professionals, a masters in statistics with an MBA or MPA may be preferable, along with plenty of experience on the ground–preferably working inside a developing country government, not an aid agency. If nothing else, you may need a different set of skills if the fad ever fades.

I’m going to see if some of the other academic bloggers want to weigh in on this debate. Owen, Dani, Shanta, Nancy, Tyler, Brad: can I tempt you? Other bloggers welcome–I will post links here.