I have pointed students to advice on how to start a thesis. I have also posted my thoughts on what to do (and not to do) in empirical research.

For the past few weeks, though, I find myself giving the same speech to PhD student after PhD student, most of whom come to my office saying they want to do a field experiment for their dissertation.

The first thing I tell them: This is probably a bad idea.

First, the market is getting very crowded, and there are a million in the pipeline. So this is a poor product placement. The premium on other types of research is high and getting higher.

Second, I am suspicious whenever someone puts the method before their question.

Most of all, however, I explain why most field experiments have the hallmarks of a bad field research project. There are four:

1. Takes a long time. Anything requiring a panel survey a year or two apart, or a year of setup time, suffers from this problem.
2. Risky. There are a hundred reasons why any ambitious project may fail, and many do.
3. Expensive. This is driven by any kind of primary data collection, but especially panel or tracking surveys, and especially any Africa or conflict/post-conflict research.
4. High exit costs. This is where experiments excel. If your historical data collection, formal theory, or secondary dataset isn’t working for you, you can put it aside. If your field experiment goes poorly, not only are you stuck with it to the bitter end, but it will take more not less time.

Most dissertation projects have two or three of these traits, and that’s normal. But most experiments have all four, and then some. In my experience, the first and fourth are especially deadly.

My (non-experimental) dissertation papers–the effects of war violence and recruitment on economic outcomes and political behavior–were off the charts on #2 and #3, but not #1 and #4. I only needed a single cross-section of data, and so I went from glimmer of an idea to a draft paper in 12 months. I also could have exited early without too much trouble.

This is completely by accident, not foresight. I’m sure if I had thought of a way to make it long, painful and impossible to exit, I would have charged ahead with even more enthusiasm. That is more or less a description of every decision I made from 2007 to 2010.

As with most advice, when someone points a finger at you, it is best to remember they have three fingers pointing at themselves.

Not all field experiments have all four problems. Something that takes little time to elapse, does not require a panel, or uses administrative or other secondary data can eliminate most of the downsides. This is one reason you see so many people successfully doing experiments around elections. But that is a narrow set of questions and crowded space for the same reasons. Downstream experiments offer more potential, but it may be hard to get a single-authored study without the original researcher, depending on the circumstance.

If you feel discouraged, but a faint glimmer of hope, then I have hit my anticipated target–to make you look before you leap. Of course, just as I ignored my committee’s unanimous advice not to start my dissertation project, I expect you all to follow in the well-worn tradition of ignoring your advisors, including me.