Is it ok for researchers to mess with elections?


The events:

  • Three political scientists sent out official-looking election mailers to 100,000 Montana voters
  • The mailer described the ideological standing of technically non-partisan candidates for the Montana Supreme Court, putting one close to Obama and one close to Romney.
  • The mailer had the state seal among other logos, and the fine print said that the mailer sent by researchers from Dartmouth College and Stanford University, part of their research into voter participation

The academic controversy, from Talking Points Memo:

…political scientists consulted by TPM described the study as “malpractice” and “improper and unethical” because, by introducing the ideological position of non-partisan candidates, the flyers could — intentionally or not — influence the results of the elections.

“It’s basically political science malpractice. That’s what I’d call it,” Jennifer Lawless, professor of government at American University in Washington, D.C., told TPM. “When you’re going to engage in an experiment as a political scientist, I think you have a responsibility not to affect election outcomes, let alone break the law.”

…”This strikes me as a lapse in judgment. If the election’s actually happening, they’re intervening in it,” Theda Skocpol, a political scientist at Harvard University, told TPM

Is this malpractice? It’s obviously questionable. But I haven’t jumped to an answer yet. In fact, the more I think it through, the more difficulty I have seeing a wrong, especially one the profession could or should ever rectify.

It sounds to me like Lawless and Skocpol are saying that you can’t do research and be an activist at the same time, even if it’s an activist for relatively benign things, such as “more information for voters”. I’m not so sure we can or should regulate this as a profession.

Let’s use a different example. I run field experiments trying to reduce poverty and violence. Sometimes I design experiments and interventions myself. Seldom do people say to me, “You are mucking around with real world outcomes. Who gave you the permission to do that?”

Maybe people should ask. It’s a good question. Thinking it through, I get a more nuanced answer than malpractice.

When you’re working in your own country (say the US for example) the law generally protects your right to help other people, or give information in elections. This is true even if you want to give very, very large sums of money for partisan causes (like Buffett or the Koch brothers do).

If these researchers were mucking around in another country’s elections, the law of the land would say whether or not the level of informed consent was a problem. In the case of Montana, it sounds like the researchers disclosed their purpose and identity, which is more than I can say for most electioneering in this country. With the exception of the State Seal question (I don’t know if this is legal or not) it sounds to a non-expert like me that they were within the law.

I don’t think it matters for what Skocpol and Lawless said. The political scientists calling this unethical don’t seem to be quibbling about the use of the seal or adequacy of disclosure. It’s the fact that an experiment could affect outcomes that they seem to question.

Let’s go back to my poverty and violence work for a minute. When I work in other countries, I generally have the consent of both the government and the participants. I also work with organizations doing this sort of intervention anyways (though not always).

I’m willing to bet that, had the researchers worked with a non-partisan institute that was mucking around in elections already, there would be no blowback to the researchers at all. They would simply be evaluating a real world intervention. Even if it was clear they had worked with the institute to tweak or even design the mailers, for most this would never be an issue.

My hunch is that it’s the fact that the academics are the interveners that makes “other political scientists” question the appropriateness.

You can also argue the finer points of informed consent in a democracy, but for me I think it really comes down to this question of whether academics can also be activists at the same time. Naturally I can come up with reasons why they shouldn’t, but I don’t find any particularly compelling. Most of us have political agendas, and we write books and articles pushing those agendas. It’s not obvious to me that as an academic in a democracy I am not allowed to use my work to achieve certain political ends.

I think it would be fair for universities to set out guidelines of appropriate behavior beyond what is required by law. And maybe disciplines and journals too. Personally I would prefer a university and discipline that restricted its members as little as possible.

I’m sure there are sides I’m not thinking of. I’d love to hear why people think intervening in an election is off limits for academic research (especially when it’s done by citizens in their own country). I promise to post again if you change my mind.

89 Responses

  1. Hi Chris: I am confused why we are calling these professors activists in this case. Do we have a clear definition of activists, against which we can measure whether a professor is ‘doing activism’ — let alone figuring out what that means and whether it’s OK.

    More importantly, the point “If the experiment broke laws, they should be punished.” My understanding is that they did, by using the state seals without appropriate permission. But — to what extent should the researchers be punished and to what extent the IRB? That’s a pretty major oversight and both parties seem to have made it. Does it suggest that current IRBs aren’t fully equipped to deal with social science research and experiments issues (e.g. http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2265757)?

  2. Bonica is a co-founder of crowdpac.com, along with Steve Hilton, a former aide to UK Prime Minister Cameron now teaching at Stanford. Cameron is a Tory, no friend of labor or common folk. Bonica is a fellow at the Hoover Institute, which doesn’t lean left.

    Bonica was in Montana in June for a conference on money in state politics. His presentation promoted crowdpac.

    The experiment, if it was an experiment and not advocacy masquerading as an experiment, may well have been a proof of concept test to validate crowdpac’s technology and impress future crowdpac clients and investors. Stanford is investigating whether avoiding review by Stanford was an attempt to hide crowdpac related research from the university. Stanford contributed 100k to the project, so perhaps Bonica, et al, were trying to cut Stanford out of a share of the profits.

    The mailers in all three states used the state seal and were designed to look as official as possible. That cannot have been a mistake. Misrepresentation was built into the adventure. And by using the official state seals, so was lawbreaking. How political scientists can defend this FURBAR affair escapes me.

    Perhaps they will strike it rich with crowdpac in 2016, but I think there’s strong possibility that Bonica and his fellow investigators could have a hard time finding jobs at struggling community colleges.

  4. Jim,

    You have an IRB check off on experimentation because they want to cover the university from liability. The IRB apparently (IF they reviewed this) found no ethical problem with the experiment, which is precisely my point. If the experiment broke laws, they should be punished. If it did not, then they should not be punished on the facile grounds that the experiment may or may not have “affected election outcomes.”

    Countless things we as researchers do COULD affect election outcomes. Polling organizations test questions all the time that could sway voters. Campaign outfits do their own experiments on mailer design and other factors–how is that different? Academics do experiments to see if receiving a phone call increases turnout–surely that could swing an election. Should I not report my findings on electoral strategies before an election, because it might influence one campaign’s strategy and help them win? What if I tell my neighbors half-truths about a candidate just for fun? Is it okay if I do it for fun, but not if I ask them how they voted after election day for my “research”?

    By the way, nonpartisan elections simply mean that the candidates’ party affiliations are not printed on the ballot. There is no law against providing partisan information about candidates–maybe there should be–so if you think this isn’t already happening in other elections, you are naive.

  5. I don’t want to pile on, but in my “big data guy” role, I have to ask why 100,000? As far as I can tell there’s exactly one binary treatment, or is there some complicated factorial interaction that leads to a number that big? For that to be the right power number, the treatment effect has to be microscopic and the instrument has to be incredibly weak — that is, it has to be a poorly designed experiment of a substantively unimportant question. Conversely, if the team believes the treatment will work, 100000 is surely big enough to actually change outcomes. (It’s also clearly big enough to get the experiment noticed, creating spillover between treatment and control that ruins the experiment.) Quoting Nolan McCarty, “there’s a reason medical experiments are small.”

  6. bp

    Look up hubris in the dictionary. This experiment falls directly under the definition. Your claim here is pretty astonishing. I can do whatever I want because it is not illegal? Why have an IRB check off on any sort of experimentation (or any other human subjects research) then? The intervention here was done for the research purposes of the three authors. If they impact the election – and the downstream policy/judicial decisions that follow from the election of one candidate rather than another – just to further their own research, the folks in the jurisdiction can surely complain loud and clear. Likewise with disciplinary colleagues. Rodden, et al. were not trying to inform voters. Any information flow was incidental to their own aims. They were trying to perform an experiment. Where they think they (or you or any other social scientist) get off manipulating voters for their own personal hobby is inexplicable to me.

  7. Jim, maybe they should have received explicit approval from each local jurisdiction first? Or maybe each HOA would like to weigh in.

    It doesn’t have anything to do with where you live, nor should it. If they broke election laws by using the seal, or by engaging in political activity without registering with the state, then they should be punished for that. But if there are no specific laws that they violated, then it does not matter who gave them approval because from a legal standpoint they don’t NEED approval.

  8. Political entities participating in elections are sanctioned by law. Ideally there is some sort of accountability. (That is a complex question). In other words this is a political matter. The three amigos got permission (if they did) from the IRB at an institution NOT located in the relevant jurisdiction. Who gives the Dartmouth IRB any standing in the Montana electoral system? None of the experimenters live in Montana as far as I know. In other words you ‘in their own country’ caveat gains no traction here. They are totally unaccountable and not citizens of the jurisdiction in question. This is precisely the sort of intervention that, since Gosnell in the 1920s, has raised concerns about field experiments in elections. Moreover, there is no consent on the part of those receiving the intervention. I am not sure how any of your rationalizations hold water.

  9. I respectfully disagree. First, I am not sure whether we can justify disproportionate power relations between those who conduct experiments and those who are subjects on the grounds that the results ultimately reduce poverty. Researchers do not only produce knowledge, but they also build careers. In some cases, these career trajectories are much more rewarding: you end up building much more capital–economic and social–than the supposed benefits that the subjects would receive–especially regarding the knowledge produced by this research agenda. The justification of these obvious inequalities in the name of advancing scientific inquiry/activism is a bit too ambitious, and perhaps, too self-righteous. Second, and more importantly, experiments in social sciences are not very strong instruments: in some cases, replicated experiments yield contradictory and insignificant/spurious results. I am not sure how one can place so much faith in an instrument that has a much more limited use in social sciences. I am not saying experiments need to be abandoned all-together; but it would be wiser to have some degree of skepticism regarding the benefits of its use to knowledge accumulation. Otherwise, our mindset would be no different than a 19th century colonizer in Africa: “but we bring civilization!!” (replace this with “but we produce knowledge for humanity!!!”)

  10. The idea that political scientists shouldn’t intervene experimentally if that intervention MIGHT affect outcomes is ridiculous. Wantchekon could have affected the election in Benin. Gerber and Green and other large-scale voter mobilization experiments could easily affect election outcomes too–everyone knows that increasing voter turnout is most likely to push the electorate to the left.

    Even if we stipulate to a “do no harm” standard, how are we to define harm? What clear harm did this experiment cause? If the claim is that it undermined the non-partisan nature of the election, we have to believe that that is harmful. Is it? It’s not quite as clear-cut as a doctor prescribing the wrong medicine and killing someone.

    If the claim is that experiments of this kind will make everyone think they are in an experiment, then I suppose we shouldn’t do any experiments at all. Or maybe not splashing them across the front page of the news would be a safer strategy.

    By the way, it is perfectly legal to provide information about policy preferences and partisanship to voters in a non-partisan election. Not that it necessarily should be legal, but there is certainly no law preventing it. The only legal issue of any potential consequence at all is the use of the state seal.

  11. If this kind of experiments are allowed and encouraged, people will start to suspect everything they come across is an experiment. The result is you didn’t just interfere in one single election, you start to interfere in everything.

  12. I think they should have been sensitive to the potential to influence the outcome. Your analogy to poverty reduction is a poor one. No one argues that poverty is a good thing. In this case, they could influence not just turnout (a defensible goal) but the outcome (a much less defensible outcome).

    My own problems are with the use of the state seal. State sanctioned communications about elections are a staple of voting in the West. By trying to fake a state communication, the researchers were undermining the non-partisan election. I believe this violates experimental protocols that a manipulation should cause no harm. Michelson makes this point very effectively in her blog post.

  14. The mailer is designed to look like it comes from the state. Hence the state seal, and the label Montana Voter Guide, and “Take this to the polls!” in big print on the other side. The mailer creates the impression that the secretary of state’s office (which administers elections) is providing partisan information in what is, by state law, a non-partisan election.

    This is misleading. It risks making voters, funding agencies, and government officials less trusting of scholars trying to study elections.

  15. It can be argued that they are violating the “golden rule” (also known as “bad general equilibrium effects”). If all researchers started interfering directly with election outcomes, any single study would no longer measure the partial effect at “no interference by researchers”.