Participatory versus executive decision-making, the randomized control trial

A new paper on Sierra Leone, Chief for a Day, presented by Maarten Voors at the ABCA conference:

in a random subsample of villages the traditional elite, including the chief, were made responsible for project management. In other villages, responsibility for project management was delegated to a committee of randomly selected villagers.

While our simple theoretical model proposes that more powerful chiefs will be more corrupt by diverting more resources from the public good to their personal benefit, will under-invest in management of the project, and will seek to undermine the performance of managing committees, our empirical findings provide a more nuanced and mixed picture of the quality of local management in Africa.

Our field experiment finds little evidence that local elites managing an aid project divert more resources than the average villager, or that more powerful chiefs divert more than less powerful ones. Moreover, the village elites are able to manage a development project better than a committee of randomly selected villagers…

Projects managed by village elites are also more likely to start and be completed on time, are better constructed and maintained, and provide more (perceived) benefits for the villagers.

I think this project is more an indictment of the often naive approach to local politics in most NGO and other development programs, rather than an indictment of participatory decision-making per se.

One example: if we look at our own communities or organizations, clearly a leader or committee can be more effective if there are established channels or organizations that provide information or implement decisions, if people understand the roles and responsibilities of the decision-makers, or if those decision-makers have legitimacy in the eyes of the governed.

What that means is that changing the identity of the decision-maker may be less important if the organizations and processes and popular understanding are not changed as well. There’s a difference between changing the identity of who is in the CEO position and changing whether or not power goes to a CEO or a brand new committee.

Another example: what matters for executive decision-making is probably how constrained and accountable is the chief. Businesses prefer CEOs to committees, but that CEO is still accountable to a board, shareholders, labor law, the market, and a set of rules governing transitions of power. SO CEOs have incentives to be effective. So “chief” does not equal “despot”.

In sum, local politics may be difficult to change if it is not understood.

Links I liked

  1. Twitter shuts down an API that saves politicians’ deleted tweets
  2. The World Bank’s annual development conference on Africa is next week at Berkeley. Download the papers or watch the livestream presentations.
  3. Rather amazing about-face from the man who posted the viral Facebook message about Caitlyn Brenner
  4. The plot thickens on the Science scandal: Qualtrics (the survey firm) tweets that they can undelete LaCours’ data, then delete the tweet
  5. Columbia journalism dean Steve Coll on the future of journalism
  6. The New Yorker random cartoon page
  7. Quote of the day, via @AdviceToWriters: “Writing nonfiction is more like sculpture, a matter of shaping the research into the finished thing.” — Joan Didion

Why you should not go to law school

Amanda Taub has a brilliant article in Vox:

 over the years, I have realized that the people asking me that question aren’t really asking for my advice about their careers in the law. Rather, their real question is almost always something else: will law school be a solution to my fears about the future?

…You know the terror of the undefined future: the fear that comes of seeing so many possibilities and pitfalls open up to you that they swirl into one vast, terrifying abyss. And what’s that faint noise you hear from its shadowy depths? Could it be the plaintive cries of your mom’s friend’s son who had so much promise but never made anything of himself? Or the laments of your second cousin who’s stuck in a dead-end job?

These fears are understandable. The world is a scary place for people just starting out in their careers. It can feel especially scary for the kinds of people who find themselves pondering whether to go to law school.

They tend to have done reasonably well in school their whole lives, and then pursued liberal arts degrees in college. That means college graduation brings a double-whammy shock of chaos: not only are they losing the structure of the academic system in which they excelled, but they’re also setting out in that new, seemingly unstructured world without the kind of clear career path that comes from a more technical degree. Of course that can make the post-college world seem like a terrifying abyss.

You could say this about any graduate study. Too many students do it because the real world does not have a nicely laid out application path.

People go into consulting and banking and a few other private sector industries for the same reason: they come recruiting and make the path as easy as possible. Hence they hire an awful lot of people who will hate their job and be doing more or less meaningless things. That is how I first ended up in accounting. Leaving early was the best thing I ever did.

Most useful is Amanda’s point that there are other ways to get across the terrifying abyss. Generically, the advice is this: seek out the less obvious doors and knock on them. The obvious doors that everyone can see are mobbed.

Related, here are my past posts on getting a job in international development and should you do a PhD?

“The Misregulation of Human-Subject Research”

That is the subtitle to a new book, The Censor’s Hand, by Carl Schneider, a Professor of Law and Medicine at University of Michigan. It’s an assault against the state of Institutional Review Boards (IRBs) that supervise research on human subjects.

From an interview with Inside Higher Ed:

Good regulation is accountable, but IRBs are effectively answerable to nobody. Good regulation has clearly defined jurisdictional limits, but IRBs may intervene as they wish. Good regulation is guided by clear rules, but IRBs have little more than empty principles. Good regulation is disciplined by fair procedures, but IRBs can ignore every fundamental precept of due process. Good regulation is transparent, but IRBs need not even explain — much less justify — their decisions. Good regulation is staffed by experts, but IRB members cannot be competent in all the specialties they regulate. Good regulation has manageable workloads, but IRBs regulate more details of more research in more ways than they can review responsibly, and they have steadily broadened and intensified their hold over research.

In short, the IRB system makes unreliable decisions because it is lawless and unaccountable, because its organization, procedures, membership and imperialism are so inappropriate. The problem is not regulation, it is bad regulation.

They are good points. The fact that I thought twice about posting this, for fear it could offend the authorities, might be evidence  in favor of his argument. “Keep your IRB happy” is one of the main things I teach my students, because your career can depend on it.

Even so, Schneider goes much too far when, later, he says, “Nor is there good evidence that IRBs actually improve research or protect its subjects.”

I agree that my human subject experiences seem more and more like lawsuit protection, focusing on risk mitigation for the University rather than the research subjects. At the same time, I see colleagues and students intent on ethical infractions, usually minor, all the time. Colleagues of mine have also had their computers seized by autocratic police, and the encryption pushed on them by the IRB was the only thing that saved their subjects’ privacy and safety.

Nonetheless, I agree: the bureaucracy is broken. A couple of weeks ago I was getting my haircut a few blocks away from campus. The daunting prospect of getting a simple human subjects approval and a data collection contract in two months had weighed on me all day, and I’d spent the afternoon banging my head against a bureaucratic wall. Purchasing, lawyers, and project management at Columbia are like nothing I’ve ever seen before. When my haircutter asked how I was doing, I replied honestly: the Columbia bureaucracy was getting me down. He started laughing. “You’re the third person to say that to me this week!” he replied.

So some places are worse than others.

IPA’s weekly links

Guest post by Jeff Mosenkis of Innovations for Poverty Action.

  • Dean Karlan, Esther Duflo and many others will be at the World Bank Thursday AM, streaming live here from 9AM to 12:45PM (Washington, DC time). They’ll be talking about the new results in Science on the programs for the ultra-poor, and what’s next for scaling up.
  • The New York Times Magazine has an interesting feature on a Russian agency devoted to creating and planting hoax news stories in the U.S., and trolling other internet targets.
  • What’s worse than Ebola and keeps Bill Gates up at night? Pandemic flu. According to a simulation run by his foundation, if a flu strain like the 1918 Spanish flu appeared today, given modern travel patterns, it would be in every major city within 60 days, and by 250 days would have killed 33 million people. That also means that low spending on health in developing countries is everybody’s problem.
  • As if to underscore his point, one business traveler brought Middle East Respiratory Syndrome back with him to South Korea, where it seems to have spread rapidly. Currently at least two people are dead, 700-1300 people are under quarantine, 200 schools are closed preventatively, and at least one other infected person brought it from there to China, where nurses reportedly held a lottery to see who would have to treat the patient.
  • Income smoothing isn’t just for the poor, according to a new paper by Carlson, Kim, Lusardi, & Camerer. Despite earning an average of $3.2 million, 15 percent of U.S. football players declare bankruptcy by the time they’ve been out of the league for 12 years. Effects are independent of how long they played or how much they made, and the bankruptcies start almost immediately on retirement.

And, France’s ambassador to Cameroon wins at podiums:

FrenchAmbassadorToCameroon

(via the Embassy’s Facebook page)

Who is a New Yorker?

The dreamers simply outbid us for our New York, and in the process, they created a city we no longer loved quite so much. This is apt to make such musings hard reading for us.

…I do not mean to insult you, Non-Native New Yorker. I am sure you are a very fine and authentic person, doing many authentic things in that fourth-floor walk-up you struggled so many years to achieve. But I have to be honest: To us, most of you will never be New Yorkers. You just don’t have it in you.

Real New Yorker status can be acquired. Both my parents had it, though it probably took them decades. I know others, scattered here and there. But it cannot be acquired by reading Joan Didion and investing all of your adolescent angst in a few square miles of concrete. It cannot be acquired by going to awesome, funky bars that are totally different from the ones at home and meeting a heroin addict bartender/freelance sex worker who wears cowboy hats and Doc Martens and quotes Allen Ginsburg while the two of you smoke cigarettes underneath the High Line. It is not displayed by your encyclopedic knowledge of the best places to buy exotic foodstuffs.

That is Meg McArdle ruminating on her past life as a New Yorker. Superb essay.

Is the single best anti-crime investment behavioral therapy?

As in Liberia, in Chicago? Further evidence that behavioral therapy focused on emotional regulation, impulse control, planfulness, social skills, and positive self-image can have large effects on crime and violence. Mainly, it seems in many cases, through emotional regulation and impulse control.

This paper describes how automatic behavior can drive disparities in youth outcomes like delinquency and dropout. We suggest that people often respond to situations without conscious deliberation.

While generally adaptive, these automatic responses are sometimes deployed in situations where they are ill-suited. Although this is equally true for all youths, disadvantaged youths face greater situational variability. This increases the likelihood that automaticity will lead to negative outcomes. This hypothesis suggests that interventions that reduce automaticity can lead to positive outcomes for disadvantaged youths.

We test this hypothesis by presenting the results of three large-scale randomized controlled trials (RCTs) of interventions carried out on the south and west sides of Chicago that seek to improve the outcomes of low-income youth by teaching them to be less automatic.

Two of our RCTs test a program called Becoming a Man (BAM) developed by Chicago-area non-profit Youth Guidance; the first, carried out in 2009-10, shows participation improved schooling outcomes and reduced violent-crime arrests by 44%, while the second RCT in 2013-14 showed participation reduced overall arrests by 31%. The third RCT was carried out in the Cook County Juvenile Temporary Detention Center (JTDC) in 2009-11 and shows reductions in return rates of 22%.

We also present results from various survey measures suggesting the results do not appear to be due to changes in mechanisms like emotional intelligence or self-control. On the other hand results from some decision-making exercises we carried out seem to support reduced automaticity as a key mechanism.

A new paper by Heller, Shah, Guryan, Ludwig, Mullainathan, and Pollack.

Sorry but I do not know of an ungated copy. But a summary is here.

Research positions at the IRC

Following in my tradition of posting job announcements only on behalf of people that marry me, I present several new positions in the International Rescue Committee’s research department. It is growing, it’s the center of impact of evaluation in the humanitarian sphere, and you will have a great boss.

  1. R&D Director
  2. Deputy Director, Evidence to Action
  3. Deputy Director, Research
  4. Grants and Finance Coordinator
  5. Policy and Communications officer

Timely publications: “The Career Effects of Scandal: Evidence from Scientific Retractions”

…we investigate how the scientific community’s perception of a scientist’s prior work changes when one of his articles is retracted.

Relative to non-retracted control authors, faculty members who experience a retraction see the citation rate to their articles drop by 10% on average, consistent with the Bayesian intuition that the market inferred their work was mediocre all along.

We then investigate whether the eminence of the retracted author, and the publicity surrounding the retraction, shape the magnitude of the penalty. We find that eminent scientists are more harshly penalized than their less-distinguished peers in the wake of a retraction, but only in cases involving fraud or misconduct.

When the retraction event had it source in “honest mistakes,” we find no evidence of differential stigma between high- and low-status faculty members.

A new paper by Azoulay, Bonatti, and Krieger. Hat tip to Dina Pomeranz.

IPA’s weekly links

Guest post by Jeff Mosenkis of Innovations for Poverty Action.

  • In a story circulating widely now, Science journalist John Bohannon, explains how he planted a fake study in the news showing chocolate can help with weight loss. The study was really done, but “p-hacked” – they compared two diet groups (one including chocolate) to a control and measured lots of outcomes so at least one one would come out as statistically significant by chance, then published in a fee-for-publication journal and put out a press release about it, which several outlets picked up without asking any critical questions.

3 points:

-There may ethical questions around deliberately misleading the public, especially about health.

-None of the outlets who picked it up were particularly big names.

-Dean Karlan (in a very cursory glance), points out that they were already dieting and 1.5 oz of chocolate is very few calories, so having that outlet could have kept people from eating other sweets (as another study suggests). The findings were in weight loss, the area one would hypothesize, rather than other areas like sodium level, so the findings might actually be legit, which would be the greatest con of all.  (h/t Everybody.)

  • Psychologist Michael Inzlicht says that researchers have to start being honest with themselves about cherry picking findings, starting with himself. He subjects his own results to three tests anybody can do: First a “p-curve” which checks your results against a pre-set rule to see if your findings are consistently too good (paper and calculator here). He did OK there, but fared worse on the Test for Insufficient Variance (TIVA) which involves converting p values to z-scores (explained here), and Replication Index (or R-index) which tries to determine the probability a finding can be replicated using power and frequency of success rates across a series of studies. However supporting Chris’ Drama Queen rule, Inzlicht wasn’t alone:

Uli Schimmack … also reanalyzed a set of psychological studies appearing in Science, finding that they achieved a painfully low R-index of 33.9%. These low values raise concerns about the empirical support for the underlying articles.

 

  • A mystery phone without many features is taking Ghana by storm, entirely by word of mouth, according to Quartz. It’s bulky, ugly and doesn’t work well, but its main feature is “power banking” – a huge battery that lets people save up power and charge their other devices during frequent outages (there’s a mobile banking joke in there somewhere).
  • NPR looks at Somali refugees who’ve spent the past 24 years in a refugee camp in Kenya (the subject of a forthcoming book by Ben Rawlence). A generation has grown up there with western-style education and job training, and under rules designed to promote gender equality. Rawlence points out that in many ways they look more like a middle class than those still in Somalia. A group of the refugees were recently taken on a U.N.-sponsored visit back to Somalia and were unimpressed, according to the story.

And, from Ian Bremmer, who titles this “Theory vs. Practice”:

BremmerTheoryVsPractice

Links I liked

  1. Steven Lubet questions whether Alice Goffman’s On the Run is all true, and Alex Tabarrok’s impression that Lubet’s making mountains out of molehills. I lean to Alex’s impression and if I ever have the time will blog about Goffman’s book. [Edit: Have heard more tales of dubiosity from colleagues since I posted this and so will take some time to consider.]
  2. Did you register your social science experiment? This foundation wants to give 1000 of you $1000 as a reward. And BITSS at UC Berkeley wants to give up to $15,000 to scholars pushing the envelope of transparency teaching and research.
  3. A twitter bot that retweets people’s thoughts about their Uber drivers: @myuberdriverbot
  4. Dad, level 9000

How do people defend eating meat?

This study asked students and adults in the United States why they find it OK to eat meat.  The largest category used to justify their choice was that that it is “necessary” followed by the other three categories.

Typical comments used to justify eating meat include these 4Ns:

  • Natural “Humans are natural carnivores”

  • Necessary “Meat provides essential nutrients”

  • Normal “I was raised eating meat”

  • Nice “It’s delicious”

Article and academic paper.

Unfortunately, “marginal effect” does not begin with an N, because that would probably be my preferred response alongside “nice”.

In terms of environmental degradation I see a strong argument for reducing meat consumption, but surely the marginal benefits of going to zero meat are small compared to reducing some other environmentally unsound behavior on the margin. Such as transatlantic flights or certain types of energy-intensive food or travel.

The zero meat move makes more sense from a behavioral perspective, where we recognize we are frail and impulsive creatures. Vegetarianism lets you draw a bright red line not to cross. Otherwise it’s easy for you to slip in the moment and eat that bacon. It’s possible that, in practice, vegetarianism is easier to stick to than reduced meat consumption, and so you could pre-commit yourself.

My alternative, unwitting solution was to marry someone who does not eat red meat and hence never cook it at home. A commitment device. Literally!

You can also get to zero meat if you object to raising animals for slaughter for moral reasons. This is a way of saying that the one’s marginal cost of going from zero to a little meat consumption is very high. But I don’t feel that way. I do favor more reasonable treatment of animals and try to buy such meat. I’d personally prefer an equilibrium where everyone chooses this, and/or it’s regulated, because I think that would make the choice cheaper and more easily available, and is probably the right thing to do.

Vegetarian readers: I await your comments or objections.

The geek heretic

My first job abroad was helping to roll out rural Internet in India, by radio. It was 2001, the peak of the dot com boom, but before the ubiquity of mobile phones. Connecting remote villages in India to the Internet via radio waves was the great frontier, and I was hired to help roll it out and evaluate it.

Fifteen years later the idea of radio-based Internet seems quaint, but I recall Paul Samuelson telling one of the professors I worked for that he thought it was the most exciting idea he’d heard in years. Like I said, dot com boom. Unless you lived through it it’s hard to explain, but everybody was a little crazy when it came to the web those days.

The project was a debacle for more reasons I can recount in a blog post. A big one: relieving an information and communication constraint was not going to cause markets to surge or governments to become accountable. Even if that information mattered, any surge would come to a screeching halt with the dozen other greater constraints holding change back.

You could say I became a geek heretic. No longer would I believe that technology is the solution. It didn’t hurt that the stock market bubble was popping at the same time.

Many years later, I met Kentaro Toyama, a kindred spirit, but one who had gone to much further lengths and depths in his fervor for and against technology in development. The onetime director of Microsoft Research India, he’s just released his book, Geek Heresy: Rescuing Social Change from the Cult of Technology:

For twelve years I worked at Microsoft, where, like every other gizmo-happy technologist, I unconsciously embraced a peculiar paradox. It revealed itself in the most innocuous things that the company said. At corporate gatherings, executives would tell us, “You are our greatest asset!” But in their marketing, they would tell customers, “Our technology is your greatest asset!” In other words, what matters most to the company is capable people, but what should matter to the rest of the world is new technology. Somehow what was best for us and what was best for others were two different things.

This book is about this subtle contradiction and its outsize consequences. I explore how a misunderstanding about technology’s role in society has infected us–not just the tech industry, but global civilization as a whole–and how it confuses our attempts to address the world’s persistent
social problems. Th e confusion expresses itself as Silicon Valley executives who evangelize cutting-edge technologies at work but send their children to Waldorf schools that ban electronics. Or as a government that spies on its citizens’ emails while promoting the Internet abroad as a bulwark of human rights.

One of the ideas in the book is that technology takes us only as far as our capabilities. A nice quote from Bill Gates sums it up:

The first rule of any technology used in a business is that automation applied to an
efficient operation will magnify the efficiency. The second is that automation applied to an inefficient operation will magnify the inefficiency.”

If you find yourself even remotely optimistic about technology and development, you should read this book.

I have an idea for another get-out-the-vote experiment

An Arizona woman was sentenced to three-and-a-half years in prison for running over her husband because he failed to to cast a vote about who would spend the next four years in the White House.

Holly Nicole Solomon, 31, hit her husband, Daniel Solomon, and he suffered a fractured pelvis from being run over after a wild chase on November 10, 2012, that left him pinned beneath her Jeep.

Solomon was upset with the 36-year-old following the November 6 re-election of President Barack Obama and believed their family would ‘face hardship’ because he had won another term.

Full story. I expect Stanford to implement within the month.

If you want people to believe your research, write it in Baskerville font?

We have entered a new, unexpected landscape. Truth is not typeface dependent, but a typeface can subtly influence us to believe that a sentence is true. Could it swing an election? Induce us to buy a new dinette set? Change some of our most deeply held and cherished beliefs? Indeed, we may be at the mercy of typefaces in ways that we are only dimly beginning to recognize. An effect — subtle, almost indiscernible, but irrefutablythere. (“Mommy, Mommy, the typeface made me do it.”)

For every thousand respondents to the Times quiz, nearly five more people agreed with Deutsch’s statement when written in Baskerville’s typeface than they did when they read it in Helvetica. A typeface that nudges (to use the vernacular of experimental psychology) us to uncritical belief? Did Baskerville, despite his opposition to the irrationalities of religion, create a typeface that has a religious pull?

That is Errol Morris in the New York Times, in 2013. Fast Company covered the experiment last week.

While we are on the subject of transparency of methods, and sharing of data and code: has anyone replicated this? Getting the data and scrutinizing it strikes me as a great term paper for a PhD student.