This early childhood program increased voting by 40% twenty years later

I find a strong relationship between the non-cognitive skills of grit, self control, behavioral control, and social skills measured in childhood and political participation in adulthood. This strong relationship holds even when considering measures of cognitive ability and other potential confounders.

Simply put, those children who develop non-cognitive skills are more likely to participate in adulthood. Going one step further, I test whether exogenous improvements in non-cognitive skills during early childhood translate into participation increases in adulthood. To do so, I use a unique 20-year, multi-site field experiment—the Fast Track intervention…

I show that this early-childhood field experiment targeted|and successfully moved|students’ non-cognitive skills, while leaving their cognitive skills and other factors relevant to political participation virtually unchanged…

Exposure to this program increased turnout among participants 11-14 percentage points—a substantial amount, constituting at least a 40% increase in baseline participation rates…

It appears that Fast Track mobilized because it taught children to regulate their thoughts, behaviors, and emotions and use these abilities to integrate in society.

A new paper by John Holbein. Unfortunately I did not see data on party affiliation.

In related news, my colleagues Julian Jamison and Dean Karlan produce a worthy contender for an Ig-Nobel prize for their Halloween candy voting experiments with children.

We decorated one side of a house porch with McCain material in 2008 (Romney material in 2012) and the other side with Obama material. Children were asked to choose a side, with half receiving the same candy on either side and half receiving more candy to go to the McCain/Romney side. This yields a “candy elasticity” of children’s political support.

Worm wars continued (but not by me)

Michael Clemens and Justin Sandfur at CGD weigh in:

Suppose a chemistry lab claimed that when it mixed two chemicals, the mixture rose in temperature by 60 degrees. Later, a replication team reviewed the original calculations, found an error, and observed that the increase in temperature was only 40 degrees.

It would be strictly correct for the replication team to announce, “We fail to replicate the original finding of 60 degrees.” That’s a true statement by itself, and it doesn’t fall within the strict purview of a pure replication to do additional tests to see whether the mix rose by 30 degrees, or 40 degrees, or whatever.

But it in this situation it would be excessive to claim that replication “debunks the finding of a rise in temperature,” because the temperature certainly did rise, by a somewhat different amount. This is basically what’s happened with the deworming replication, as we’ll explain.

I haven’t seen many non-economist responses, other than Stéphane Helleringer’s comments on this blog. Have I missed them or they don’t exist?

IPA’s weekly links

Guest post by Jeff Mosenkis of Innovations for Poverty Action.

  • With the debate over deworming in danger of overtaking actual worms in terms of lost productivity, a reminder that much arguing about what analysis methods were chosen can be solved with a pre-analysis plan. The Journal of Economic Perspectives has two helpful articles:
    • Olken’s Promises and Perils of Pre-analysis Plans goes over a checklist of advantages, such as allowing for choosing unconventional tests without accusations of cherry picking, and tips for compromises, like making some of the initial data cleaning choices “blinded” (without separating treatment/control).
    • Coffman and Niederle argue Pre-analysis Plans Have Limited Upside, Especially Where Replications Are Feasible, and the problem is really that there’s little incentive for academic researchers to replicate.
    • One thought: A stats prof of mine who was a former physicist in his 80’s (at least), used to say back when stats were done by hand or after waiting days for a turn at the university’s basement-sized mainframe computer, people chose their stats tests and planned their analysis far in advance and much more carefully. When there was a “cost” to each analysis, the process was slower & more deliberative – essentially right in between “pre-analysis plan before study starts” and “test as you go.”

And Nigerian lawyer and satirist Elnathan John offers the Gospel According to Aid.

This tool will help you engineer the results you want from randomized trials!

Clinical drug trials are conducted by pharmaceutical firms to establish the effectiveness and safety of new treatments, but failure to publish the results of all trials is skewing medical science (as highlighted in our story from this week’s Science section). Using our interactive simulator, run a series of clinical trials for yourself and discover how to play the system by publicising results in favour of your own product

The Economist has a clinical trials simulator. Hat tip Ben Goldacre.

What if Spiderman were black, and Uncle Ben was shot by police?

I keep thinking how much more powerful the Spiderman origin story would be if Peter Parker was an African American kid, whose Uncle Ben was shot by police while being arrested for a minor parking infraction. There is no formal investigation, and Peter decides to put himself on the line to prevent it happening again. He tackles the white crimes that go unpunished, punishes POC criminals fairly. He is the leveler, always fighting to be without bias, to be just. To protect people like his uncle.

This not only mirrors so much of what’s happening in America, but feeds right into the complex relationship between Spiderman, the authorities and the media.

Peter Parker is a brilliant student, awkward, a nerd, but is branded a thug, a gang member, a criminal, because of his appearance. The media latch on to that and misrepresent him totally.

The police, humilitated by the fact that he refuses to work with them and often punishes cops themselves for brutalizing innocent people, or guilty people who still deserve better treatment than they get, attempt to hunt him down.


Best comment: “J. Jonah Jameson’s attitude would be remarkably unchanged in this scenario.”

Hat tip to Suresh Naidu.

The 10 things I learned in the trenches of the Worm Wars

If you have no idea what I’m talking about, either count yourself lucky or see yesterday’s post. The rest of you, carry on.

  1. One of the things I love about the Internet is that it brought a lot of very smart people to a key intellectual problem, the discussion brought out the central assumptions and claims, and they were answered within about a day or two. See Berk Ozler, for example. My conclusion is that the Kenya deworming results are relatively robust. Yay hive mind.
  2. On the other hand, the hive mind has a tendency to be grumpy, rude, shrill and angry. I found the debate more dignified than some, but vicious at times.
  3. I am guilty as well. I was too quick to suspect and insinuate bad faith on the part of the replicators. I can hold suspicions but I shouldn’t publicly insinuate or accuse without grounds. I am sorry for that.
  4. I do find any big, coordinated media push of a scientific finding to be problematic, to say the least. This drove and drives my suspicion of bias, even if accidental.
  5. To me the big failure in this entire business was by the editor of the academic journal. The competing claims on whether or not the results are fragile or not, and why, should never have been allowed to remain ambiguous.
  6. To me, a glitzy media push undermined the credibility and intentions of the journal further. This is a general problem in medicine and hard science that I do not see as much in social science (where the journals could care less about news coverage).
  7. On the journalist side, I can’t blame any of the writers for not following the finer statistical points. I had trouble myself. But almost none of the journalists read the reply by Miguel and Kremer (published by the same journal) and maybe none called Miguel or Kremer on the phone. I am told I was the first. Tell me if I’m wrong, but isn’t this the definition of sloppy journalism?
  8. I think the deworm the world movement has also tended to exaggerate or selectively quote the evidence in order to justify the cause. GiveWell has a much more balanced account: the evidence is not that great, but it’s good enough. Sort of. I thought GiveWell had one of the best posts. Do read it.
  9. To me, the real tragedy is that, 18 years after the Kenya deworming experiment (which was not even a real experiment) we do not have large-scale, randomized, multi-country, long-term evidence on the health, education, and labor market impacts of deworming medicine. This is not some schmuck cause. This is touted as one of the most promising development interventions in human history.
  10. I also fear for the reputation of replications in development economics. I imagine some of the problems could be addressed by getting more clarity into pre-analysis plans for replications. But the incentives to make mountains out of molehills is huge. Maybe everyone should sign a “no glitzy media push” pledge.

Okay, I am DONE with worms.

IPA’s weekly links

Guest post by Jeff Mosenkis of Innovations for Poverty Action.

Our apologies, the links are a bit late, but you’ll never believe what happened this week:

  • So Ted Miguel …. was on NPR’s All Things Considered, talking about why plans to use small solar panels to power Africa isn’t the answer, when most Kenyans already live right under power lines.
  • And in open science/replication news – some researchers at Berkeley shared their data publicly – and another research team beat them to publication. (Follow up from the researchers.)
  • Also, in a respected British source … Survey finds 60% of problems in replications can be solved if research teams would just talk to one another.
  • Seriously, new replications of the classic deworming findings reaffirmed some conclusions, but also called some into question, along with the Cochrane collaboration. Then twitter exploded. Discussion on Chris’ blog as well as GiveWell’s independent analysis both suggest unchanged policy and investment recommendations.
    • Other views from the Guardian, Vox, and Buzzfeed, original authors (PDF), and Berk Ozler. (It’s worth noting when reading general audience reporting on this, how statistical terms like “error” and “bias” with dual meaning can cloud the discussion when repeated). 
  • We’ve reported on the new Chinese-led development bank, there’s now a newer BRICS (Brazil, Russia, India, China and South Africa) development bank. (Literally, it’s called the New Development Bank).
  • The RISE program is accepting proposals for education research through August 1:

    RISE is a new multi-country research programme that aims to build an understanding of education systems and how they can be transformed to accelerate learning for all.


Dear journalists and policymakers: What you need to know about the Worm Wars

One of my favorite science writers, Ben Goldacre, enters the so-called Worm Wars. He’s not alone, with a flurry of new articles today. The question is simple: is a deworming pill that costs just a few cents one of the most potent anti-poverty interventions of our time?

Below is the picture from Goldacre’s post. I assume Buzzfeed editors chose it. It’s a nice illustration that nothing you will read in this debate is dispassionate. Everyone wants one thing: your clicks (and retweets, and likes, and citations). Most writers sincerely want the truth too. Sadly the two are not always compatible.

In brief: Ted Miguel and Michael Kremer are Berkeley and Harvard economists who ran the original deworming study that showed big effects of the medicine on school attendance in Kenya—one of the few to attempt to measure such impacts. That study ignited the impact evaluation movement in international development, especially through their students (like me). It also ignited a movement to deworm the world. This is a big claim, worth investigating. Calum Davey led the team who did a replication.

I know this study. In fact, as a first year graduate student I spent a summer working for Miguel and Kremer designing their long term follow up survey. Relationships are incestuous on all sides of the deworming debate, so you can hardly call me an impartial judge. Nonetheless, bear with me as I try.

I haven’t paid much attention to the deworming world for more than a decade. So I spent last night and this morning reading as much as I could. There’s an overwhelming amount to process, but I’ve drawn a few early conclusions.

The bottom line is this: both sides exaggerate, but the errors and issues with the replication seem so great that it looks to me more like attention-seeking than dispassionate science. I was never convinced that we should deworm the world. There are clearly serious problems with the Miguel-Kremer study. But, to be quite frank, you have throw so much crazy sh*t at Miguel-Kremer to make the result go away that I believe the result even more than when I started. Continue reading

Statistician Neal Beck just justified my longstanding hatred and loathing of logit

Neal is probably horrified by my slightly inaccurate title, but we all know this blog ain’t the New York Times.

In 2010 I was on sabbatical at NYU’s political science department. Neal asked me why I always used ordinary least squares regressions (OLS) when my dependent variable was a 1 or 0. Why not logit instead of this linear probability model? He had seen economists do this before, and was surprised at my reply—that it had become general practice in a lot of applied economics.

I think the reasons economists often go the linear route is because it generates very simple to interpret estimates, which is not the case with logit. And they are basically correct, with logit unlikely to yield a different answers. Clarity wins in my book.

It interested him enough to run some simulations, and what he found didn’t dissuade me from my sloppy clarity. Several years later, I get an email from Neal titled “At last!” It is a paper. The abstract:

This article deals with a very simple issue: if we have grouped data with a binary dependent variable and want to include fixed effects (group specific intercepts) in the specification, is Ordinary Least Squares (OLS) in any way superior to a (conditional) logit form? In particular, what are the consequences of using OLS instead of a fixed effects logit model in respect to the latter dropping all units which show no variability in the dependent variable while the former allows for estimation using all units.

First, we show that the discussion of fixed effects logit (and the incidental parameters problem) is based on an assumption about the kinds of data being studied; for what appears to be the common use of fixed effect models in political science the incidental parameters issue is illusory.

Turning to linear models, we see that OLS yields a perhaps odd linear combination of the estimates for the units with variation in the dependent variable and units without such variation, and so the coefficient estimates must be carefully interpreted.

The article then compares two methods of estimating logit models with fixed effects, and shows that the Chamberlain conditional logit is as good as or better than a logit analysis which simply includes group specific intercepts (even though the conditional logit technique was designed to deal with the incidental parameters problem!).

Related to this, the article discusses the estimation of marginal effects using both OLS and logit. While it appears that a form of logit with fixed effects can be used to estimate marginal effects, this method can be improved by starting with conditional logit and then using the those parameter estimates to constrain the logit with fixed effects model. This method produces estimates of sample average marginal effects that are at least as good as OLS, and better when group size is small. However, this is based on simulations favorable to the logit setup.

So it can be argued that OLS is not “too bad” and so can be used when its use simplifies other matters (such as endogeneity). These issues are simple to understand, but it appears that applied researchers have not always taken note of these issues.

The problem with evidence based policy change is we don’t have evidence on the important policies

Peter Singer has a Boston Review piece telling us we should all be “effective altruists”—to make a difference by giving our time and our money, and giving only to causes that demonstrate their effectiveness through evidence.

There are many good replies, including from Angus Deaton and Daron Acemoglu. Here is Acemoglu:

More evidence is always preferred, but precise measurement of the social value of a donated dollar may be impossible. What is the social value of a dollar given to Amnesty International as opposed to Oxfam or an NGO providing vaccines or textbooks?

…But the problem is thornier still. A large body of research shows that economic development is the best way to lift millions out of poverty and improve their health, education, and access to public amenities. So one has to take into account how charities’ activities affect economic development, which is essentially impossible. If, as some economists and political scientists suggest, changes in political and economic institutions are critical for long-run economic growth, then watchdog organizations such as Amnesty may be essential for transforming dysfunctional regimes. Effective altruists don’t (yet?) see the importance of these more political organizations.

To his critique (and Deaton’s as well): Yes! At the same time, some reservations:

  • It’s hard to overstate how many stupid and dead end causes people give money to. Singer probably sees very rich people giving to idiotic boutique causes all the time. I avoid those people, but I have to contend with the World Bank and others spending billions on things like vocational training, which has basically zero impact. This is another way of saying Singer is right on the margin, Acemoglu is right as we move away from the marginal decision.
  • I’m not worried about “too much” effective altruism. It would be a problem if it happened, but the world won’t even get close. Aid donors and the very rich are (1) stubborn, and (2) don’t read.
  • But Acemoglu is right that institutional and political change are more important and the evidence-based crowd have done very little here. Most of that evidence is about anti-corruption or election monitoring or other things that I doubt change politics very much.
  • Meanwhile all the good political economy research (like Acemoglu’s) has no clear implication for social and political change in the world. There is a big disconnect. These scholars have mostly ignored this gap either because… I don’t know why. Maybe it’s too treacherous or hard, or they don’t find it interesting enough, or they are cynical about policy change. I don’t know. Someone explain it to me.
  • Actually, this is not entirely true. You could view a lot of research says “you should stop violent conflicts, and here are concrete steps to do so”. I can think of few better short-term investments. More work along these lines strikes me as a good thing.

If fixing gender imbalances in academia didn’t seem hard enough already…

We analyze how a larger presence of female evaluators affects committee decision-making using information on 100,000 applications to associate and full professorships in all academic disciplines in two countries, Italy and Spain.

These applications were assessed by 8,000 evaluators who were selected through a random draw. A larger number of women in evaluation committees does not increase either the quantity or the quality of female candidates who qualify. If anything, when evaluators’ are not familiar with candidates’ research area, gender-mixed committees tend to be less favorable towards female candidates than all-male committees, with the exception of evaluations to full professorships in Spain.

Data from 300,000 individual voting reports suggests that men become less favorable towards female candidates as soon as a woman joins the committee.

Article. You may be thinking, “oh Southern Europeans are not like us” but I am not so sure.

Then again, my colleague Bob Erikson finds that female judges on US appellate courts influence the votes of male judges to be more liberal on sex discrimination cases.

Comics guru Alan Moore eats his young

To my mind, this embracing of what were unambiguously children’s characters at their mid-20th century inception seems to indicate a retreat from the admittedly overwhelming complexities of modern existence…a

It looks to me very much like a significant section of the public, having given up on attempting to understand the reality they are actually living in, have instead reasoned that they might at least be able to comprehend the sprawling, meaningless, but at-least-still-finite ‘universes’ presented by DC or Marvel Comics.

I would also observe that it is, potentially, culturally catastrophic to have the ephemera of a previous century squatting possessively on the cultural stage and refusing to allow this surely unprecedented era to develop a culture of its own, relevant and sufficient to its times.

Article. It’s an interesting interview.

I personally enjoy a good superhero movie. Presumably the entertainment industry will always find a way to serve people their childhoods twenty to thirty years later. This is normal. Even My Little Pony is back in force.

A healthy art will do more than this. While serving the kitsch, it will push the frontier of what’s possible. I think science fiction has been doing this. Fantasy not so much. Comics I can’t say, since I haven’t been paying attention. Readers? Is there an artistic frontier?

Bleg: Colombia with small children

We leave in a week and so, naturally, we have just begun to plan.

Actually, that’s not entirely true. From July 25 until August 8 we have a home exchange in Bogota, and I have an office at the excellent Universidad de los Andes, and many work-related plans. But the two weeks from August 8 to 23 are completely unscripted (so far) except for the knowledge we will be trying to keep ourselves relaxed and a 2- and 4-year old stimulated. Suggestions welcome.

Based on past recommendations, we’ve been thinking about (a) Santa Marta and a nearby natural reserve, (b) coffee fincas or other spots in the coffee triangle, and (c) Medellin. Recommendations of specific hotels, resorts, sights or restaurants welcome. (Hotel/resort wise, our tastes run more towards isolated rustic cabanas with intermittent electricity over full-service family resorts. Though I won’t say no to babysitting.)

If we have missed the ideal destination, do say so. Our plans are very flexible (read: disorganized). With small kids we find it easiest to stay in one place for a few days or a week and make day trips, so those locales are most attractive.

IPA’s weekly links

Guest post by Jeff Mosenkis of Innovations for Poverty Action.

  • It’s remarkably hard to solve the puzzle of why small and medium enterprises in the developing world don’t grow more, in fact IPA has a whole program devoted to answering that question. Training programs often don’t work, but new RCT results from Egypt show one program that does – “learning by exporting” – giving carpet makers access to foreign markets led to productivity and quality improvements that stuck, and higher profits. More on the blog here.
  • The Millennium Villages Project, which takes a grand (and expensive) approach to reshaping systems in a select number of poor villages, has come under significant criticism, both for results (or lack thereof) and for not having included a robust evaluation plan to tell if the $120 million initially raised was worth it. Apparently there has been a new evaluation plan accepted as a paper into the Lancet, but Michael Clemens points out it still hides the overall cost of the project, which is crucial to making decisions on any intervention. More on their analysis plan here. (The Japanese government just gave an additional $1.4 million to the project in Rwanda). 3ie video of a talk starting at midline results here.
  • As the RISE program gets ready to embark on six years of research on improving education in the developing world, Lee Crafurd and Justin Sandefur present twenty-five of the best ideas out there as a good place to start.
  • A new paper (PDF) in PLoS Neglected Tropical Diseases reports worm infection rates of over 40% among school children in southern China, with lower cognitive functioning among infected children.
  • The non-profit GiveDirectly which just gives cash to poor people, in addition to publishing real time performance data on their website, randomly chose six of their first recipients from 2012 to return to and see how they are doing. On their blog, they report what recipients said the money did for them. (IPA’s evaluation of GiveDirectly is here.)

And: Robert Smith from NPR has a new macroeconomic indicator – banks in New York are running out of Euros because of all the reporters flocking to Greece.

But the best thing in recent memory comes from Chris if you didn’t see it earlier. Princeton Economist Uwe Reinhardt, dissatisfied with economists’ ability to do anything for global economies, is giving up and wrote a syllabus for teaching Korean Television Drama (PDF).

The UCLA sexual harassment case that every professor should be aware of

This sexual harassment case at UCLA is jaw-dropping. From one plaintiff’s complaint, against history Professor Gabriel Piterberg:

51. He then started talking about the famous philosophers Hannah Arendt and Martin Heidegger, who met when Arendt was Heidegger’s student and subsequently carried on a clandestine love affair for more than forty years. He told her that relationships like theirs were normal and that “If it is done right, professor and student relationships are supposed to be intimate.”

52. Professor Piterberg then told her that he masturbated while imagining the two of them together.

53. Throughout this meeting, Plaintiff Takla continued to voice her discomfort with him as her advisor and his comments, but Professor Piterberg was upset with Plaintiff Takla for wanting a new advisor. He told her, “If anything happened between us, it might be while you are writing the conclusion to your dissertation.”

This is a small fraction of the terrible things alleged. There are two women with similar complaints so far. The most staggering aspects: the UCLA ombudsperson effectively hushes both. As did the victim’s other adviser. And beyond this institutional failure, a disturbing detail is that both the ombudsperson and other adviser were also women, and apparently also aware of other complaints.

This article summarizes, but the full text of the legal complaint is so much more powerful and disturbing. And important for professors to read. It is short, and you will find it hard to put down.

With the caveat that these are allegations for the time being, some reflections:

  • How many times has this happened before over two decades with this faculty member? How many times has this been hushed by the university, or a colleague, or self-censoring? Staggering.
  • A friend commented: this is the culture of humanities profession, where older male professors compensate for relatively poor salaries with these non-wage benefits. That’s an exaggerated and unkind interpretation, but I can’t convince myself it’s false.
  • Actually the other cause might be undue concentration of power. In smaller, more specialized, fragmented disciplines, where the costs of switching sub-disciplines are high (e.g. learn a new country and language), advisers will exert more power over their students. So these disciplines might want to think about how to break down internal disciplinary barriers to decentralize power.
  • Not all the facts are known, but the big failure to me is the institutional one: the UCLA coverup. There will always be deviant individuals. The institutional failure to investigate and prosecute is shameful. It’s like 1990 in the Vatican.
  • I’m pretty sure most big organizations and universities would behave in the same way, if allowed. This is not a UCLA problem.
  • Some colleagues of mine criticized the media coverage of the Lacour scandal—the UCLA student who faked a gay marriage study. They felt that UCLA had a process and would take care of it. I disagreed then and I feel even more confident now. Big bureaucracies do not want to deal with this.
  • The most blatant case of academic fraud I ever caught? My university fumbled it so badly it had to be purposeful, and the culprit is now a prominent politician in his/her country.
    • No I won’t tell you whether this was Harvard, Berkeley, Yale or Columbia, as my experience is that none of them are that different in this regard.
  • There has been virtually no news coverage since the UCLA story broke in mid-June. This strikes me as ominous.
  • Here is political economist Michael Chwe on Project Callisto, a web-based system for sexual assault reporting under development.
    • Document and time stamp harassment as it happens to you. It goes nowhere, until at a later date when (a) it gets worse and you need the records, or (b) someone else accuses the same person and you can add your complaint more easily. (b) can be made automatic when someone else reports.

Links I liked

  1. A theory of civil disobedience
  2. Is Amy Schumer the Jon Stewart of feminism? This and this brilliant skit. She might be the most refreshing and talented comic today.
  3. Foreigners who love to love the President of Rwanda: Remember today as your day of Peak Kagame. Sadly, let the fall begin.
  4. What happens when you hold another man’s hand and walk around in Moscow
  5. A lego compatible prosthetic arm allows for kids to build their own attachments
  6. The Comic-Con promotion of Star Wars VII is basically one big message: Don’t worry, we are not George Lucas
  7. “Slavery was the earliest form of Social Security in the United States.” An Alabama history textbook.

Syllabus of the year

From Uwe Reinhardt, James Madison Professor of Political Economy and Professor of Economics and Public Affairs at Princeton University, a new lecture series:

After the near‐collapse of the world’s financial system has shown that we economists really do not know how the world works, I am much too embarrassed to teach economics anymore, which I have done for many years. I will teach Modern Korean Drama instead.

Although I have never been to Korea, I have watched Korean drama on a daily basis for over six years now. Therefore I can justly consider myself an expert in that subject.

Reinhardt is one of the world’s leading health care economists. And hence.

Finally, every good Korean drama has many scenes at super clean hospitals with good‐looking doctors and nurses. Koreans love their hospitals and seem to run to them whenever they have a cold or a headache or are lovesick or simply feel “stress.”

To an economist like me, this fondness for hospitals is surprising, because hospitals are expensive in Korea and much of the bill is not covered by Korea’s National Health Insurance system. Price‐elasticity of demand does not seem to work in Korean drama.

The introduction and first lecture (pdf).

hat tip to Selvin Akkus-Clemens.