Putting yesterday in perspective

If one were to create a dime-sized hole between thumb and forefinger and hold it out at arm’s length, in that small region the largest telescopes today, like those in Chile or Hawaii, could discern literally hundreds of thousands of other galaxies like our own Milky Way.

An NYRB article that I not only found humbling, but made me think we do not invest enough tim and money as a society in physics and astronomy.

…Since Edwin Hubble’s groundbreaking discovery in 1929 that the universe is expanding, we have recognized that the entire observable universe, all 100 billion or so galaxies, each containing 100 billion or so stars, was, some 13.8 billion years ago, confined to a region that was perhaps smaller than a single atom today. If this is the case, then the initial conditions that determined the origin, makeup, and nature of the largest cosmic objects today were determined on subatomic scales. So to understand the universe on its largest scales we ultimately must push forward our understanding of the fundamental structure of matter and forces on the smallest scales.

“Let me translate my emails for you”

“I’m just checking in.” = Where is that thing you promised I’d have by now?

“Sorry to bother you again.” = Why can’t you do your fucking job?

“I feel bad for making you do this.” = You should feel bad for not having done this already.

“This was helpful.” = This would’ve been helpful two weeks ago.

“Sorry if I somehow missed your email.” = We both know you never emailed me.

“Thanks for the explanation.” = If you had told me this last week, you would’ve saved me a lot of time.

“Perhaps there was a misunderstanding.” = You didn’t fucking listen to me.

More at McSweeneys. Hat tip to John Thorne.

The proletariatarian production of pop

A fascinating article on the economics of pop music.

There’s inequality of opportunity.

In the pop version of income inequality, perpetuating the gulf between haves and have-nots, the most successful performers get first dibs on the hottest producers and songwriters of the moment. Yet while singers come and go, an oligarchy of producers endures.

And pin-factory-like specialization and division of labor:

…Potential hits may also be assembled in high-pressure pop think tanks called “writer camps,” where a deep-pocketed star—Beyoncé, perhaps—convenes dozens of producers, composers, and lyricists in hotels and studios, where they run through every permutation of producers and topliners. The campers are often a mix of longtime pros and newcomers from the hipper fringes, sharing their innovations or eccentricities for the chance at a pop payoff. “Camp counselors” schedule teams to come up with a song before lunch, then reshuffle the teams to come up with another afterward, with daily playbacks to keep everyone competitive. “If the artist happens to be present,” Seabrook writes, “the artist circulates among the different sessions, throwing out concepts, checking on the works in progress, picking up musical pollen in one session and shedding it on others.”

In a different, less physically proximate kind of songwriting contest, one very simple track—a single beat or a chord progression—gets sent simultaneously to dozens of potential collaborators. Then the producer and singer might choose a verse from one response, a chorus from another, an instrumental hook from yet another: digital brainstorming. Many of the songwriter-producers in the book are blunt about describing their work more as a business than a form of self-expression—though that may be more a matter of our era’s MBA mentality, combined with a hip-hop culture of competitive striving gone mainstream. The Beatles wanted hits, too.

Preregistration of clinical trials causes medicines to stop working!

Screenshot 2016-02-29 21.25.38

Something must be done to combat this public health hazard. In 2000, the National Heart Lung, and Blood Institute (NHLBI) began requiring that researchers publicly register their research analysis plan before starting their clinical trials. From a new PLOS paper:

We identified all large NHLBI supported RCTs between 1970 and 2012 evaluating drugs or dietary supplements for the treatment or prevention of cardiovascular disease.

17 of 30 studies (57%) published prior to 2000 showed a significant benefit of intervention on the primary outcome in comparison to only 2 among the 25 (8%) trials published after 2000 (χ2=12.2,df= 1, p=0.0005). There has been no change in the proportion of trials that compared treatment to placebo versus active comparator. Industry co-sponsorship was unrelated to the probability of reporting a significant benefit. Pre-registration in clinical trials.gov was strongly associated with the trend toward null findings.

Hat tip @rlmcelreath.

Randomized trials revisited

Following my defense of randomized trials this week, come this thoughtful reader comment:

This response seems to miss, or perhaps obscure, the point. In my understanding, Hausmann is suggesting that development organizations take a Toyota-style approach to innovation, in which front-line workers have authority to adapt, make suggestions, and eventually change the way the organization works. In this case, power to innovate lies in the front-lines, among implementers.

In contrast, Blattman seems to depart from the premise that high-level managers, or academics, are the ones authorized to have ideas, and these ideas are then transmitted to the fieldworkers who implement them. Thanks to rigorous testing, the best ideas can be disseminated. Power is centralized, and held by the proper authorities.

So the debate is not about methods or rigor, it is about authority to innovate and power to decide.

I agree, but in that case we both have it wrong.

To see this, imagine a Toyota that gives customers a car whether they like it or not. Front-line worker innovation might or might not develop cars that people want to own. The problem is there’s no vote or market test where the ultimate users decide.

I think this is called the Lada.

This is the fundamental problem of aid. A bunch of planners have the power to decide and are only accountable to donors, most of whom seem happy to remain ignorant of the details or the actual success of their interventions.

People have put ahead randomized trials as an improvement over the current mess (including donors who quite fairly don’t know any other way to make this better).

Now, trial-and-error innovation combined with randomized trails could be more powerful. That was my point. But make no mistake: both are still the tools of the inept planner.

Of course, none of these are reasons that social scientists like randomized trials. They are interested in using these field experiments to try to test ideas or estimate parameters, sometimes to produce general knowledge for the public good. Maybe an adaptive mechanism could do this even better.

To extend the metaphor, I think that means we academics who do field experiments are the intellectual wing of the Communist Party, who have captured Lada’s production for our own purposes, both selfish and noble. The term randomista sounds more appropriate than ever?

How much of the data you download is made up?

On the sister blog I report on a new paper, “Don’t Get Duped: Fraud through Duplication in Public Opinion Surveys,” by Noble Kuriakose, a researcher at SurveyMonkey, and Michael Robbins, a researcher at Princeton and the University of Michigan, who gathered data from “1,008 national surveys with more than 1.2 million observations, collected over a period of 35 years covering 154 countries, territories or subregions.”

They did some forensics, looking for duplicate or near-duplicate records as a sign of faked data, and they estimate that something like 20% of the surveys they studied had “substantial data falsification via duplication.”

These were serious surveys such as Afrobarometer, Arab Barometer, Americas Barometer, International Social Survey, Pew Global Attitudes, Pew Religion Project, Sadat Chair, and World Values Survey. To the extent these surveys are faked in many countries, we should really be questioning what we think we know about public opinion in these many countries.

That is Andrew Gelman.

At quick glance, the paper’s approach to calling data “duplicated” is a bit crude, but I’ve worked with several of the survey firms that have produced these surveys in Africa. I have no trouble imagining that the data have very serious problems.

Of course, if 20% of the surveys have a 5% duplication problem, I don’t know that this makes them much worse than the other data scholars use. Researchers use national statistics or cross-national databases all the time as if the data are valid, while most are terrible. When I referee a paper, it’s obvious who knows whats under the data and who never bother to look, and simply download blindly.

But back to the surveys. Duplication is terrible, but the least of my worries. For instance:

  • The questions from most of these surveys sound perfectly sensible until you sit down and ask them to someone in a village. Then the absurdities become immediately apparent. Researchers: If you have a chance, print out any one of these surveys and test them out sometime. You will never operate the same again.
  • Then there’s the poor quality of much of the legitimately-collected data from rushed, tired, poorly incentivized enumerators.
  • Finally, these survey firms are for-profit enterprises with very different incentives and constraints than the researchers. They often have limited cash flow, middling middle management, and their average customer is a private firm or development agency that pays little attention to data quality.

If I want reliable data, mostly I do not use private survey firms. I hire and train teams myself (when I can through a local non-profit research organization or an international one like Innovations for Poverty Action). And if I must us a firm, I hire a researcher I trust to keep an eye on things full time. I recommend nothing else.

Links I liked

  1. Wild gorillas compose happy songs that they hum during meals
  2. “For the love of God, rich people, stop giving Stanford money.”
  3. For those of you aspiring to PhDs, Princeton is looking for applicants for its Emerging Scholars in Political Science program, with an emphasis on applicants from historically underrepresented groups.
  4. A history of randomized trials in social science
  5. What works in reducing community violence? A meta review
  6. I have a new curse: “Carry yourself with the confidence of Harvard students who believe they invented cabins.”

IPA’s weekly links

Guest post by Jeff Mosenkis of Innovations for Poverty Action.


  • If your work involves transcribing audio or video, Trint looks interesting. It extracts the text and synchronizes the result with the audio/video file alongside it to make it easier to go through and make corrections (h/t Brian Boyer).
  • UN Secretary-General Ban Ki-moon endorsed using cash transfers whenever possible in humanitarian crisis aid (h/t GiveDirectly).
  • On the other hand when it comes to development in general, Angus Deaton reaffirmed at the Council on Foreign Relations that he’s just as skeptical of cash transfers as about other outside aid. He feels (as I understand it) that having outside governments and NGOs doing the jobs local governments should be doing undermines the contract between those governments and their own people. He also fears cash transfers, if they get big enough, will also attract the attention of greedy officials:

    “…it’s the unintended consequences of what happens in the long run if you give people lots of money. And, you know, if there’s enough money out there, the guys that run the country, you know, they’re going to get it. I mean, that’s what happens. That’s the nature of power.”

  • EconTalk has an interesting episode on “medical reversals.” Up to 40% of treatments that become common based on initial data turn out not to be helpful, and sometimes harmful, years later when the RCT data comes in. There’s a nice comparison about 45 min in between the dilemmas that health vs. financial regulators face.
  • Among the Hillary Clinton emails made searchable by The Wall Street Journal, is a compelling one from Chelsea Clinton who went to observe in post-earthquake Haiti. After being there for a short time, she paints a pretty vivid portrait of disorganized NGO/UN efforts.
  • American behavioral econ draws on the Kahneman and Tversky school of thought exploring flaws in human reasoning. The Bounded Rationality approach associated with Gerd Gigerenzer focuses on how the mental shortcuts we use are often adaptive. The Max Plank Institute in Berlin is hosting a summer institute for scientists interesting in learning more (h/t Decision Science News).

And after the uproar surrounding the American hunter who killed Cecil the lion, fewer hunters have come to a Zimbabwean park. It now has an overpopulation of lions and officials there are considering killing 200 of them.

Photo above via Flickr/brianscott

More reasons to bemoan millennials

Sorry guys. I know you’re my main audience, but this is my generation’s new favorite pastime. And there is new material.

On Monday, the New York Times published a story about the breakfast favorite, and the most disconcerting part was this:

Almost 40 percent of the millennials surveyed by Mintel for its 2015 report said cereal was an inconvenient breakfast choice because they had to clean up after eating it.

The industry, the piece explained, is struggling — sales have tumbled by almost 30 percent over the past 15 years, and the future remains uncertain. …

A large contingent of millennials are uninterested in breakfast cereal because eating it means using a bowl, and bowls don’t clean themselves (or get tossed in the garbage). Bowls, kids these days groan, have to be cleaned. …

Via James Choi.

Has the randomized trial movement has put the auditors in charge of the R&D department?

My main problem with RCTs is that they make us think about interventions, policies, and organizations in the wrong way. As opposed to the two or three designs that get tested slowly by RCTs (like putting tablets or flipcharts in schools), most social interventions have millions of design possibilities and outcomes depend on complex combinations between them. This leads to what the complexity scientist Stuart Kauffman calls a “rugged fitness landscape.”

Getting the right combination of parameters is critical. This requires that organizations implement evolutionary strategies that are based on trying things out and learning quickly about performance through rapid feedback loops, as suggested by Matt Andrews, Lant Pritchett and Michael Woolcock at Harvard’s Center for International Development.

RCTs may be appropriate for clinical drug trials. But for a remarkably broad array of policy areas, the RCT movement has had an impact equivalent to putting auditors in charge of the R&D department. That is the wrong way to design things that work. Only by creating organizations that learn how to learn, as so-called lean manufacturing has done for industry, can we accelerate progress.

That’s Harvard’s Ricardo Hausmann writing in Project Syndicate.

I had the following reactions:

  • Absolutely, organizations should be doing innovating through rigorous trial and error. And the case needs to be made, since many organizations don’t know how to do this.
  • But let’s be honest: most governments and NGOs did not have R&D departments that got hijacked by randomized trials. Most organizations I know were not doing much in the way of systematic or rigorous research of any kind. Outside one or two donors and development banks, the usual research result was a mediocre consulting report rigged to look good.
  • In fact, most organizations I know have spent the majority of budgets on programs with no evidence whatsoever. In the realm of poverty alleviation, for example, it turns out that two of the favorites, vocational training and microfinance, have almost no effect on poverty.
  • This goes to show that, without a market test, some kind of auditing or other mechanism is probably needed. Especially the money-wasting behemoths of programs that are still so common.
  • Sometimes the answer will be large-scale randomized trials. The way I see it, trial-and-error-based innovation and clinical trials are complements not substitutes. Most of the successful studies I’ve run have followed a period of relatively informal trial-and-error.
  • There are a few radicals in academia and aid who say everything should have a randomized trial, but I think the smart ones don’t really mean it, and the others I don’t take seriously. They are also the exception. If you look at the research agenda of most of the so-called randomistas, experiments are only a fraction of their work.
  • In political science, the generation before me fought (and still fights) the methodological war. My generation mostly gets on with doing both qualitative and quantitative research more harmoniously. I feel the same way about the randomista debate. People like me do a little observational work, a little forecasting, a little qualitative work, some randomized trials, and I’m even starting to do some trial-and-error style work with police in Latin America. I don’t think I’m the exception.
  • If anything, the surge of randomized trials have paved the way for rigorous trial-and-error. I’ve seen this at my wife’s organization, the International Rescue Committee. Eight years of randomized trials showed their organization and their donors that some of their biggest investments were not making a difference in the lives of poor people. This has built a case for going back to the drawing board on community development or violence prevention, and now they are starting an R&D lab that looks very similar to Hausmann’s vision. They can do this because expanding a research department to manage randomized trials brought in the people, skills, and evidence base to make a case for innovation.
  • There are some structural problems in academic research that make this hard. Organizations like Innovations for Poverty Action and the Poverty Action Lab have drawn bright red lines around randomized trials, and most of the time don’t facilitate other kinds of research. But I can see adaptive and rigorous innovation fitting in.
  • (Updated) Some people have said “oh but there are too many randomized trials and too much emphasis.” This is the nature of new research technologies. People overdo them at first, since the opportunities are so large. Not so long ago everyone ran cross-country regressions, or wrote a little theoretical game. These are still useful, but they’ve receded as new methods appear. So, this too will pass. Randomized trials will join the pantheon of mediocre methods at our disposal. (The saddest statement is that, to the aid industry, and to much of social science, a randomized trial is “new”. Scientists are aghast at this.)

My view: we can push rigorous trial and error up without pushing other approaches to learning down.

Podcast recommendations

  1. I’m a new fan of The Weeds, a podcast where Ezra Klein, Sarah Kliff, and Matt Yglesias  talk about US politics, usually starting with a new research paper. I always learn something. Last week’s episode, for instance, was on why the US government might have been better at getting things done when it was more corrupt and transactional.
  2. Klein also just started a new long firm interview podcast, The Ezra Klein Show. There have been four interviews, two previews on the Weeds feed, with Ben Bernanke and Ross Douthat, and two on the new feed, with Rachel Maddow and “superlobbyist” Tony Podesta. By the standard of “risking my life to write lots of notes to myself while on a treadmill” the Douthat and Maddow interviews are fantastic.
  3. I’ve been enjoying Sampler, a podcast about podcasts. Host Brittany brings a truly eclectic mix, with unusual co-hosts, including actor Molly Ringwald or comedian W. Kamau Bell. More entertaining than intellectual, but some of both.
  4. Dani Rodrik’s conversation with Tyler is a must-listen for anyone interested in globalization or development. The Kareem Abdul-Jabbar interview was not bad, but I found his answers lacking in depth for someone Tyler thinks is one of the great intellectuals of the day.


Development as coercion

Worldwide, extreme poverty is often concentrated in spaces where people and property are not safe enough to sustain effective markets, and where development assistance is dangerous – and might even induce violence. Expanding governance by coercively taking control of territory may enable markets and development programs, but costs to local residents may exceed benefits, especially if that expansion is violent.

We estimate for the first time whether a large counterinsurgency program improves welfare. We exploit the staggered roll-out of the Philippine “Peace and Development Teams” counterinsurgency program, which treated 12% of the population between 2002 and 2010.

Though treatment temporarily increased violence, the program progressively reduced child malnutrition: by 10% in the first year, and by 30% from year three onwards. Improved nutritional status was not due to increased health and welfare expenditures, but instead to improved governance. Treatment effects are comparable to those of conventional child health interventions, though conventional programs are likely infeasible in this setting.

Rebels apparently react to treatment by shifting to neighboring municipalities, as malnutrition worsens there – with statistically significant ‘treatment’ effects of similar size. Thus overall program effects are close to zero. These findings invite an evidence-based discussion of governance expansion, an extensive margin of development.

A new paper by Eli Berman, Mitch Downey, and Joseph Felter.

I was a discussant for this paper at the NBER last year. I remember being conflicted with both admiration and caution.

On some level, the message is the antithesis of Amartya Sen’s. Development through force rather than freedom.

It is easy to find this kind of work distasteful and dismiss it, but as far as I know Sen never dealt with the thorny problem of what to do in places where the state is weak and armed men with guns are trying to seize power.

Something you learn in political science, but sometimes gets overlooked elsewhere: For most of history, the thing we call development has been associated with a state, and most states arose through force and coercion. In fact, most states look like organized crime. However miserable, the thuggish state is typically better than the alternative: the roving thug with guns. So it might be naive to think that development is always and everywhere a peaceful process.

There are always winners and losers from policy, but here the stakes seem a little higher: If a state-building intervention saves the lives of 1000 children but kills 1000 young men, this is not an easy trade-off to make. At least for some.

What makes this more tricky is the question: who gets to decide? “A more-or-less democratically-elected Philippine government” seems like a reasonable answer, though none of the children or most of the young men whose lives are in the balance had a chance to vote.

There are good governments and bad. Some are worse thugs than others. So maybe the answer is to back the good guys. But I get a little worried about our ability to judge. I have uncomfortable memories of Ronald Reagan describing the Nicaraguan contras: “These gentlemen are the moral equivalent of the founding fathers.” (Those of you scoffing at Reagan: I could have easily picked a Democratic President as an example.)

These trade-offs are going to become more common, and more difficult, as development efforts focus more on the ungoverned spaces of the world. In 15 years, most other places will be getting rich. If you are working in aid, the issues in this paper are your future.

College women aren’t feminist enough… yet?

Kate Cronin-Furman & Mira Rapp-Hooper on why young women are not excited about electing a woman President:

Looking around in college or grad school, it’s easy to believe that, in the United States at least, gender equality has largely been achieved. It was for us: We both self-identified as feminists from an early age, but it was a feminism mostly concerned with solidarity—with survivors of wartime sexual violence, with women campaigning against female genital mutilation in their communities, with girls forced to work inside the home instead of going to school. It didn’t feel like our rights or well-being on the line.

But it turned out they were — we just didn’t know it yet. …Because once women enter the professional world, the rosy picture of progress begins to dull.

…These dynamics can be a rude awakening for young women who have excelled all their lives, often at institutions that have invested resources, time, and attention into recruiting promising women. They’re experiencing something we call “late-breaking sexism.” It’s the sudden realization that you don’t have the same opportunities as a man, that you will struggle to have both a family and a career, that your participation in the public sphere will always be caveated by your gender.

White people sleep better too

According to a new report from the Centers for Disease Control and Prevention, about 65 percent of Americans polled said they usually get at least seven hours of sleep per night, the benchmark recommendation. It’s self-reported data, not confirmed with any kind of tracking, but it’s fairly consistent with other estimates, the CDC says.

When the responses were broken down by race, they found that non-Hispanic whites had the highest rate of healthy sleep duration, at 66.8 percent. Close to 66 percent of Hispanics got seven-plus hours, as did 62.5 percent of Asians and 59.6 percent of Native Americans. Black people were at 54.2 percent, and multiracial people were at the bottom, with 53.6 percent. Overall, people who were employed and college-educated slept better, too.

Source: The Science of Us.

21st century prison diaries

From one of an amazing, and funny, series of articles by Barrett Brown, an imprisoned journalist.

In fact, the gangs really don’t have control over the prison. But then neither does the administration, if by “control” we mean the ability to make uncontested decisions over what happens within a given space, in which case control is always a matter of degree. The federal and state governments of the United States, for instance, exercise some degree of overlapping control over their territory, but not to such an extent that the various law-enforcement agencies — er, law enforcement agencies — arrest any but a small minority of residents who violate the law.

This is just as well, since the law requires that the tens of millions of Americans who use drugs or gamble or involve themselves in prostitution be imprisoned — and that’s not even counting federal law, which, as convincingly estimated by civil liberties attorney Harvey Silverglate in his book Three Felonies a Day, the average American unwittingly violates every day. And thus it is that the U.S. can continue to exist above the level of an unprecedented gulag state only to the extent that its laws are not actually enforced — an extraordinary and fundamental fact of American life that one might hope in vain to see rise to the level of an election issue, but which is at least worth keeping in mind when it comes to the debate over whether or not we should keep granting the state ever more powerful methods of surveillance until it becomes the All-Seeing God Against Whose Laws We All Have Sinned. (Personally I’d vote “no,” but then I’m a felon and can’t vote anyway.)

…The reality is that control is shared by way of a sort of makeshift federalism that varies in particulars from prison to prison but in which real power is always divided among the various gangs, the staff, and local and regional administrators in an arrangement that’s best described as a cross between the old Swiss canton system and China during the Warring States period, which I’ll be the first to acknowledge is not especially helpful. Suffice to say that it will take me the remainder of my sentence to provide a real sense of this remarkable state-within-a-state and its inimitable politics — the politics of the literally disenfranchised, who live their lives in the very guts of government without being able to rely on its protections, and so are forced to provide their own. Really, it’s a state-within-a-state-within-a-state.

Hat tip to Patrick Ball.

IPA’s weekly links

Guest Post by Jeff Mosenkis of Innovations for Poverty Action.


  • We’ve collected some of our favorite #DevelopmentValentines from last week.
  • Africa is a Country has a new economics column.
  • The New Yorker has the great story of the science behind the announcement last week of detection of faint gravitational waves. It involved decades of collaboration, $472 Million, spheres held in place by glass strands so thin that if touched they would disintegrate, and moles on the research team who hid false positives in the data. They ended up with a system so sensitive it had to counterbalance interference from a wolf walking nearby or a lightning strike in Africa.
  • Get Uganda election updates from reporters Simone Schlindwein and Malcolm Webb.
  • A sociology Ph.D. student published an ethnography (open access) after working in three infant cognition labs about the process by which researchers make an inherently messy topic (unpredictable infant behavior) seem clean, scientific and publishable. In the process he documents processes like selective inclusion/exclusion of data, peeking at data as it comes in and redirecting the study if it looks like it won’t turn out as expected, and HARKing (Hypothesizing After Results Known).
    • Sanjay Srivastava has a summary of the article here and a deeper discussion here.
    • If you wonder why a researcher would do this, read this tweetstorm from a researcher whose paper was rejected because the findings weren’t statistically significant and what it means for publication bias.
    • Recall what Chris has pointed out before, whose research practices would come out of deep outside scrutiny unscathed?
  • The Global Fund to Fight AIDS, Tuberculosis and Malaria has a “repo man” who goes after governments to recover missing or misused grant money.

Legislators need to follow Eliot Engels’ lead in naming their bills: