Can we use data and machine learning to predict local violence in fragile states? As it turns out, yes.

After years of working on program evaluation and related things, it is with great joy that I toss causation out the window and learn to data mine.

A few years ago, a foundation said to me, “hey, all that data you’re collecting to study property disputes and other violence in Liberia–could you use it to test early warning systems for riots and major crimes?” My reaction: “That sounds crazy. As if that’s possible.” Their response, “We will fund your survey if you try.” My reply: “Did I say crazy? I meant that sounds like a great idea.”

After six years of data collection, Rob Blair and Alex Hartman and I finally have a paper:

We use forecasting models and new data from 242 Liberian communities to show that it is to possible to predict outbreaks of local violence with high sensitivity and moderate accuracy, even with limited data.

We train our models to predict communal and criminal violence in 2010 using risk factors measured in 2008. We compare predictions to actual violence in 2012 and find that up to 88% of all violence is correctly predicted. True positives come at the cost of many false positives, giving overall accuracy between 33% and 50%.

From a policy perspective, states, international organizations, and peacekeepers could use such predictions to better prevent and respond to violence. The models also generate new stylized facts for theory to explain.

In this instance, the strongest predictors of more violence are social (mainly ethnic) cleavages, and minority group power-sharing.

This is not precisely “big data” in that it’s a small number of villages and three years of events. But it’s “big” in the sense of having lots and lots of detailed information about the villages themselves, which is rare. We think of this as a pilot, or proof of concept for the approach, and plan to test it next on much bigger data from other countries.

The most interesting finding, to me, was how power-sharing at the local level was associated with more violence. There’s actually a number of papers looking at national power-sharing right now that find the same thing. And yet the common political response to a crisis nowadays is to push for power-sharing. Worth investigating.

I would have liked to name this paper “I just ran 32 million regressions,” but besides other drawbacks, the more honest title would be “My RA just ran 32 million regressions,” which is slightly less compelling.

Cash transfers to Syrian refugees: The evaluation

Fine, cash transfers work okay in Kenyan villages, but should the world use them in wars and refugee crises? Apparently yes.

In the first scientifically rigorous evaluation of emergency cash for refugees, the International Rescue Committee unveiled striking findings based on research in partnership with Daniel Masterson of Yale University and Christian Lehmann of the University of Brasilia, focused on the Syria crisis, specifically looking at Lebanon.

The results are consistent with what we know about the impact of money distribution in more stable, albeit poor, settings: it does not create disincentives to work, it enables people to study and spend money on things that improve their lives; and it doesn’t cause them to go out and squander it on drugs, alcohol or gambling.

…We found that households receiving cash assistance were half as likely to send their children out to work. Cash also increased access to education, and there is evidence of reduced tensions within the household and between the refugee and host community.

Op-ed and full report.

The authors (who are close colleagues) are presenting in London tomorrow (Oct 2). Info here. RSVP here.

A scientific journal that raises the level of maturity in referee reports

Frontiers in Neuroscience for Young Minds is a scientific journal that includes young people (from 8 to 15) in the review of articles. This has the double benefit of bringing kids into the world of scientific research – many of them for the first time – and offering active scientists a platform for reaching out to the broadest of all publics.

The journal is Frontiers for Young Minds.

Who runs the UN?

A new paper from Paul Novosad and Eric Werker:

We examine, over a 60 year period, the nationalities of the most senior positions in the United Nations Secretariat, ostensibly the world’s most representative international institution.

…The most overrepresented countries are small, rich democracies like the Nordic countries. Statistically, democracy, investment in diplomacy, and economic/military power are predictors of senior positions–even after controlling for the U.N. staffing mandate of competence and integrity.

National control over the United Nations is remarkably sticky; however the in influence of the United States has diminished as US ideology has shifted away from its early allies. In spite of the decline in US influence, the Secretariat remains pro-American relative to the world at large.

A dictator’s handbook

At the end of August, the prime minister of Lesotho fled to neighboring South Africa, saying he had been overthrown by the military. He subsequently returned to the capital with the assistance of South African police, and although the situation remains murky, he appears to have survived a failed coup attempt. In June, when the coalition government began to break down, the Monkey Cage reviewed the situation in Lesotho and expected military intervention to be unlikely, primarily because popular support for democracy was high and popular support for military rule was extremely low. It is useful to understand why this expectation is wrong-headed.

That is Naunihal Singh on what we misunderstand about coups. He has literally written the book on them.

What’s the effect of TOMS shoes on local development?

A new paper from Wydick, Katz and Janet (link now fixed):

We carry out a cluster randomized trial among 979 households in rural El Salvador to test whether shoe donations exhibit negative impacts on local shoe markets. Households in half of our communities were given a pair of children’s shoes at baseline (treatment communities), while all households were given coupons that could be used for shoe purchases at a local shoe store.

Although point estimates on coupon redemption and difference-in-difference estimations indicate shoe purchases to be slightly lower among households receiving the donated shoes, we find no statistically significant difference in market shoe purchases between treatment and control households.

Fakebook friends

A Dutch woman faked a five week trip to Southeast Asia through facebook to make a philosophical point.

She posted pictures of herself sitting next to a Buddhist monk in a temple – it was just that the temple and monk happened to be in Amsterdam.

…Zilla even redecorated her own bedroom to make it look like an Oriental hotel room so that she could have Skype conversations with her family – at random times in the night, of course – without raising suspicion.

…The reasons behind her actions, however, are noble: it was all part of a university project, in which she wanted to show how Facebook activity is not necessarily reflective of real life.

…“My goal was to prove how common and easy it is to distort reality. Everybody knows that pictures of models are manipulated. But we often overlook the fact that we manipulate reality also in our own lives.”

I like this person.

I’m also now wondering which of the blogs I follow are actually Dutch girls in disguise. I have my suspicions.

Corruption and dirty elections are a symptom not the disease

It’s easy to despair at the patronage, vote-buying, and authoritarian turn being taken in so many new democracies. Surely democracy was different in the West? I give you Spain:

The stage of convulsive liberalism in Spain was long, from 1810 to 1874, compounded by riots, military revolt, civil war, and severe regional discord.

…The nineteenth-century Spanish political intelligentsia and elites persistently pushed through sweeping constitutional reforms, for brief periods giving Spain the most democratic suffrages and the most liberal political structures in continental Europe… No other polity attempted such advanced political structures on the basis of such limited education, so little civic training, such an unproductive economy, such poor communications, such extreme regional dissociation, and so much institutionalized opposition in sectors of the Church and Carlism. Stability was eventually achieved by a more modest, restricted form of liberalism in the oligarchic system of the restored monarchy of 1874.

…universal male suffrage was reintroduced in 1890, but its effects were at first spurious if for no other reasons than the illiteracy, lack of civic interest, and poor communications among the lower classes. The existing patronage and party-boss system, commonly known as caciquismo, largely contained or deflected popular voting for about thirty years.

That’s from The Franco Regime by Stanley Payne, which is long but excellent.

Most Western democracies doled out the right to vote very slowly. Maybe more correct: middle classes seized the vote where and when they could. Not Spain. The case of Spain looks more like new democracies today, where everyone suddenly (and sometimes unexpectedly) gets the the right to vote.

The tricky part with this: the people who choose the leaders (the masses) aren’t the ones who control the wealth or weapons or other institutions, and they might have little education or civic organization. So the people who are powerful remain powerful, but now have to work through the quasi-democratic system and the masses, who now have a little more power than before. Thus you get patronage and party boss systems, or the rolling back of rights from the least powerful. At least for a time.

Some people use this as an argument for autocracy. I don’t. I’d rather see rights with corruption than no rights at all. That right to sell your vote is still a power most people didn’t hold before, and it helps them. For me, it’s a reminder to have patience. Also, it’s an argument for for tamping down the Western anti-corruption fetish, and for not setting a standard for new democracies that we never set for ourselves.

Is popular support for same sex marriage a charade?


Public opinion polls consistently show that a growing majority of Americans support same-sex marriage. Critics, however, raise the possibility that these polls are plagued by social desirability bias, and thereby may overstate public support for gay and lesbian rights. We test this proposition using a list experiment embedded in the 2013 Cooperative Congressional Election Study. List experiments afford respondents an anonymity that allows them to provide more truthful answer potentially sensitive survey items.

Our experiment finds no evidence that social desirability is affecting overall survey results. Indeed, our efforts provide new evidence that a national opinion majority favors same-sex marriage.

A new paper by Lax, Phillips and Stollwerk.

Amazon is not profitable, for now, because taking over the world costs a lot of money

I’d wondered whether Amazon had no profits because their business was simply expensive and unsustainable at current prices. It seems the answer is “not necessarily”.

Amazon’s business is delivering very rapid revenue growth but not accumulating any surplus cash or profits, because every penny of cash is being ploughed back into expanding the business further. But, this is not because any given business runs permanently at a loss – it is because the profits from what is already there are spent on making new businesses.

…Amazon has perhaps 1% of the US retail market by value. Should it stop entering new categories and markets and instead take profit, and by extension leave those segments and markets for other companies? Or should it keep investing to sweep them into the platform? Jeff Bezos’s view is pretty clear: keep investing, because to take profit out of the business would be to waste the opportunity. He seems very happy to keep seizing new opportunities, creating new businesses, and using every last penny to do it.

…When you buy Amazon stock (the main currency with which Amazon employees are paid, incidentally), you are buying a bet that he can convert a huge portion of all commerce to flow through the Amazon machine. The question to ask isn’t whether Amazon is some profitless ponzi scheme, but whether you believe Bezos can capture the future. That, and how long are you willing to wait?

The full post is interesting. Hat tip to Tyler Cowen.

What The Economist should have read before suggesting that US slavery wasn’t always so bad

First, remind me, when I’m writing my first book, to try to get The Economist to write a racially insensitive review. I’m pretty sure Edward Baptist’s sales are pretty terrific right now.

The Economist has withdrawn the offending book review and apologized (the book in question, and the article and apology). Here’s the uncontroversial bit:

Mr Baptist, an historian at Cornell University, is not being especially contentious when he says that America owed much of its early growth to the foreign exchange, cheaper raw materials and expanding markets provided by a slave-produced commodity. But he overstates his case when he dismisses “the traditional explanations” for America’s success: its individualistic culture, Puritanism, the lure of open land and high wages, Yankee ingenuity and government policies.

Nothing in history (least of all the growth of the largest economy humankind has ever known) has a single explanation. Academics like to overstate their case and need to be reined in a little.

Even so, here’s the jawdropping finale:

…Slave owners surely had a vested interest in keeping their “hands” ever fitter and stronger to pick more cotton. Some of the rise in productivity could have come from better treatment. Unlike Mr Thomas, Mr Baptist has not written an objective history of slavery. Almost all the blacks in his book are victims, almost all the whites villains. This is not history; it is advocacy.

What could have shed light on this, had The Economist writer bothered to read the literature (and had the academics bothered to write in comprehensible prose)?

First, when do employers use coercion and how well does it work? There’s a pretty new and exciting literature here:

  • Violence and pain work better in labor markets where people have really poor options, and are easily controlled, like children or the least educated. You see this in child labor during British industrialization, or even in child soldiering in Uganda (my own work). Here’s a graph of how long someone stayed with a rebel army in Uganda based on his age of conscription. The paper argues that ones you can scare and indoctrinate the easiest (in this case, kids) stay longest:

Screen Shot 2014-09-05 at 10.07.24 AM

  • Adults will tend to escape if you use violence, so slavery and serfdom work best when the overlords control the legal system or can hunt you down. You see this with servants in 19th century Britain or with European feudalism and US slavery
  • When you make it harder for employers to use force, wages go up. You see this in 19th century Puerto Rico coffee growing, or in the Emirates today
  • It’s not unusual to see a mix of rewards and coercion. For instance, in the child soldiering paper, rewards are more likely for the people who can run away, and they’re also useful (with violence) if you’re trying to indoctrinate and brainwash.
  • And when you turn the entire system against them, yes, whipped people work harder. Here’s an unpublished graph from Suresh Naidu from one US plantation and the correlation between the number of whippings a slave received and her productivity at cotton picking:whipping

So a moral of the story is that yes, rewards can be a substitute for violence, but in a coercive labor market, better pay or food is just service to your larger evil plan to enslave more people more profitably.

Then, on the longer term consequences of slavery (again, hat tip to Suresh, who breathes this stuff):


Is anyone else feeling depressed and hopeless?

More suggestions welcome.

More subways

Public transit accounts for only 1% of U.S. passenger miles traveled but nevertheless attracts strong public support. Using a simple choice model, we predict that transit riders are likely to be individuals who commute along routes with the most severe roadway delays. These individuals’ choices thus have very high marginal impacts on congestion.

We test this prediction with data from a sudden strike in 2003 by Los Angeles transit workers. Estimating a regression discontinuity design, we find that average highway delay increases 47% when transit service ceases.

This effect is consistent with our model’s predictions and many times larger than earlier estimates, which have generally concluded that public transit provides minimal congestion relief. We find that the net benefits of transit systems appear to be much larger than previously believed.

A new paper by Michael Anderson.

I’m curious about the politics and economics of actually increasing subways. It seems to me that affordable subways were more feasible when fewer people could vote and the economic value of the land was lower. So many people stand to lose in the short term from a new subway line (say, because your store loses traffic or you live with noise) and wages and other costs so high, that I’m skeptical a city like New York could ever expand the network.

Then again, I’d be happy with inframarginal changes, like stations that are less than 90 degrees hot and do not smell like urine.

And you thought the news about Twitter becoming more like Facebook was bad. Look what Apple is doing:

Soon your Twitter feed may be curated by some kind of prediction algorithm. Even worse, from Apple’s website, the new autocorrect:

“As you type, you’ll see choices of words or phrases you’d probably type next, based on your past conversations and writing style. iOS 8 takes into account the casual style you might use in Messages and the more formal language you probably use in Mail. It also adjusts based on the person you’re communicating with, because your choice of words is likely more laid back with your spouse than with your boss.”

Full story.

Besides the specter of bad and mediocre writing perpetuated endlessly, I wonder how long before bots are predicting passable term papers. I guess if it lets me predict grades automatically it wouldn’t be all bad.

At the same time, in my experience the prediction algorithms are pretty poor.

  • Exhibit A: Facebook feeds
  • Exhibit B: Netflix recommendations. I most certainly do not want to watch National Lampoon Vacation VII
  • Exhibit C: My iPhone occasionally suggests “thou” instead of “you”

So I do not fear the replacement of writer by machine anytime soon.