More than half the world lives in cities, and a lot of those cities (especially those in the Americas) are plagued with homicides and crime.

Americans often think this violence is an individual problem: greed, passions, feuds, and hot reactive thinking drive killers. That’s true to an extent. But this view overlooks something important: that, in an awful lot of cities, crime is also organized and group-based. That makes some killings calculated and strategic, not simply crimes of passion or reflex.

Criminal groups are most organized in Latin America. That’s also true in some American cities and (most of all) in American prisons. When crooks get together, governments often face a terrible trade-off: You can get peace or you can get weak gangs; but you can’t easily get both.

The dilemma is simple: any policy that reduces violence risks making organized criminal groups more influential, in hard-to-observe ways. That’s the message of a short piece I wrote with Ben Lessing and Santiago Tobon, part of a broader set of policy papers on how to deal with serious and organized crime

In a city like Chicago, for example, you have disorganized killings and crimes of passion in part because the government spent 20 years cutting off the head of every gang and criminal group that tried to organize the streets. That’s made for weak, fractured organizations, more competitive drug markets, and a slew of blood feuds and small turf battles. It’s so disorganized that the old gang leaders I know don’t even like to call today’s groups “gangs”. They call them crews, mobs, and cliques—small and disorganized factions who don’t listen to anyone.

Now, it’s not like Chicago was a peaceful place back when supergangs like the Vice Lords or the Gangster Disciples lorded over the smaller gangs in the city. But if you step back and look across the Americas, there’s a tendency of these “supergangs” to at least try to keep the peace. Many succeed.

You see, organized crime is often hierarchical. In Chicago and many American cities you had supergangs that coordinated loose confederations of street gangs. In Brazil you have prison gangs like the PCC who govern smaller criminal groups. In Medellin, Colombia, where I’ve been working, you have a set of shadowy bands that are sometimes called razones—spanish for “reasons”.

If there’s one thing these higher-level criminal structures hate, it’s gang war. No one sells as many drugs. Soldiers get killed. And governments and police start to come after the big bosses. Fighting eliminates the invisibility that was their shield.

Mayors and police chiefs understand this. In fact, when these criminal governors exist, governments try to coordinate peace in every way possible. Not always officially. Sometimes it’s a set of ultimatums, or an informal recognition. Other times it’s quite explicit.

How states make peace with gangs

US cities are fond of focused deterrence strategies. City authorities meet with criminal leaders and pledge swift, certain, and severe punishments for future homicides. Comparing cities that do this to those that don’t suggests the threats have some effect.

Violence interrupters also try to mediate between rivals. Usually they leap between feuding individuals, but some also try to mediate between leaders and arrange pasts. Comparing neighborhoods suggests this too works reasonably well.

In Brazil and Colombia, the state inadvertently helps criminals keep the peace by putting leaders in the same cell blocks. This can help criminal heads strike truces, work out differences, and even fuse their organizations.

In Medellin—where I do a lot of my work with Ben, Santiago, and Gustavo Duncan—the syndicates have made a deal they call El Pacto del Fusil: The pact of the machine gun. And when that pact starts to fray, the state helps them keep it. Last year, when two factions started to skirmish, and killings spiked, everyone feared a new gang war. Suddenly, every criminal boss in prison found themselves transferred on the same day. They were scattered across the country, but for a few days were place in the same holding area. By sheer coincidence, a trusted criminal negotiator was arrested and placed in the same cellblock. A few days later, killings plummeted to their normal, modest levels. Medellin, despite being filled with more than 400 armed gangs, has a homicide rate that’s sometimes a third of that of Chicago.

Finally, a few governments directly negotiating pacts and ceasefires. This dealmaking is often secret and unpopular. El Salvador’s president Bukele is thought to have arranged a truce for a time, as have his predecessors. Mexico’s single party state once did the same with cartels. Why wage an expensive fight when you can compromise and share the spoils? Democracy makes this difficult, however. It’s too unpopular, and it’s harder to make deals with rotating politicians. Which might be why elected governments are more likely to fight forever wars against the cartels.

All this makes a certain sense. In my book Why We Fight, I talk about nations, political factions, and even gangs wage to fight. It’s ruinous. Better to find a deal that leaves everyone with their share of the pie. The trouble is that, when it coms to organized crime, peace has some drawbacks.

The darker side of peace

The trouble is that all these strategies create an invisible but terrible trade-off: helping gangs make peace makes them more profitable and stronger. Especially the criminal syndicates who provide this valuable service to the smaller gangs and groups.

People don’s buy as many drugs or pay extortion in the middle of a gang war. So peace is an opportunity to extract protection money and gain legitimacy from local people.

Also, the same syndicates that help keep peace can also coordinate to form cartels—in the economic sense of the term—coordinating everyone to sell as monopolistic prices. Certainly this is what’s happened in Medellín.

Finally, mayors face a tough choice. Voters hate homicides. Why disturb any Pacto del Fusil through crackdowns on the  gangs? They’d just find their cities engulfed in turf wars so bloody and destructive that they undermine their election prospects next term.

The trade-off is often invisible

Every city has to make their own choice. Chicago decided to crackdown, as have successive Mexican governments. Violence is part fo the price. Medellin and, at times, various Brazilian cities coordinate a quiet peace.

The thing  that worries me—how many policymakers are even aware of this trade-off? How many crackdowns actually understand these dynamics? How many voters realize that fewer shootings might mean stringer gangs? My own experience is “not many”.

The key problem is that voters and governments are unaware of the terrible trade-off. Most police and city governments have detailed information on one thing: crime and homicides. Officials get daily reports, and often have fine-grained data on crimes down to the level of street corners. Mayors can hold their city staff or the police accountable for improving these measures every year. If they don’t, newspapers report them every week, and voters pay attention.

But they have no way of knowing whether criminal groups are growing stronger or weaker, or more or less extortionate. Nobody collects data on gang strength or drug profits. No one tracks when gangs provide local security, or ask citizens whether they find the gangs as legitimate as the state.

As a result, too many make the terrible trade-off ignorant of the consequences.

Over the past few years we’ve been showing how these data are possible. We run annual surveys on gang strength extortion, and other measures. Here, for example, is a heat map for Medellin, showing who (on average) is more likely to help you resolver a dispute with your neighbor, address a theft, or protect your street: the state (blue) or the local gang (red).

Here’s how gang (combo) loyalty and legitimacy correlates with this criminal governance:

In many neighborhoods, the gang has about as much trust and respectability as the state. This is a high orice to pay for lower violence. It might be the right price to pay, but few mayors are even aware of it.

Want more?

Here’s the policy note for more detail.

Here’s a guide we wrote for thinking about how to build these data systems in other countries. We’re interesting in expanding this across Latin America, so reach out to me and Innovations for Poverty Action if you’re a think tank or police force or government looking to do this in your city.

Here’s one paper on what we did with the data, to show how increased state rule can actually lead gangs to competitively increase the services they provide to civilians.

And you can read about why gang and other wars happen (and why they often don’t) in my new book, Why We Fight.

For more policy notes on this topic, check out the University of Birmingham’s Serious Organised Crime & Anti-Corruption Evidence (SOC ACE) Research Program, funded by the UK’s FCDO.