The trials of randomization

One of my busiest projects at the moment is working on behavior change in Liberian street youth, aiming to reduce poverty, crime and violence.

One of the interventions dispenses $200 cash grants to half the cohort. “We don’t want to get $200 by lottery,” they say, “we’d like $100 each.” I sympathize. They are pleased to be in the program but being a research subject is not all smiles.

We dispensed cash to our pilot group just a few weeks ago. As the lottery was won or lost, and the cash handed out or not, our assistants noticed money changing hands. “Risk-sharing groups” was the answer. Good grief. At least they didn’t totally unwind the experiment. Winners kept the bulk while sharing smaller amounts with losers — $50 or maybe just $25. An interesting outcome in itself.

The researchee defeats the researcher. I wonder if they realized that they could completely insure one another and get the outcome they want. Would I be that surprised if one managed to track down my blog and saw the idea as it is? I’m now not so sure…

16 thoughts on “The trials of randomization

  1. If you designed the trial again would you do cluster randomization instead? Why or why not? Seems like an “easy” solution to such leakage me (in hindsight of course!) though it’d be a bit harder to execute and you’d need a bit bigger sample — curious to hear what other limitations you see in this particular setting.

  2. It’s almost as if they see themselves as human beings with agency and not research subjects. Weird! Someone should do a study on this phenomenon.

  3. That’s kind of the point of the post, no?

    (I hate responding to snark, especially anonymous snark, and most especially anonymous snark that doesn’t really get the post, but sometimes I can’t help myself.)

  4. I’m fascinated by what we learn when delivering interventions (in addition to the effectiveness evidence, of course). I’d love to know if the researchees would share their cash in similar ways if it was their hard earned cash – or is there something in the fact that they had ‘got lucky’ that put them in the mood to share. How do they share out their good fortune? Who with? Would they hold on to it if they needed it more? So many questions…
    Are you writing up a process evaluation along side the trial? Would love to hear more.

  5. I was thinking of clusters that are separated geographically (or some other way so that the intervention group don’t know individuals in the control group) so that everyone i one area / one group of friends is getting the money, and no one is getting it in the other group. Or are you saying you did something like that but that people are still giving their friends – who aren’t in the study – the money? In one case you’d have leakage between intervention and control people, which seems more problematic than if you’re getting leakage from people in the study to those outside of it (ie, they’re just choosing to use the money by giving it to friends).

  6. It certainly is a nice finding by itself! Maybe field experiments are sometimes not so much different from lab experiments.

  7. Or actually it appears as if participants have a preference for an experiment the researcher did not plan to set up. You might have a nice proxy for some of the social preferences at least.

  8. Hi Chris,

    Thanks for sharing. We have a similar intervention in TZ, where vulnerable folks got some grants to run small enterprises from the Social Action Funds and we have topped this up with individual grants that vary in size. We thought about the sharing in advance (or the possibility that people with cash would have pressures to help others) and considered giving the transfers privately, but there was no way to do this in villages in TZ. We think of this sharing as a positive thing (and money into risk-sharing groups may well be), but if you need to give to friends, family, and kin often, your likelihood of putting the funds into productive investments may be limited. In such environments, getting people out of poverty traps (if they exist) may require different approaches than getting them over some hump individually: the whole network/community may need to get over the hump…

    Also, we were wondering how much of this behavior you observed has to do with (a) these are young people, and (b) former or current street youths. The solidarity within this group may be stronger than others, no?

    And, totally agree with your sentiments on anonymous snark. We protect anonimity on the blogs because it is needed under certain circumstances, but not for being snarky…

  9. even small amounts of money in resource-poor environments get distributed far and wide. there is tremendous social pressure to share the wealth. it’s a huge barrier to capital accumulation.

    add to that the fact that in liberia, patronage pyramids are ubiquitous as a form of social structure. everyone aspires to belong to one and/or to lead one. that’s why everyone asks everyone else for money, all the time.

  10. Were your keen-eyed researchers able to track the people that shared and how much each gave/got? If so, as I am sure you are aware, you can do a “dosage effect” experiment to determine how much cash is necessary for the intended outcome. The project may be a bit more complicated, I’m sure, but this seems like a great opportunity!

  11. I think we expected sharing. I think we didn’t expect explicit risk-sharing groups. In retrospect, it’s obvious that subjects can completely undo the randomization of any experiment if treatment status is known and treatment is divisible, as it is with many economic interventions.

  12. Thanks for sharing this, it´s really interesting.

    Do you think that risk-sharing can be thought of as a form of investment in these cases? They never know when/whether they will get some cash, and by risk sharing this time, they ensure that the next time they are out of luck?

    In terms of the questions you ask regarding preferences for investment and savings, it is may be that given the environmnet, they invest where they might get a higher pay-off…
    I know nothing about the specifics, so sorry if this is way out of the question and the experiment, but, can it be that it is not so only preferences for investment and saving that is affecting their decision but also their expectations on the return?
    In the sense that investing in buying a long term good which may be stole or damaged, or getting education when there are no job opportunities or you might get killed soon, may be as not that good investment for them…