So excited, yet so nervous…

Anyone who has ever run a survey will know the nightmare that is data entry. How can double entry data entry still yield crap results? Oh, let me count the ways.

The last nightmare (in Uganda) was one too many, and I vowed: never again. It was also time to end the medieval and excessive use of paper.

Tomorrow, with 6,750 surveys about to launch in Liberia, I bring you my savior:

What, you ask? Is that a 1990s era Palm Pilot? Why yes it is! Low battery consumption. Easy to use. And best of all: no Internet connectivity for my enumerators to waste time on. (What could be more boring than a calendar application?)

Each survey has about 300 questions programmed in, using a little program called Pendragon. Data entry nightmare: you have been replaced. Thy name is Pendragon. I love my little units. I love the paperless, real time entry. I LOVE getting data downloads the day they are collected. But this little program is keeping me up nights.

Correction: technically, it’s keeping my intrepid RA, Bryan, up nights. But I feel his pain.

What keeps me up is the 0.005% chance we missed something. Forgot a little program code. Planned one too few backup and safety procedure. Paper surveys are, well, paper. You can hold them. Re-enter them. Cross your fingers for my survey integrity…

19 thoughts on “So excited, yet so nervous…

  1. this is smart, smart, smart. i’d love to see a post about enumerator training and IRR with pendragon.

  2. Really interesting, especially in light of me just sitting here cursing at my own medievalness whilst doing my data entry from a survey in TZ. Would have loved to have a non-paper solution.

    That said, I was wondering one thing though: How will you react if you – or more importantly your enumerators – realise that some particular questions are phrased/spelled out in the wrong way or that respondents can’t answer them in the way you predicted it and use another ‘answer format’? Can you adjust in such a (virtually inevitable) occasion?

  3. From my Experience, this is going to be a pain.
    With J-PAL we’re trying these things, but you still get illogical data entries. (hm, your kids are 77 years old ? They look like 11 years old to me).

  4. @amelia: Enumerator training was a breeze. they stayed up ALL NIGHT they were so excited. I have no idea what IRR is.

    @Eddie: One drawback is the difficulty of non-standard responses. This is one sacrifice we make. In general, these are exceptions and they don’t arise in the major variables of interest.

    @OMW: Caught red-handed. My African development class writes a book review of Sachs as a mid-term assignment.

    @ Anonymous: On illogical data entries, I think we see just as many in paper surveys. Maybe just of a different kind. So far, on the practice data, we see excellent results. Also, I am planning to have an RA analyze data in real time to check for persistent errors and low enumerator quality.

  5. I remember some disasters with the survey software (can’t remember its name). Occasionally there was an error and the software would delete the entire survey form.

    More importantly, respondents in rural Kenya found these things really spooky: how did their name be in that little box. Explaining that it came from a sampling frame did not convince most.

    That said if it works it would be great – keep us posted! Seb

  6. Addendum to the earlier comment: if some people are scared to dictate into your handhelds you might have a serious sample selection issue. I’d be curious about your experiences with this, and effects on response rates.

  7. actually that’s a very good point! Even if the respondents aren’t directly spooked, I would certainly expect them to be totally distracted by the thing, much more concentrating on it than the actual questions. In such a situation it might become very difficult to finish a questionnaire with 300 (!) questions in a satisfactory way without haste and rushing through questions/answers.

  8. Good Luck. I once witnessed those bad boys delay a 10 day survey in Kakamega by 7 days. My advice: pretest like hell, and if you have the manpower, send an extra enumerator to collect paper data for the first few days to test accuracy and error rates/usage problems across enumerators.

  9. Oh, and in case you’re not already doing this, I also recommend sending your enumerators with paper versions of the survey just in case the electronic version fails unexpectedly in the field.

  10. IRR is inter-rater reliability. usually more appropriate to the coding a narrative, but in this case i imagine that navigating the software could lead to some systematic data entry quirks. if i had had my head on straight, i’d have put it as katie did: was there “backup” enumeration? enumeration testing? how often did enumerators agree with one another?

  11. Having been involved in fieldwork in rural South Africa, I feel your pain. Good luck!

  12. Really interesting! In fact I am currently looking for such product to conduct a household survey in Mexico. i realized it is a relatively old post, but I haven’t read any feedbacks on how it went once in the field (or maybe I missed it)… How did it go, Chris? Any problems with the material once in the field? Any suggestions of what would be the best material and software? I would really need to have more first-hand testimonies of do’s and don’t before buying such costly material. thank you in advance for taking time to reply.

  13. Pingback: weighing in on using electronic devices to collect survey data « haba na haba