Measuring measurement error is under-appreciated and hence underdone. Fortunately there are exceptions.
Beegle, De Weerdt, Friedman, and Gibson on errors in household consumption (our main measure of poverty):
A field experiment in Tanzania tests eight alternative methods to measure household consumption on a sample of 4,000 households. There are significant differences between consumption reported by the benchmark personal diary and other diary and recall formats.
Under-reporting is particularly relevant in illiterate households and for urban respondents completing household diaries; recall modules measure lower consumption than a personal diary, with larger gaps among poorer households and households with more adult members.
And Beegle, Carletto and Himelein with unexpected findings on recall bias:
Due to survey logistics, agricultural data are usually collected by asking respondents to recall the details of events occurring during past agricultural seasons that took place a number of months prior to the interview. This gap can lead to recall bias in reported data on agricultural activities.
To test for such recall bias, the length of time between harvest and interview is examined for three African countries with respect to several common agricultural input and harvest measures. The analysis shows little evidence of recall bias impacting data quality. There is some indication that more salient events are less subject to recall decay.
4 Responses
“more salient events are less subject to recall decay”
Seriously?
The Michael Boozer and Taveneet Suri paper on measuring consumption is also very interesting. It is referenced here, but is not public yet http://blogs.worldbank.org/impactevaluations/love-and-secrets
Basic story: spouses have “private” information and asking one spouse about another’s private information will not yield accurate estimates. So asking the female head of household about household alcohol really doesn’t give you accurate information, if the male is the primary purchaser. My take-away is that if you are building an index of HH consumption, you might want to leave out things like alcohol, or analyze them separately.
And if you are interested in micro-enterprises, these are rather useful “Testing alternative measures of microenterprise profits and net worth” (http://onlinelibrary.wiley.com/doi/10.1002/jid.781/abstract) , and this one too “Measuring microenterprise profits: Must we ask how the sausage is made?” (http://www.sciencedirect.com/science/article/pii/S0304387808000138)
Relatedly, see the new paper in JDE on “Do household definitions matter in survey design? Results from a randomized survey experiment in Mali“.