John List, Sally Sadoff, and Mathis Wagner have a new working paper with simple rules of thumb for optimal experiment design:
A casual perusal of the literature presents a striking consistency concerning sample sizes and their arrangement: most studies uniformly distribute at least 30 subjects into each cell. This approach holds whether the analyst is making use of a purely dichotomous treatment (i.e., pill or no pill) as well as when the analyst is exploring levels of treatment (i.e., various dosage levels).
Discussion of whether such a sample arrangement is efficient is more mature in other literatures, but has not been properly vetted in the experimental economics community. Our paper attempts to fill this gap.
An older ungated version is here.
One Response
This is a great paper–both useful and surprisingly readable for a technical guide. Budding experimental economists interested in microfinance might also find this Financial Access Initiative Brief on randomized experiments helpful: http://financialaccess.org/sites/default/files/Randomized_Experiments_MF_JB_AD_MFInsights_FINALpdf.pdf