How to Account for Alternatives When Comparing Effects
Updated: Apr 14, 2021
This is a collaborated project with Dana Burde and Cyrus Samii at New York University, and Joel Middleton at UC Berkeley.
Interventions that offer new programs to individuals, such as new forms of schooling, typically vary in their effectiveness across contexts. Variation in effectiveness may be because contexts vary in terms of the ways that individuals use already-existing alternatives and how the intervention changes such use patterns. We show how to use a “principal strata” approach to account for such variation in use patterns when comparing effects across contexts. We use these methods to examine 2008 and 2015 replications of a schooling experiment aiming to increase access to education for girls in rural Afghanistan. Our analysis helps us to understand why average effectiveness was substantially lower in 2015 as compared to 2008. Our findings suggest that even though the share of girls for whom the intervention would be especially valuable dropped considerably in 2015 as compared to 2008, the efficaciousness of the intervention may have increased, meaning that it would still make sense to sustain the intervention.