Module 1: Why Evaluate?

Overview

This module introduces the concept of impact evaluation and discusses the ways that impact evaluation is different from other monitoring and evaluation activities. The video lecture provides an introduction to the use of randomized trials in the evaluation of anti-poverty interventions. The empirical exercise is a gentle Stata warm-up.

This module contains one reading, a video (guest) lecture (approximately 15 minutes), slides for in-class discussion, and an empirical exercise.

Readings

Impact Evaluation in Practice: Chapters 1 and 2

Review Questions

1. What were the objectives of Mexico's Progresa program?
2. How did Progresa impact poor families?
3. How is impact evaluation different from other monitoring and evaluation activities?
4. What is the difference between cost-benefit analysis and cost-effectiveness analysis?
5. What are prospective evaluations, and how do they differ from retrospective evaluations?
6. According to the authors, why are prospective evaluations more likely to produce credible estimates of program impacts?
7. What is the difference between an efficacy study and an effectiveness study?
8. What were the inputs and outputs in the Piso Firme project in Mexico? What was the theory of change?
9. What are the steps in a results chain?
10. What does it mean for an outcome variable to be SMART?

Video Lectures

Esther Duflo: Social Experiments to Fight Poverty (TED talk)

Review Questions

1. Why can't we know whether aid given to Africa has reduced poverty?
2. How did Professor Duflo and her collaborators attempt to increase vaccination rates in India? What two policies did they implement, and why did they choose these policies? What did they find?
3. How did the demand for anti-malarial bednets change as a function of price in Kenya? Were people who paid more for bednets more likely to use them?
4. Which policies had the largest impact on educational attainment per dollar spent? Why do you think these policies had such a large impact?

Slides for Class Discussion

There is no video lecture for Module 1. I use the slides below for class discussion.

pdf   tex   materials

Empirical Exercise

This exercise makes use of the data set E1-CohenEtAl-data.dta, a subset of the data used in the paper Price Subsidies, Diagnostic Tests, and Targeting of Malaria Treatment: Evidence from a Randomized Controlled Trial by Jessica Cohen, Pascaline Dupas, and Simone Schaner, published in the American Economic Review in 2015. The authors examine behavioral responses to various discounts (“subsidies”) for malaria treatment, called “artemisinin combination therapy” or “ACT.” The J-PAL summary of the experiment and the findings is here.

The aim of this empirical exercise is to review key Stata commands. You can download the activity as a do file or a pdf. There is a web page for the exercise here.

Further Reading

The Poverty Lab (A New Yorker profile of Esther Duflo, from 2010)

A Nobel Prize for the Randomistas (a Center for Global Development blog post by me)

Interview: MIT Economists Esther Duflo and Abhijit Banerjee

Experimental Conversations by Timothy Ogden

Additional Resources

Georgetown University’s EdX course on Impact Evaluation with Applications in Low and Middle Income Countries

Professor Andrew Heiss’ Program Evaluation for Public Service