Selection Bias and the Experimental Ideal

Overview

This module introduces the potential outcomes framework that we’ll use throughout the course as well as the concept of selection bias before explaining how average treatment effects can be estimated when treatment status is randomly assigned.

This module includes one reading, four video lectures (approximately 75 minutes total), and an empirical exercise.

Readings

Mastering Metrics: Chapter 1

Review Questions

How do Americans with health insurance differ from those without health insurance? Are those differences likely to represent the causal effects of having health insurance? Why or why not?
What are potential outcomes? How do potential outcomes lead to a missing data problem in causal inference?
What is selection bias, and what implications does it have for program evaluation?
What is the Law of Large Numbers, and why is it important in randomized experiments?
Based on the evidence presented in the reading, what are the impacts of access to health insurance (in the United States)?

Video Lectures

Lecture 2.1 The Potential Outcomes Framework (26:44)

Review Questions

What is the fundamental problem of causal inference?
What are potential outcomes, and how do they create a missing data problem in program evaluation?
What is selection bias? When and why is it likely to bias estimates of program impacts?

Lecture 2.2 The Experimental Ideal (13:17)

Review Questions

How can random assignment eliminate selection bias?
What is the Law of Large Numbers, and why is it important in randomized experiments?
How can we estimate the causal impacts of a program when treatment assignments are randomized?

Lecture 2.3 A Short History of Randomized Experiments (27:35)

Review Questions

Who was Ronald Fisher, and how did he contribute to the development of randomized experiments?
When were randomized trials first used in medicine? When were they first used in the social sciences?
What were the first randomized evaluations used in the international development context?

Lecture 2.4 Analysis of Randomized Experiments (8:32)

Review Questions

When treatment is randomly assigned, how can we test the null hypothesis thta the average treatment effect is equal to zero?
How can linear regression be used to analyze data from randomized experiments?

Lecture Slides

A handout version of the lecture slides is available here.

If you wish to use these slides for teaching, the underlying beamer files and supporting materials are here.

Papers Mentioned in the Lectures

The Design of Experiments, Chapter 2

The Entry of Randomized Assignment into the Social Sciences by Julian Jamison

Do Conditional Cash Transfers Improve Child Health? Evidence from PROGRESA’s Control Randomized Experiment by Paul Gertler

Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities by Edward Miguel and Michael Kremer

Empirical Exercise

In this exercise, we’ll use Stata’s rnormal() command to generate draws from a normally-distributed random variable. This approach - simulating data according to a known data-generating process - is an incredibly useful tool in empirical microeconomics (both for checking your econometric intuitions and your anlayis code).

We’ll use “locals” (also know as “local macros”) to easily change the number of observations and other parameters of our data set. This will allow us to explore the way the properties of randomly-assigned treatment groups in larger and smaller samples.

This exercise introduces a range of practical coding tools: rnormal(), locals, and the return list and display commands. By varying the sample size, we’ll build a better understanding of the role that the Law of large Numbers plays in randomized evaluations.

You can download the activity as a do file or a pdf.

ECON 379