11 Text as Data

Readings

Text Mining with R: 1 and 3

Who Invented Instrumental Variable Regression? by James Stock and Francesco Trebbi

Text as Data the book is also great, but beyond the scope of this course.

Lecture

Slides from Lecture 11

Lab

The objective of this lab is to adapt the template (available in R or Python) to cluster NBER working papers from two programs based on the abstracts. The template does this for Development Economics and Labor Studies. You must choose two different NBER programs. You should identify a (relatively) stable clustering that groups papers in a sensible way. You should extend the template as needed to identify the best possible clustering, including at least some of the following:

• Increasing the number of clusters
• Including additional preprocessing or stemming to group related words or exclude context specific stopwords
• Excluding any words that only appear in a single working paper
• Including (selected?) bigrams in addition to individual words
• Using TF-IDF instead of term frequencies
• Clustering by principal components rather than term frequencies themselves