Threats to validity

Content for Thursday, February 16, 2023

Readings

• Guido Imbens, “Statistical Significance, p-Values, and the Reporting of Uncertainty” (Imbens 2021). This is available on iCollege, and it’s also free here.
• Randall Munroe, “Significant”
• Alexander Coppock, “10 Things to Know About Statistical Power”
• Play around with FiveThirtyEight, “Hack Your Way To Scientific Glory”
• Chapter 9 in Impact Evaluation in Practice (Gertler et al. 2016)
• Chapter 11 in The Effect (Huntington-Klein 2021)

Slides

The slides for today’s lesson are available online as an HTML file. Use the buttons below to open the slides either as an interactive website or as a static PDF (for printing or storing for later). You can also click in the slides below and navigate through them with your left and right arrow keys.

View all slides in new window Download PDF of all slides

Introduction

Construct validity

Statistical conclusion validity

Internal validity

External validity

Tip

Fun fact: If you type ? (or shift + /) while going through the slides, you can see a list of special slide-specific commands.

Videos

Videos for each section of the lecture are available at this YouTube playlist.

• Introduction
• Construct validity
• Statistical conclusion validity
• Internal validity
• External validity

You can also watch the playlist (and skip around to different sections) here:

In-class stuff

Here are all the materials we’ll use in class:

• Session 6 FAQ slides (PDF)
• Zilch!
• Live R script
• restaurant_inspections.csv

References

Gertler, Paul J., Sebastian Martinez, Patrick Premand, Laura B. Rawlings, and Christel M. J. Vermeersch. 2016. Impact Evaluation in Practice. 2nd ed. Inter-American Development Bank; World Bank. https://openknowledge.worldbank.org/handle/10986/25030.

Huntington-Klein, Nick. 2021. The Effect: An Introduction to Research Design and Causality. Boca Raton, Florida: Chapman and Hall / CRC. https://theeffectbook.net/.

Imbens, Guido W. 2021. “Statistical Significance, p-Values, and the Reporting of Uncertainty.” Journal of Economic Perspectives 35 (3, Summer): 157–74. https://doi.org/10.1257/jep.35.3.157.