class: center middle main-title section-title-7 # Diff-in-diff I & II .class-info[ **Sessions 8–9** .light[PMAP 8521: Program evaluation<br> Andrew Young School of Policy Studies ] ] --- name: outline class: title title-inv-8 # Plan for today -- .box-6.medium.sp-after-half[Quasi-experiments] -- .box-2.medium.sp-after-half[Interactions & regression] -- .box-5.medium.sp-after-half[Two wrongs make a right] -- .box-3.medium.sp-after-half[Diff-in-diff assumptions] --- name: quasi-experiments class: center middle section-title section-title-6 animated fadeIn # Quasi-experiments --- .box-6.large.sp-after[RCTs are great!] -- .box-6.large[Super impractical to do<br>all the time though!] --- layout: true class: title title-6 --- # Quasi-experiments -- .box-inv-6.medium[You can't always randomly<br>assign people to do things] -- .box-inv-6.medium[So let other people (or the government,<br>or nature, or something else) do it for you] --- # Quasi-experiments .box-inv-6.medium.sp-after[**Quasi-experiment**<br>A situation where you, as researcher,<br>did not assign people to treatment/control] -- .center.float-left.sp-after[ .box-6[External validity 👍] .box-6[Selection 👎] ] -- .box-inv-6.medium[Assignment to treatment is "as if" random] ??? Good bc of external validity; bad bc of selection --- # Quasi-experiments vs. DAG adjustment -- .box-inv-6[We did a lot of work with DAGs!<br>You're good at closing backdoors with matching and IPW] -- .box-inv-6.sp-after[DAGs can work for any kind of observational data,<br>even without a quasi-experimentalish situation] -- .box-6[Quasi-experiments are a little different:<br>the **context** isolates pathway between treatment and outcome] -- .box-6.smaller[They're wildly popular in social sciences (especially economics!),<br>maybe more credible (?) there than just making DAG adjustments] -- .box-6.smaller[You can still draw a DAG for a quasi-experiment though!] --- # Analyzing quasi-experiments -- .box-inv-6.medium[Difference-in-differences] .box-6.small.sp-after[DiD; DD; diff-in-diff] -- .box-inv-6.medium[Regression discontinuity] .box-6.small.sp-after[RD; RDD] -- .box-inv-6.medium[Instrumental variables] .box-6.small[IV] --- layout: false name: interactions-regression class: center middle section-title section-title-2 animated fadeIn # Interactions & regression --- layout: true class: title title-2 --- # Sliders and switches .center[ <figure> <img src="img/02/slider-switch-annotated-80.jpg" alt="Switch and slider" title="Switch and slider" width="100%"> </figure> ] --- layout: false .smaller[ $$ \widehat{\text{Happiness}} = \beta_0 + \beta_1 \text{Life expectancy} + \beta_2 \text{Latin America} + \varepsilon $$ ] -- .small-code[ ```r model1 <- lm(happiness_score ~ life_expectancy + latin_america, data = world_happiness) tidy(model1) ``` ``` ## # A tibble: 3 × 5 ## term estimate std.error statistic p.value ## <chr> <dbl> <dbl> <dbl> <dbl> ## 1 (Intercept) -2.08 0.537 -3.87 1.61e- 4 ## 2 life_expectancy 0.102 0.00745 13.7 1.95e-28 ## 3 latin_americaLatin America 0.623 0.173 3.61 4.17e- 4 ``` ] -- .pull-left[ .box-inv-2.small[Life expectancy = continuous / slider] .box-2.smaller["For every 1-year increase in life expectancy,<br>happiness is associated with a β<sub>1</sub> increase"] ] -- .pull-right[ .box-inv-2.small[Latin America = categorical / switch] .box-2.smaller["Being in Latin America is associated<br>with a β<sub>2</sub> increase in happiness"] ] --- class: title title-2 # Indicators and interactions .box-inv-2.medium[Indicators .tiny[(dummies)]] -- .box-2.sp-after[Change in <span style="color: #F6D645;">intercept</span> for specific group] --- layout: false <img src="08-slides_files/figure-html/plot-indicator-1-1.png" width="100%" style="display: block; margin: auto;" /> --- <img src="08-slides_files/figure-html/plot-indicator-2-1.png" width="100%" style="display: block; margin: auto;" /> .box-2.small[World slope = 0.102] --- <img src="08-slides_files/figure-html/plot-indicator-3-1.png" width="100%" style="display: block; margin: auto;" /> .box-2.small[Latin America intercept shifted up 0.62; line has same slope as world (0.102)] --- .smaller[ $$ `\begin{aligned} \widehat{\text{Happiness}} = &\beta_0 + \beta_1 \text{Life expectancy} + \beta_2 \text{Latin America} + \\ &\beta_3 (\text{Life expectancy} \times \text{Latin America}) + \varepsilon \end{aligned}` $$ ] -- .small-code[ ```r model2 <- lm(happiness_score ~ life_expectancy + latin_america + (life_expectancy * latin_america), data = world_happiness) tidy(model2) ``` ``` ## # A tibble: 4 × 5 ## term estimate std.error stati…¹ p.value ## <chr> <dbl> <dbl> <dbl> <dbl> ## 1 (Intercept) -2.02 0.545 -3.70 2.98e- 4 ## 2 life_expectancy 0.102 0.00757 13.4 1.65e-27 ## 3 latin_americaLatin America -1.52 3.36 -0.450 6.53e- 1 ## 4 life_expectancy:latin_americaLatin America 0.0288 0.0453 0.637 5.25e- 1 ## # … with abbreviated variable name ¹statistic ``` ] -- .box-2.small["In Latin America, for every 1-year increase in life expectancy,<br>happiness is associated with a <span style="color: #F6D645;">β<sub>1</sub> + β<sub>3</sub></span> increase *and* the intercept is β<sub>2</sub> lower"] --- class: title title-2 # Indicators and interactions .box-inv-2.medium[Indicators .tiny[(dummies)]] .box-2.sp-after[Change in <span style="color: #F6D645;">intercept</span> for specific group] .box-inv-2.medium.sp-before[Interactions] -- .box-2[Change in <span style="color: #F6D645;">slope</span> for specific group] --- <img src="08-slides_files/figure-html/plot-interaction-1-1.png" width="100%" style="display: block; margin: auto;" /> .box-2.small[Latin America slope is 0.029 + 0.102 = <span style="color: #F6D645;">0.13</span>; different from rest of the world] --- class: title title-2 # Interactions .box-inv-2[What would happen if you ran this?] .small-code[ ```r model3 <- lm(happiness_score ~ (life_expectancy * latin_america), data = world_happiness) ``` ] -- .small-code[ ``` ## # A tibble: 4 × 5 ## term estimate std.error stati…¹ p.value ## <chr> <dbl> <dbl> <dbl> <dbl> ## 1 (Intercept) -2.02 0.545 -3.70 2.98e- 4 ## 2 life_expectancy 0.102 0.00757 13.4 1.65e-27 ## 3 latin_americaLatin America -1.52 3.36 -0.450 6.53e- 1 ## 4 life_expectancy:latin_americaLatin America 0.0288 0.0453 0.637 5.25e- 1 ## # … with abbreviated variable name ¹statistic ``` ] -- .box-2.small[**It still works!**<br>Both terms have to be in the model; R will add them for you if you leave them out] --- class: title title-2 # Interactions .box-inv-2[What would happen if you ran this?] .small-code[ ```r model4 <- lm(happiness_score ~ life_expectancy * region, # region has multiple categories data = world_happiness) ``` ] -- .pull-left-wide[ .small-code.smaller[ ``` ## # A tibble: 14 × 5 ## term estim…¹ std.e…² stati…³ p.value ## <chr> <dbl> <dbl> <dbl> <dbl> ## 1 (Intercept) -2.81 2.05 -1.37 1.73e-1 ## 2 life_expectancy 0.112 0.0271 4.12 6.33e-5 ## 3 regionEurope & Central Asia -2.78 2.76 -1.01 3.16e-1 ## 4 regionLatin America & Caribbean -0.724 3.72 -0.195 8.46e-1 ## 5 regionMiddle East & North Africa -3.13 3.14 -0.997 3.21e-1 ## 6 regionNorth America 2.88 23.2 0.124 9.01e-1 ## 7 regionSouth Asia 4.98 5.54 0.898 3.71e-1 ## 8 regionSub-Saharan Africa 6.33 2.48 2.55 1.18e-2 ## 9 life_expectancy:regionEurope & Central Asia 0.0367 0.0361 1.02 3.11e-1 ## 10 life_expectancy:regionLatin America & Caribb… 0.0187 0.0497 0.376 7.07e-1 ## 11 life_expectancy:regionMiddle East & North Af… 0.0410 0.0419 0.978 3.30e-1 ## 12 life_expectancy:regionNorth America -0.0221 0.288 -0.0767 9.39e-1 ## 13 life_expectancy:regionSouth Asia -0.0768 0.0790 -0.972 3.33e-1 ## 14 life_expectancy:regionSub-Saharan Africa -0.101 0.0354 -2.84 5.12e-3 ## # … with abbreviated variable names ¹estimate, ²std.error, ³statistic ``` ] ] -- .pull-right-narrow[ .box-2.small[Changes in<br>slopes and intercepts<br>for each region] ] --- class: title title-2 # General idea of interactions .box-inv-2.medium[The *additional* change that happens when<br>combining two explanatory variables] -- .box-2[Life expectancy effect] -- .box-2[Latin America effect] -- .box-2[Additional life expectancy effect in Latin America] --- .pull-left[ .box-2[Is there a discount when combining cheese and chili?] .box-inv-2[What is the cheese effect?] .box-inv-2[What is the chili effect?] .box-inv-2[What is the<br>chili × cheese effect?] ] .pull-right.center[ <figure> <img src="img/08/hotdogs.jpg" alt="Hot dog prices" title="Hot dog prices" width="90%"> </figure> ] ??? - Cheese effect: 0.35 - Chili effect: 0.35 - Chili × cheese effect: 0 https://twitter.com/kevin_stange/status/1186056025897684992 --- layout: false name: two-wrongs class: center middle section-title section-title-5 animated fadeIn # Two wrongs make a right --- class: bg-full background-image: url("img/08/federalism.png") ??? https://twitter.com/causalinf/status/1021392181209255938 --- layout: true class: title title-5 --- # Raising the minimum wage .box-inv-5.medium.sp-after-half[What happens if you raise the minimum wage?] -- .box-5.sp-after[Economic theory says there<br>should be fewer jobs] -- .box-inv-5.medium[New Jersey in 1992] .box-inv-5.medium[$4.25 → $5.05] --- # Before vs. after .box-inv-5.medium.sp-after-half[Average # of jobs per fast food restaurant in NJ] -- .box-5[New Jersey<sub>Before change</sub> = 20.44] -- .box-5[New Jersey<sub>After change</sub> = 21.03] -- .box-5.sp-after[∆ = 0.59] -- .box-inv-5.medium[Is this the causal effect?] ??? No! Not possible because maybe other things happened in NJ to cause that – there's no control state to compare with NJ --- # Treatment vs. control .box-inv-5.medium.sp-after-half[Average # of jobs per fast food restaurant] -- .box-5[Pennsylvania<sub>After change</sub> = 21.17] -- .box-5[New Jersey<sub>After change</sub> = 21.03] -- .box-5.sp-after[∆ = −0.14] -- .box-inv-5.medium[Is this the causal effect?] ??? No! There's no pre-level to compare with. Maybe both states went up, maybe both went down. Can't tell. --- # Problems -- .box-inv-5.medium[Comparing only before/after] -- .box-5.small[You're only looking at the treatment group!] -- .box-5.small.sp-after[Impossible to know if change happened because of treatment or just naturally] -- .box-inv-5.medium.sp-before[Comparing only treatment/control] -- .box-5.small[You're only looking at post-treatment values] -- .box-5.small[Impossible to know if change happened because of natural growth] --- layout: false <img src="08-slides_files/figure-html/min-wage-dag-1.png" width="100%" style="display: block; margin: auto;" /> --- <table style="margin-left: auto; margin-right: auto;"> <thead> <tr> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Pre mean </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Post mean </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> <span class="color-light-5">∆ <span class="smaller">(post − pre)</span></span> </th> </tr> </thead> <tbody> <tr> <td style="text-align:center;background-color: #FFFFFF !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Control </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>A</b><br><span class="small">(never treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>B</b><br><span class="small">(never treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;background-color: #DDDDDD !important;color: #FFFFFF !important;background-color: #FFFFFF !important;"> <b>B − A</b> </td> </tr> <tr> <td style="text-align:center;background-color: #FFFFFF !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Treatment </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>C</b><br><span class="small">(not yet treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>D</b><br><span class="small">(treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;background-color: #DDDDDD !important;color: #FFFFFF !important;background-color: #FFFFFF !important;"> <b>D − C</b> </td> </tr> <tr> <td style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;color: #FFFFFF !important;background-color: #FFFFFF !important;"> ∆<br><span class="smaller">(treatment − control)</span> </td> <td style="text-align:center;color: #FFFFFF !important;background-color: #FFFFFF !important;"> <b>C − A</b> </td> <td style="text-align:center;color: #FFFFFF !important;background-color: #FFFFFF !important;"> <b>D − B</b> </td> <td style="text-align:center;background-color: #DDDDDD !important;color: #FFFFFF !important;background-color: #FFFFFF !important;color: #FFFFFF !important;background-color: #FFFFFF !important;"> <span class=""><b>(B − A) − (D − C)</b></span> <i>or</i><br><span class=""><b>(B − D) − (A − C)</b></span> </td> </tr> </tbody> </table> --- <table style="margin-left: auto; margin-right: auto;"> <thead> <tr> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Pre mean </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Post mean </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> ∆ <span class="smaller">(post − pre)</span> </th> </tr> </thead> <tbody> <tr> <td style="text-align:center;background-color: #FFFFFF !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Control </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>A</b><br><span class="small">(never treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>B</b><br><span class="small">(never treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;background-color: #DDDDDD !important;"> <b>B − A</b> </td> </tr> <tr> <td style="text-align:center;background-color: #FFFFFF !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Treatment </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>C</b><br><span class="small">(not yet treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>D</b><br><span class="small">(treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;background-color: #DDDDDD !important;"> <b>D − C</b> </td> </tr> <tr> <td style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;color: #FFFFFF !important;background-color: #FFFFFF !important;"> ∆<br><span class="smaller">(treatment − control)</span> </td> <td style="text-align:center;color: #FFFFFF !important;background-color: #FFFFFF !important;"> <b>A − C</b> </td> <td style="text-align:center;color: #FFFFFF !important;background-color: #FFFFFF !important;"> <b>B − D</b> </td> <td style="text-align:center;background-color: #DDDDDD !important;color: #FFFFFF !important;background-color: #FFFFFF !important;"> <span class=""><b>(B − A) − (D − C)</b></span> <i>or</i><br><span class=""><b>(B − D) − (A − C)</b></span> </td> </tr> </tbody> </table> .box-5[∆ (post − pre) = **within-unit growth**] --- <table style="margin-left: auto; margin-right: auto;"> <thead> <tr> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Pre mean </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Post mean </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> <span class="color-light-5">∆ <span class="smaller">(post − pre)</span></span> </th> </tr> </thead> <tbody> <tr> <td style="text-align:center;background-color: #FFFFFF !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Control </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>A</b><br><span class="small">(never treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>B</b><br><span class="small">(never treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;color: #FFFFFF !important;background-color: #FFFFFF !important;"> <b>B − A</b> </td> </tr> <tr> <td style="text-align:center;background-color: #FFFFFF !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Treatment </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>C</b><br><span class="small">(not yet treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>D</b><br><span class="small">(treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;color: #FFFFFF !important;background-color: #FFFFFF !important;"> <b>D − C</b> </td> </tr> <tr> <td style="text-align:center;background-color: #DDDDDD !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> ∆<br><span class="smaller">(treatment − control)</span> </td> <td style="text-align:center;background-color: #DDDDDD !important;"> <b>C − A</b> </td> <td style="text-align:center;background-color: #DDDDDD !important;"> <b>D − B</b> </td> <td style="text-align:center;background-color: #DDDDDD !important;color: #FFFFFF !important;background-color: #FFFFFF !important;"> <span class=""><b>(B − A) − (D − C)</b></span> <i>or</i><br><span class=""><b>(B − D) − (A − C)</b></span> </td> </tr> </tbody> </table> .box-5[∆ (treatment − control) = **across-group growth**] --- <table style="margin-left: auto; margin-right: auto;"> <thead> <tr> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Pre mean </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Post mean </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> ∆ <span class="smaller">(post − pre)</span> </th> </tr> </thead> <tbody> <tr> <td style="text-align:center;background-color: #FFFFFF !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Control </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>A</b><br><span class="small">(never treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>B</b><br><span class="small">(never treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;background-color: #DDDDDD !important;"> <b>B − A</b> </td> </tr> <tr> <td style="text-align:center;background-color: #FFFFFF !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Treatment </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>C</b><br><span class="small">(not yet treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>D</b><br><span class="small">(treated) </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;background-color: #DDDDDD !important;"> <b>D − C</b> </td> </tr> <tr> <td style="text-align:center;background-color: #DDDDDD !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> ∆<br><span class="smaller">(treatment − control)</span> </td> <td style="text-align:center;background-color: #DDDDDD !important;"> <b>C − A</b> </td> <td style="text-align:center;background-color: #DDDDDD !important;"> <b>D − B</b> </td> <td style="text-align:center;background-color: #DDDDDD !important;background-color: #DDDDDD !important;"> <span class="color-5"><b>(D − C) − (B − A)</b></span> <i>or</i><br><span class="color-5"><b>(D − B) − (C − A)</b></span> </td> </tr> </tbody> </table> .box-5[∆<sub>within units</sub> − ∆<sub>within groups</sub> =<br>Difference-in-differences =<br>causal effect!] --- class: middle .medium[ `$$\begin{aligned} \text{DD}\ =\ &(\bar{x}_\text{treatment, post} - \bar{x}_\text{treatment, pre}) \\ &- (\bar{x}_\text{control, post} - \bar{x}_\text{control, pre}) \end{aligned}$$` ] --- <table style="margin-left: auto; margin-right: auto;"> <thead> <tr> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Pre mean </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Post mean </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> ∆ <span class="smaller">(post − pre)</span> </th> </tr> </thead> <tbody> <tr> <td style="text-align:center;background-color: #FFFFFF !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Pennsylvania </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>23.33</b><br><span class="small">A </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>21.17</b><br><span class="small">B </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;background-color: #DDDDDD !important;"> <b>-2.16</b><br><span class="small">B − A </span> </td> </tr> <tr> <td style="text-align:center;background-color: #FFFFFF !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> New Jersey </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>20.44</b><br><span class="small">C </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b>21.03</b><br><span class="small">D </span> </td> <td style="text-align:center;background-color: #FFFFFF !important;background-color: #DDDDDD !important;"> <b>0.59</b><br><span class="small">D − C </span> </td> </tr> <tr> <td style="text-align:center;background-color: #DDDDDD !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> ∆<br><span class="smaller">(NJ − PA)</span> </td> <td style="text-align:center;background-color: #DDDDDD !important;"> <b>-2.89</b><br><span class="small">C − A </span> </td> <td style="text-align:center;background-color: #DDDDDD !important;"> <b>-0.14</b><br><span class="small">D − B </span> </td> <td style="text-align:center;background-color: #DDDDDD !important;background-color: #DDDDDD !important;"> <b>(0.59) − (−2.16) = <br><span class="color-5">2.76</span></b> </td> </tr> </tbody> </table> --- <img src="08-slides_files/figure-html/dd-graph-1.png" width="100%" style="display: block; margin: auto;" /> --- class: title title-5 # An easier way? -- .box-inv-5.medium[Finding all the group<br>means is tedious!] -- .box-inv-5.medium.sp-after[What if there are other<br>backdoors to worry about?] -- .box-5.medium[Regression to the rescue!] --- ![](08-slides_files/figure-html/min-wage-dag-1.png) --- `$$\begin{aligned} \color{#2ECC40}{Y_{it}}\ =\ &\alpha + \beta\ \color{#0074D9}{\text{Group}_i} + \gamma\ \color{#39CCCC}{\text{Time}_t} + \\ &\delta\ \color{#FF4136}{(\text{Group}_i \times \text{Time}_t)} + \varepsilon_{it} \end{aligned}$$` -- .center.sp-after[ <code class ='r hljs remark-code'>model <- lm(<b><span style="color:#2ECC40">outcome</span></b> ~<b><span style="color:#0074D9"> group </span></b>+<b><span style="color:#39CCCC"> time </span></b>+ <b><span style="color:#FF4136">(group * time)</span></b>)</code> ] -- <span class="box-5", style="background-color: #0074D9">Group = 1 or TRUE if treatment</span> -- <span class="box-5", style="background-color: #39CCCC">Time = 1 or TRUE if after</span> --- `$$\begin{aligned} \color{#2ECC40}{Y_{it}}\ =\ &\color{#666666}{\alpha} + \color{#0074D9}{\beta\ \text{Group}_i} + \color{#39CCCC}{\gamma\ \text{Time}_t} + \\ &\color{#FF4136}{\delta\ (\text{Group}_i \times \text{Time}_t)} + \varepsilon_{it} \end{aligned}$$` .center.sp-after[ <code class ='r hljs remark-code'>model <- lm(<b><span style="color:#2ECC40">outcome</span></b> ~<b><span style="color:#0074D9"> group </span></b>+<b><span style="color:#39CCCC"> time </span></b>+ <b><span style="color:#FF4136">(group * time)</span></b>)</code> ] -- <span class="box-5", style="background-color: #666666">α = Mean of control, pre-treatment</span> -- <span class="box-5", style="background-color: #0074D9">β = Increase in outcome across groups</span> -- <span class="box-5", style="background-color: #39CCCC">γ = Increase in outcome over time within units</span> -- <span class="box-5", style="background-color: #FF4136">δ = Difference in differences!</span> --- `$$\begin{aligned} \color{#2ECC40}{Y_{it}}\ =\ &\color{#666666}{\alpha} + \color{#0074D9}{\beta\ \text{Group}_i} + \color{#39CCCC}{\gamma\ \text{Time}_t} + \\ &\color{#FF4136}{\delta\ (\text{Group}_i \times \text{Time}_t)} + \varepsilon_{it} \end{aligned}$$` <table style="margin-left: auto; margin-right: auto;"> <thead> <tr> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Pre mean </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Post mean </th> <th style="text-align:center;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> ∆ <span class="smaller">(post − pre)</span> </th> </tr> </thead> <tbody> <tr> <td style="text-align:center;background-color: #FFFFFF !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Control </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b><span style="color: #666666;">α</span></b> </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b><span style="color: #666666;">α</span> + <span style="color: #39CCCC;">γ</span></b> </td> <td style="text-align:center;background-color: #FFFFFF !important;background-color: #EEEEEE !important;"> <b><span style="color: #39CCCC;">γ</span></b> </td> </tr> <tr> <td style="text-align:center;background-color: #FFFFFF !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> Treatment </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b><span style="color: #666666;">α</span> + <span style="color: #0074D9;">β</span></b> </td> <td style="text-align:center;background-color: #FFFFFF !important;"> <b><span style="color: #666666;">α</span> + <span style="color: #0074D9;">β</span> + <span style="color: #39CCCC;">γ</span> + <span style="color: #FF4136">δ</span></b> </td> <td style="text-align:center;background-color: #FFFFFF !important;background-color: #EEEEEE !important;"> <b><span style="color: #39CCCC;">γ</span> + <span style="color: #FF4136">δ</span></b> </td> </tr> <tr> <td style="text-align:center;background-color: #EEEEEE !important;font-weight: bold;color: #CF4446 !important;background-color: #FFC6C6 !important;"> ∆ <span class="smaller">(trtmt − ctrl)</span> </td> <td style="text-align:center;background-color: #EEEEEE !important;"> <b><span style="color: #0074D9;">β</span></b> </td> <td style="text-align:center;background-color: #EEEEEE !important;"> <b><span style="color: #0074D9;">β</span> + <span style="color: #FF4136">δ</span></b> </td> <td style="text-align:center;background-color: #EEEEEE !important;background-color: #EEEEEE !important;"> <b><span style="color: #FF4136">δ</span></b> </td> </tr> </tbody> </table> --- .pull-left[ .center[ <figure> <img src="img/08/hotdogs-small.jpg" alt="Hot dog prices" title="Hot dog prices" width="50%"> </figure> ] .small-code[ ```r hotdogs ``` ``` ## # A tibble: 4 × 3 ## price cheese chili ## <dbl> <lgl> <lgl> ## 1 2 FALSE FALSE ## 2 2.35 TRUE FALSE ## 3 2.35 FALSE TRUE ## 4 2.7 TRUE TRUE ``` ] ] -- .pull-right.small-code[ ```r model_hotdogs <- lm(price ~ cheese + chili + cheese * chili, data = hotdogs) ``` ```r tidy(model_hotdogs) ``` ``` ## # A tibble: 4 × 2 ## term estimate ## <chr> <dbl> ## 1 (Intercept) 2 ## 2 cheeseTRUE 0.35 ## 3 chiliTRUE 0.35 ## 4 cheeseTRUE:chiliTRUE 0 ``` ] --- class: bg-full background-image: url("img/08/pokemon1.png") ??? https://doi.org/10.1136/bmj.i6270 --- class: bg-full background-image: url("img/08/pokemon2.png") ??? https://doi.org/10.1136/bmj.i6270 --- class: bg-full background-image: url("img/08/pokemon3.png") ??? https://doi.org/10.1136/bmj.i6270 --- layout: false name: diff-diff-assumptions class: center middle section-title section-title-3 animated fadeIn # Diff-in-diff assumptions --- class: title title-3 # Assumptions .box-inv-3.medium[Parallel trends] -- .box-3[Treatment and control groups might have different values<br>at first, but we assume that the treatment group would<br>have changed like the control group in the absence of treatment] --- <img src="08-slides_files/figure-html/plot-trends-1-1.png" width="100%" style="display: block; margin: auto;" /> --- <img src="08-slides_files/figure-html/plot-trends-2-1.png" width="100%" style="display: block; margin: auto;" /> --- <img src="08-slides_files/figure-html/plot-trends-3-1.png" width="100%" style="display: block; margin: auto;" /> --- class: title title-3 # Assumptions .box-inv-3.medium[Parallel trends] -- .box-3[Check by pretending the treatment happened earlier;<br>if there's an effect, there's likely an underlying trend] --- <img src="08-slides_files/figure-html/plot-trends-2-back-1.png" width="100%" style="display: block; margin: auto;" /> --- <img src="08-slides_files/figure-html/plot-trends-3-back-1.png" width="100%" style="display: block; margin: auto;" /> --- <img src="08-slides_files/figure-html/plot-trends-1-back-1.png" width="100%" style="display: block; margin: auto;" /> --- class: title title-3 # Assumptions .box-inv-3.medium[Treatment timing] -- .box-3[Units often receive treatment at different times,<br>which can distort your estimate!] --- <img src="08-slides_files/figure-html/gb-trends-1-1.png" width="100%" style="display: block; margin: auto;" /> --- <img src="08-slides_files/figure-html/gb-trends-2-1.png" width="100%" style="display: block; margin: auto;" /> --- class: title title-3 # Assumptions .box-3[You can check how big of an issue this is<br>with Goodman-Bacon decomposition] -- .box-inv-3[R package: `bacondecomp`] .center[ <figure> <img src="img/08/goodman-bacon.png" alt="Goodman-Bacon paper" title="Goodman-Bacon paper" width="45%"> </figure> ] ??? https://www.nber.org/papers/w25018.pdf