class: center middle main-title section-title-3 # In-person<br>session 4 .class-info[ **February 2, 2023** .light[PMAP 8521: Program evaluation<br> Andrew Young School of Policy Studies ] ] --- name: outline class: title title-inv-8 # Plan for today -- .box-1.medium.sp-after-half[Super quick R FAQs] -- .box-4.medium.sp-after-half[Regression!] -- .box-2.medium.sp-after-half[Measuring outcomes] -- .box-5.medium[DAGs] --- layout: false name: r-stuff class: center middle section-title section-title-1 animated fadeIn # Quick useful<br>R tips --- layout: true class: middle --- .box-1.large[Weird figure/table<br>placement in PDFs] ??? https://bookdown.org/yihui/rmarkdown-cookbook/figure-placement.html --- .box-1.large[Figure sizing in R] --- .box-1.large[Figure and table<br>captions and numbers] ??? - <https://bookdown.org/yihui/rmarkdown-cookbook/figure-number.html> - <https://rmd4sci.njtierney.com/figures-tables-captions-.html> - <https://rmd4sci.njtierney.com/start.html> - <https://quarto.org/docs/authoring/cross-references.html> --- .box-1.large[Make nicer tables when<br>knitting with `kable()`] .box-inv-1[(Or even fancier tables with [kableExtra](https://cran.r-project.org/web/packages/kableExtra/vignettes/awesome_table_in_html.html)!)] ??? - <https://stats.andrewheiss.com/testy-turtle/notebook/models_total-aid.html#table> - <https://stats.andrewheiss.com/testy-turtle/notebook/models_aid-recipients.html#table> --- layout: false name: regression class: center middle section-title section-title-5 animated fadeIn # Regression! --- layout: true class: middle --- .box-4.large.sp-after[Drawing lines through points] -- .center.sp-after[<https://evalsp23.classes.andrewheiss.com/slides/02-slides.html#17>] -- .box-4[<span style="color: #F6D645;">Lo</span>cally <span style="color: #F6D645;">e</span>stimated/<span style="color: #F6D645;">we</span>ighted <span style="color: #F6D645;">s</span>catterplot <span style="color: #F6D645;">s</span>moothing (LOESS/LOWESS)<br>is a common method (but not the only one!)] --- .center[ <video controls> <source src="img/10/loess_window.mp4" type="video/mp4"> </video> ] --- .box-4.large[Regression equations] .box-inv-4[And is the intercept ever useful,<br>or should we always ignore it?] ??? <https://evalsp23.classes.andrewheiss.com/slides/02-slides.html#38> <https://www.andrewheiss.com/blog/2022/05/20/marginalia/#regression-sliders-switches-and-mixing-boards> --- .box-4.medium[What does it mean to<br>hold something constant?] .box-4.medium[Why is one category always left out<br>when you use a categorical variable?] --- .center[ <figure> <img src="img/02-class/slider-switch-annotated-80.jpg" alt="Single slider and switch" title="Single slider and switch" width="100%"> </figure> ] --- .center[ <figure> <img src="img/02-class/mixer-board-annotated-80.jpg" alt="Mixer board" title="Mixer board" width="100%"> </figure> ] --- .box-4.medium.sp-after[Why use two steps to create a regression in R?<br>(i.e. assigning it to an object with `<-`?)] .box-4.medium[Why use `tidy()`<br>from the broom package?] ??? Show model with `lm()`; show t-test with `t.test()`; show both through `tidy()` Use **marginaleffects** --- .box-4.medium[How was the 0.05 significance<br>threshold determined?] .box-4.medium[Could we say something is significant<br>if p > 0.05, but just note that it is at<br>a higher p-value?<br>Or does it have to fall under 0.05?] --- .box-4.large[Why all this convoluted<br>logic of null worlds?] --- layout: false class: title title-4 # Different "dialects" of statistics .pull-left[ .box-4.medium[Frequentist] `$$P(\text{data} \mid H_0)$$` .box-inv-4["Regular" statistics;<br>what you've learned<br><small>(and are learning here)</small>] ] -- .pull-right[ .box-4.medium[Bayesian] `$$P(H \mid \text{data})$$` .box-inv-4[Requires lots of<br>computational power] ] --- layout: true class: middle --- .box-4.medium[Do we care about the actual coefficients<br>or just whether or not they're significant?] .box-4.medium[How does significance relate to causation?] .box-4.medium[If we can't use statistics to assert causation<br>how are we going to use this information<br>in program evaluation?] --- .box-4.large[What counts as a "good" R²?] --- .center[ <figure> <img src="img/02-class/nice-plot-1.png" alt="Euler diagram" title="Euler diagram" width="45%"> </figure> ] --- .center[ <figure> <img src="img/02-class/plot-diagram-prediction-1.png" alt="R2 prediction" title="R2 prediction" width="75%"> </figure> ] --- .center[ <figure> <img src="img/02-class/plot-diagram-estimation-1.png" alt="R2 estimation" title="R2 estimation" width="75%"> </figure> ] --- .center[ <figure> <img src="img/02-class/prediction-vs-estimation.jpg" alt="R2 estimation vs prediction" title="R2 estimation vs prediction" width="100%"> </figure> ] --- layout: false name: outcomes class: center middle section-title section-title-2 animated fadeIn # Measuring<br>outcomes --- layout: true class: title title-2 --- # Outcomes and programs .box-2.medium[Outcome variable] .box-inv-2.sp-after-half.SMALL[Thing you're measuring] -- .box-2.medium[Outcome change] .box-inv-2.sp-after-half.SMALL[∆ in thing you're measuring over time] -- .box-2.medium[Program effect] .box-inv-2.SMALL[∆ in thing you're measuring over time *because of* the program] --- # Outcomes and programs <figure> <img src="img/04/program-effect.png" alt="Outcomes and program effect" title="Outcomes and program effect" width="100%"> </figure> --- layout: false class: middle .box-2.large[Abstraction]