Syllabus

Instructor

Course details

  •   Thursdays
  •   January 9–May 2, 2023
  •   7:15–9:45 PM
  •   Aderhold Learning Center 204
  •   Slack

Course objectives

By the end of this course, you (1) will be literate in the language of causal inference, (2) will communicate evaluation outcomes clearly, and (3) will understand the ethics and limits of data analysis by designing, critiquing, coding, and running rigorous, valid, and feasible evaluations of public sector programs focused on society’s most pressing problems.

Specifically, you’ll be able to:

  • Explain the philosophy of causation
  • Identify and diagram program logic models
  • Outline theories of change with directed acyclic graphs (DAGs)
  • Summarize key threats to causal inference, identify these threats in evaluations, and mitigate these threats with research design
  • Develop rigorous and valid statistical measures
  • Run statistical models
  • Explain the theory, research design, methods, and results of evaluations to all types of stakeholders, from highly trained econometricians to the general public
  • Share your analyses and data with the public
  • Identify ethical issues and limits in data science and program evaluation
  • Become curious and confident in consuming and producing evaluations

Course philosophy

Classical statistics classes spend substantial time covering probability theory, null hypothesis testing, and other statistical tests first developed hundreds of years ago. Some classes don’t use software or actual real data and instead live in the world of mathematical proofs. They can be math-heavy and full of often unintuitive concepts and equations.

In this class, we will take the opposite approach. We begin with data and learn how to tidy, wrangle, manipulate, and visualize it with code. Later in the semester we’ll turn to the powerful toolbox of causal inference approaches, but continue to keep the focus on data as we do so.

In other words, there’s way less of this:

\[ f(x) = \dfrac{1}{\sqrt{2\pi}} e^{-\frac12 x^2} \]

And way more of this:

summary_monthly_temp <- weather %>%
  group_by(month) %>%
  summarize(mean = mean(temp),
            std_dev = sd(temp))

Over the last decade there has been a revolution in statistical and scientific computing. Open source languages like R and Python have overtaken older (and expensive!) corporate software packages like SAS and SPSS, and there are now thousands of books and blog posts and other online resources with excellent tutorials about how to analyze pretty much any kind of data.

This class will expose you to R—one of the most popular, sought-after, and in-demand statistical programming languages. Armed with the foundation of R skills you’ll learn in this class, you’ll know enough to be able to find how to analyze any sort of data-based question in the future.

Important pep talk!

I promise you can succeed in this class.

Learning R can be difficult at first—it’s like learning a new language, just like Spanish, French, or Chinese. Hadley Wickham—the chief data scientist at RStudio and the author of some amazing R packages you’ll be using like ggplot2—made this wise observation:

It’s easy when you start out programming to get really frustrated and think, “Oh it’s me, I’m really stupid,” or, “I’m not made out to program.” But, that is absolutely not the case. Everyone gets frustrated. I still get frustrated occasionally when writing R code. It’s just a natural part of programming. So, it happens to everyone and gets less and less over time. Don’t blame yourself. Just take a break, do something fun, and then come back and try again later.

Even experienced programmers and evaluators find themselves bashing their heads against seemingly intractable errors. If you’re finding yourself taking way too long hitting your head against a wall and not understanding, take a break, talk to classmates, e-mail me, etc.

Alison Horst: Gator error

Course materials

Most of the readings in this class are free.

Books

There are two official textbooks for the class. Both are available digitally and both are free.

Both of these books are written at a more general, easy-to-understand level with relatively minimal math. You do not need to understand all the equations and notation. If your eyes start to gloss over the Greek letters and subscripts, it’s okay. Try to learn them, but don’t stress out about it too much.

In previous versions of this class I used two different books instead of The Effect, and they’re still fantastic and I’d recommend checking them out. The Mixtape is a phenomenal, more detailed, more economics-focused, and mathier approach to these causal inference methods (and it’s free!), while ’Metrics Matter is a fairly canonical (and accessible!) book in the world of econometrics. I refer to these books a few times in the video lectures—don’t worry, you’ll get pretty much the same content in The Effect.

  • Scott Cunningham, Causal Inference: The Mixtape (New Haven, CT: Yale University Press, 2021), https://mixtape.scunning.com/. (Also free(!), but there’s also a print version for $35 that you might want to get for reference after the class is over)
  • Joshua D. Angrist and Jörn-Steffen Pischke, Mastering ’Metrics: The Path from Cause to Effect (Princeton, NJ: Princeton University Press, 2015). ($25 used or $30 new at Amazon)

Articles, book chapters, and other materials

There will also occasionally be additional articles and videos to read and watch. When this happens, links to these other resources will be included on the reading page for that week.

R and RStudio

You will do all of your analysis with the open source (and free!) programming language R. You will use RStudio as the main program to access R. Think of R as an engine and RStudio as a car dashboard—R handles all the calculations and the actual statistics, while RStudio provides a nice interface for running R code.

R is free, but it can sometimes be a pain to install and configure. To make life easier, you can (and should!) use the free RStudio.cloud service, which lets you run a full instance of RStudio in your web browser. This means you won’t have to install anything on your computer to get started with R! We will have a shared class workspace in RStudio.cloud that will let you quickly copy templates for labs and problem sets.

RStudio.cloud is convenient, but it can be slow and it is not designed to be able to handle larger datasets or more complicated analysis. Over the course of the semester, you’ll probably want to get around to installing R, RStudio, and other R packages on your computer and wean yourself off of RStudio.cloud. This isn’t necessary, but it’s helpful.

You can find instructions for installing R, RStudio, and all the tidyverse packages here.

Online help

Data science and statistical programming can be difficult. Computers are stupid and little errors in your code can cause hours of headache (even if you’ve been doing this stuff for years!).

Fortunately there are tons of online resources to help you with this. Two of the most important are StackOverflow (a Q&A site with hundreds of thousands of answers to all sorts of programming questions) and RStudio Community (a forum specifically designed for people using RStudio and the tidyverse (i.e. you)).

If you use Twitter, post R-related questions and content with #rstats. The community there is exceptionally generous and helpful.

Searching for help with R on Google can sometimes be tricky because the program name is, um, a single letter. Google is generally smart enough to figure out what you mean when you search for “r scatterplot”, but if it does struggle, try searching for “rstats” instead (e.g. “rstats scatterplot”).

Additionally, we have a class chatroom at Slack where anyone in the class can ask questions and anyone can answer. I will monitor Slack regularly and will respond quickly. (It’s one of the rare Slack workspaces where I actually have notifications enabled!) Ask questions about the readings, assignments, and project. You’ll likely have similar questions as your peers, and you’ll likely be able to answer other peoples’ questions too.

Course structure

We meet weekly from 7:15–9:45 PM on Thursdays in 204 Aderhold. However, despite policies and rhetoric to the contrary, we’re still in the middle of a severe global pandemic, and the BA.5 variant of COVID-19 is raging.

Accordingly, this semester we’re going to have a flipped classroom, which will let us maximize flexibility during this pandemic.

We will not have lectures during our regularly scheduled class time. Instead, you will do the readings and watch recorded lecture videos prior to each in-person class session. You can do the readings and watch the videos on your own schedule at whatever time works best for you. Many of you work full time and you have childcare and parental care responsibilities, leaving you with only evenings for coursework. I’ve designed this asynchronous system with you specifically in mind. I also can only really do teaching work at night when my kids are in bed—I recorded all these videos between like 10 PM and 2 AM. We’re all in similar pandemic boats.

We will do several things during our Thursday in-person classes:

  • Extensive Q&A: As you do the readings and watch the videos prior to class, you will inevitably have questions. In your weekly check-in assignment, you’ll submit (at least) 3 of those questions to me prior to class. We’ll spend a good chunk of each class answering, clarifying, debating, and discussing your questions.
  • Activities: In some weeks, we’ll do some in-class activities to help solidify concepts about logic models, DAGs, and other evaluation and causal inference principles.
  • R labs: We’ll spend a substantial time during each class learning and working with R together. You’ll need to bring a computer.

Attendance and participation

You’re expected to come to class each Thursday prepared—having read the material and watched the videos—and ready to discuss the content and work with R.

Attendance and participation are important to your success in this course. However, again, we’re still in the middle of a global pandemic. If you are sick or are incapable of participating meaningfully in class (e.g. you have stayed up all night and are going to fall asleep in class), please stay home. I will stream our in-person sessions on Thursdays via Webex (and will set up my own camera and microphone system to do it), so if you cannot attend, you can tune in if needed.

GSU has some new process for getting absences excused because of illness, but I don’t care about that process. If you’re sick, I don’t need a doctor’s note or anything. If a relative dies and you have to attend a funeral, do it!—don’t worry about sending me confirmation or anything. If your childcare situation falls through one week, focus on your kids!—again, don’t worry about sending me confirmation. You’re all adults—I trust you. Given the flipped nature of the course, you’ll be able to catch up on the material.

If you test positive for COVID-19, report it at https://cc-gsu.force.com/s/, hunker down, and focus on getting better! Please do not come to class.

Pandemic stuff

Masks and vaccines

GSU does not require this and I can’t require this but I am allowed to urge it so here’s me urging it:
Please get the COVID-19 vaccination and a booster shot (sign up for one here!). It is free. It saves lives. (I am fully vaccinated and boosted.)

GSU and the University System of Georgia do not have a mask mandate for students or faculty. However, I personally will be wearing a mask when the CDC’s community transmission levels in the Atlanta area are medium or above.

I strongly recommend/urge/encourage you to wear a mask in class on Thursdays regardless of your vaccination status. I will place a box of disposable masks by the door if you would like one.

Again, GSU does not require either vaccines or masks, and if you aren’t vaccinated or don’t wear a mask, there are no penalties.

Learning during a pandemic

Life absolutely sucks right now. None of us is really okay. We’re all just pretending.

You most likely know people who have lost their jobs, have tested positive for COVID-19, have been hospitalized, or have even died (I myself know people in all those categories). You all have increased (or possibly decreased) work responsibilities and increased family care responsibilities—you might be caring for extra people (young and/or old!) right now, and you are likely facing uncertain job prospects (or have been laid off!).

I’m fully committed to making sure that you learn everything you were hoping to learn from this class! I will make whatever accommodations I can to help you finish your problem sets, do well on your projects, and learn and understand the class material. Under ordinary conditions, I am flexible and lenient with grading and course expectations when students face difficult challenges. Under pandemic conditions, that flexibility and leniency is intensified.

If you tell me you’re having trouble, I will not judge you or think less of you. I hope you’ll extend me the same grace.

You never owe me personal information about your health (mental or physical). You are always welcome to talk to me about things that you’re going through, though. If I can’t help you, I usually know somebody who can.

If you need extra help, or if you need more time with something, or if you feel like you’re behind or not understanding everything, do not suffer in silence! Talk to me! I will work with you. I promise.

Please sign up for a time to meet with me during student hours at https://calendly.com/andrewheiss/. I’m also available through e-mail and Slack. I’ve enabled notifications on my Slack account, so I’ll see your messages quickly!

I want you to learn lots of things from this class, but I primarily want you to stay healthy, balanced, and grounded during this crisis.

Course policies

Be nice. Be honest. Don’t cheat.

We will also follow Georgia State’s Code of Conduct.

This syllabus reflects a plan for the semester. Deviations may become necessary as the semester progresses.

Student hours

Please watch this video:

Student hours are set times dedicated to all of you (most professors call these “office hours”; I don’t).1 This means that I will be in my office at home (wistfully) waiting for you to come by talk to me remotely with whatever questions you have. This is the best and easiest way to find me and the best chance for discussing class material and concerns.

Because of the pandemic, it is easiest to meet with me online via Webex for student hours. Make an appointment with me here—the confirmation e-mail will contain a link for a Webex meeting. You can also find me through e-mail and Slack.

Late work

You will lose 1 point per day for each day a problem set is late. This is designed to not be a huge penalty (3 days late = 27/30 points on a problem set that gets a ✓), but instead is a commitment device to help you stay on schedule.

Counseling and Psychological Services (CPS)

Life at GSU can be complicated and challenging (especially during a pandemic!). You might feel overwhelmed, experience anxiety or depression, or struggle with relationships or family responsibilities. Counseling and Psychological Services (CPS) provides free, confidential support for students who are struggling with mental health and emotional challenges. The CPS office is staffed by professional psychologists who are attuned to the needs of all types of college and professional students. Please do not hesitate to contact CPS for assistance—getting help is a smart and courageous thing to do.

Basic needs security

If you have difficulty affording groceries or accessing sufficient food to eat every day, or if you lack a safe and stable place to live, and you believe this may affect your performance in this course, please contact the Dean of Students for support. They can provide a host of services including free groceries from the Panther Pantry and assisting with homelessness with the Embark Network. Additionally, please talk to me if you are comfortable in doing so. This will enable me to provide any resources that I might possess.

Lauren’s Promise

I will listen and believe you if someone is threatening you.

Lauren McCluskey, a 21-year-old honors student athlete, was murdered on October 22, 2018 by a man she briefly dated on the University of Utah campus. We must all take action to ensure that this never happens again.

If you are in immediate danger, call 911 or GSU police (404-413-3333).

If you are experiencing sexual assault, domestic violence, or stalking, please report it to me and I will connect you to resources or call GSU’s Counseling and Psychological Services (404-413-1640).

Any form of sexual harassment or violence will not be excused or tolerated at Georgia State. GSU has instituted procedures to respond to violations of these laws and standards, programs aimed at the prevention of such conduct, and intervention on behalf of the victims. Georgia State University Police officers will treat victims of sexual assault, domestic violence, and stalking with respect and dignity. Advocates on campus and in the community can help with victims’ physical and emotional health, reporting options, and academic concerns.

Academic honesty

Violation of GSU’s Policy on Academic Honesty will result in an F in the course and possible disciplinary action.2 All violations will be formally reported to the Dean of Students.

Special needs

Students who wish to request accommodation for a disability may do so by registering with the Office of Disability Services. Students may only be accommodated upon issuance by the Office of Disability Services of a signed Accommodation Plan and are responsible for providing a copy of that plan to instructors of all classes in which accommodations are sought.

Students with special needs should then make an appointment with me during the first week of class to discuss any accommodations that need to be made.

Assignments and grades

You can find descriptions for all the assignments on the assignments page.

Assignment Points Percent
Weekly check-ins (14 × 10) 140 15.1%
Problem sets (9 × 30) 270 29.0%
Evaluation assignments (4 × 30) 120 12.9%
Exam 1 100 10.8%
Exam 2 100 10.8%
Final project 200 21.5%
Total 930 —
Grade Range Grade Range
A 93–100% C 73–76%
A− 90–92% C− 70–72%
B+ 87–89% D+ 67–69%
B 83–86% D 63–66%
B− 80–82% D− 60–62%
C+ 77–79% F < 60%

Star Wars

Once you have read this entire syllabus and the assignments page, please click here and e-mail me a picture of a cute Star Wars character.3 Brownie points if it’s animated.

Baby Yoda with IG-11

Footnotes

  1. There’s fairly widespread misunderstanding about what office hours actually are! Many students often think that they are the times I shouldn’t be disturbed, which is the exact opposite of what they’re for!↩︎

  2. So seriously, just don’t cheat or plagiarize!↩︎

  3. Baby Yoda, Babu Frik, porgs, etc. are all super fair game.↩︎