Program Evaluation for Data Science

This course empowers you to apply program evaluation to your data science projects. Learn the best practices of some of the main methods for program evaluation including A/B testing, difference-in-difference, regression discontinuity, interrupted time series, and matching problems. As part of these best practices, learn how to identify when A/B testing is not sufficient or possible and then apply your knowledge to determine if programs had an impact across a large variety of examples that leverage different methods.

Learning objectives
Analyze program evaluation methods by comparing different evaluation approaches, identifying appropriate methods for specific scenarios, evaluating the strengths and limitations of each method, and assessing the requirements for causal inference.
Apply A/B testing methodology through implementing randomization procedures, conducting quality checks on randomization, evaluating balance in treatment groups, interpreting results from randomized experiments.
Evaluate whether other causal inference methods (beyond A/B testing) are appropriate, including matching methods, difference-in-difference approaches, regression discontinuity designs, and interrupted time series analysis.
Demonstrate the ability to select appropriate methods based on context, implement best practices for each approach, assess method-specific quality checks, and interpret results within methodological limitations.
Design program evaluations by creating evaluation protocols, selecting appropriate methodological approaches, implementing quality control measures, and developing analysis strategies.
Interpret evaluation results through analyzing method-specific outputs, drawing appropriate causal conclusions, identifying limitations of findings, and applying results to practical scenarios.