Module 2
Comparison Groups
The heart of causal evaluation: why before-and-after misleads, how to build a valid comparison, and why value-based models rarely randomize.
By the end of this module, you will be able to:
- Explain why before-and-after comparisons cannot isolate a program's effect
- Apply difference-in-differences to separate a program effect from background trend
- Judge how much weight a study deserves based on how its participants were selected
- Why Pre-Post Comparisons Mislead Comparing a program's population before and after seems obvious and is usually wrong. This lesson shows why you need a comparison group. About 4 min
- Building a Valid Comparison A comparison group only works if it is truly comparable. This lesson covers matching and difference-in-differences in plain language. About 5 min
- Randomized vs Observational Randomization solves the comparison problem cleanly, and value-based models almost never use it. This lesson explains the trade-off you are always reading around. About 5 min
Module quiz
Answer all questions to see your score.
1. Why is a before-and-after comparison with no comparison group unreliable?
Without a comparison group experiencing the same background trends, pre-post conflates the program's effect with everything else that changed.
2. What does difference-in-differences measure?
Subtracting the comparison group's change nets out shared trends, leaving the program's added effect.
3. Why is randomization the strongest evaluation design?
Only randomization balances the unmeasured, which matching cannot do; its rarity in voluntary value-based models is why most evidence is weaker.