Module 3
The Traps
The specific biases that fool smart people: regression to the mean, selection effects, and assumption-driven benchmarks.
By the end of this module, you will be able to:
- Detect regression to the mean in a program that targets the highest-cost patients
- Identify how joining, leaving, and patient mix can manufacture a result on their own
- Explain how benchmark and trend assumptions can create or erase measured savings
- Regression to the Mean The single most common way health programs fool themselves. This lesson explains why targeting last year's most expensive patients guarantees a false success. About 5 min
- Selection Effects Who joins, who stays, and who leaves can create a result all by itself. This lesson covers the selection biases that haunt voluntary programs. About 5 min
- Benchmark and Trend Assumptions When the counterfactual is a constructed number, the assumptions inside it decide the answer. This lesson shows how benchmark design can create or erase savings. About 5 min
Module quiz
Answer all questions to see your score.
1. A program enrolls last year's highest-cost patients, and their costs fall. Why is that not proof it worked?
Without a comparison group selected the same way, natural regression to the mean looks exactly like a program effect.
2. An ACO's average improves after several weak practices drop out. What does this most likely reflect?
When who leaves is related to performance, the surviving group's average shifts on its own, a selection effect, not a program effect.
3. Why can two analysts reach opposite conclusions about the same program?
When savings are measured against a constructed benchmark, its trend and baseline assumptions largely determine the result.