Measuring What Matters

The previous module established that risk arrangements for behavioral health mostly did not measure whether patients improved. That is a choice rather than a limitation, because the tools exist and are brief.

The instruments work

Validated symptom instruments are short, free, well studied, and can be completed by the patient in a waiting room or on a phone. Depression and anxiety both have widely used measures with established thresholds for severity and for clinically meaningful change.

The collaborative care trials that produced the results in Module 2 used exactly this kind of measurement, defining response as a 50 percent or greater reduction in depressive symptoms from baseline. That definition is only possible because symptoms were measured repeatedly, in a comparable way, for every patient.

Why it still does not happen

Four reasons, in rough order of how often they are the real one:

• The score is not structured data. A questionnaire scanned into a document is invisible to any analytic system. The data course covered this: unstructured data is not data for these purposes.
• Nothing depends on it. If the score does not trigger a response, administering it is unpaid work, and unpaid work that changes nothing gets dropped.
• Nobody owns the denominator. Measuring outcomes requires knowing who should have been measured, which requires the registry from Module 2.
• Quality programs do not require it. Behavioral measures in most measure sets are process measures such as follow-up after hospitalization, which can be satisfied without knowing whether the patient improved.

Process, outcome, and the gap between

Measure type	Behavioral example	What it tells you
Structure	Whether a psychiatric consultant is contracted	Capability exists
Process	Follow-up visit within 7 days of discharge	Something happened
Outcome	Share of patients achieving symptom response at 6 months	Whether it worked

The introductory course drew this distinction generally. Behavioral health is where the gap between the second and third rows does the most damage, because a follow-up visit for a patient whose depression is unchanged satisfies the measure and does nothing for the patient.

Worth remembering: the argument that behavioral outcomes are too subjective to measure does not survive contact with the evidence. Trials measured them, found large effects, and published confidence intervals. What is actually true is that measuring them requires a workflow that produces no revenue and satisfies no current requirement, so it competes for attention against tasks that do both. That is a solvable problem and it is solved by making something depend on the number, either a payment, a required response, or a report someone will read. Until something depends on it, the instrument will be administered inconsistently and filed unused.

Key takeaways

• Brief validated instruments exist and were the basis of the collaborative care evidence.
• Scores must be structured data and must trigger a treatment response to constitute measurement.
• Outcome measurement requires a registry defining who should have been measured.
• Most behavioral quality measures are process measures satisfiable without patient improvement.

Sources

• Unützer et al., Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial, JAMA 2002 (opens in a new tab)
• Carlo et al., Association of Alternative Payment and Delivery Models With Outcomes for Mental Health and Substance Use Disorders: A Systematic Review, JAMA Network Open 2020 (opens in a new tab)