Domain Atlas / Behavioral-health & crisis triage
Kaiser Permanente Suicide-Risk Model
Work with this case in the PAN Lab ↗
Kaiser Permanente Northern California has embedded a machine-learning suicide-attempt risk model in the electronic health record of a large virtual mental-health program that handles more than 5,000 intake visits a month; the model is scored in near-real-time (about a 30-minute delay after an encounter trigger) and, at pre-set thresholds, flags high-risk patients to the intake clinician, routing them into the same suicide-risk-assessment and outreach workflow that a positive self-report screen (the PHQ-9 and Columbia-Suicide Severity Rating Scale) triggers, so the machine flag and the self-report alert are effectively OR-merged. In a study of 1,623,232 intake appointments (2012 to 2022, base rate 0.17 percent) the model reached an area under the ROC curve of 0.77 and its top risk decile captured 48.8 percent of appointments later followed by an attempt, but with a positive predictive value of about 0.8 percent.[3]
What happened
Kaiser Permanente Northern California embedded a machine-learning suicide-attempt risk model into the electronic health record of a large virtual mental-health program that handles more than 5,000 intake visits a month. The model is a penalized (LASSO) logistic regression that scores the probability of a suicide attempt in the 1 to 90 days after a mental-health intake appointment, computed from structured record features only — sociodemographics, mental-health diagnoses, psychiatric medications, medical comorbidities, service-utilization history, and self-reported suicidal ideation drawn from the PHQ-9 and Columbia-Suicide Severity Rating Scale screens. An internal predictive-analytics core computes the score in near-real-time, an approximately 30-minute delay gated to specific encounter triggers, and custom dashboards built within the electronic health record surface a flag to the intake clinician at pre-set thresholds. The defining design choice is integration rather than autonomy: the machine flag does not replace or gate care, it augments the pre-existing self-report alert, and because either channel triggers the same downstream workflow — a reminder to conduct a suicide-risk assessment at the upcoming visit, plus up to two additional outreach attempts if the patient does not attend — the two alert sources are effectively OR-merged. Intake was chosen as the point of care because, by the deploying team's own account, over 70 percent of treatment disengagement occurs after the first or second visit.
A large validation study put numbers on what the flag does and does not catch, restated independently in the trade press. Across 1,623,232 intake appointments from 835,616 patients (January 2012 to April 2022), 2,800 appointments (0.17 percent) were followed by a suicide attempt within the window and 78 of those attempts (3.9 percent) were fatal. The model reached an area under the ROC curve of 0.77; sensitivity was 37.2 percent at 95 percent specificity and 18.8 percent at 99 percent specificity; and the top 10 percent of appointments by risk captured 48.8 percent of appointments later followed by an attempt — but the positive predictive value in that top decile was only 0.8 percent. Against a 0.17 percent base rate, in other words, the large majority of flagged patients will not attempt in the window, so the flag imports a substantial false-positive and clinician-workload burden at scale, a caution the implementation team itself raised. Discrimination also varied across subgroups, with the area under the ROC curve ranging from 0.69 for one race group to 0.89 for another and higher for women (0.79) than men (0.74), which the team cited as an equity consideration.
What distinguishes this deployment is how carefully it was assembled for its stage. The implementation used a five-step community-engaged process — assess model validity including for demographic subgroups; interview patients with lived experience of suicide-related thoughts; gather clinician views; assemble the workflow with operational leaders; and educate and iteratively test — under institutional-review-board approval and a Delivery Science Grant, with the authors stressing that deployment must respect patient preferences, avoid reinforcing inequities, and manage workforce demands. It is also, importantly, a feasibility and design report, not an effectiveness study: it documents the model-augmented workflow and its rationale but presents no evaluation showing the deployment reduced suicide attempts. The peer-reviewed case study appeared in NEJM Catalyst Innovations in Care Delivery in February 2026, mirrored by an April 2025 medRxiv preprint that is not peer reviewed; a related Mental Health Research Network study across four systems found this class of model stayed stable in discrimination despite care-delivery changes. Two honest caveats travel with the case. The identification of the electronic health record as an Epic system reflects that Kaiser Permanente Northern California runs KP HealthConnect (an Epic system) and that the sources describe dashboards built within the record; the peer-reviewed text says electronic health record rather than naming a vendor. And no large-language-model or generative component is involved — this is a structured-data penalized-regression risk model, and conflating it with generative-AI tools overstates it.
The sociotechnical reading
Most tools in this Atlas point a score at a decision the model then dominates — screen in, allocate hours, deny a claim. Kaiser is the integration case: the score is deliberately wired as an additional, redundant sensor on top of a working human screen, OR-merged so that either the machine flag or the pre-existing self-report screen routes a patient into the same assessment-and-outreach workflow. That single design choice reframes the governance question away from accuracy and away from the override, and toward the workflow the sensor joins. Three properties follow, and each is a governable surface the map makes visible.
The first is that an OR-merge is additive by construction. A sensor merged with OR can raise the flagged volume but can never lower it, so what a low-PPV model contributes to a working screen is, at scale, mostly extra false positives: with a 0.17 percent base rate and a top-decile positive predictive value near 0.8 percent, more than 5,000 intakes a month generate a large volume of flags that the assessment step must absorb. The map's honest instruction is a tiering that respects real scarcity and never reads a clear flag as an all-clear — because by the arithmetic of a rare event, most attempts occur among the un-flagged. The second is that adding a machine sensor next to a human one quietly pulls on the human one. The clinician assessment is the safeguard, and the standing risk is deference drift: the assessment narrowing into "the model flagged this, so I will confirm it," the human step going ceremonial exactly where the score already reads some subgroups better than others. A live human loop is a safeguard only while it stays live, which is why the deskilling and vigilance material in the Field Guide attaches here even though — unusually for this Atlas — the loop was never broken. The third, and the defining absence, is that the two channels are summed, not reconciled. OR-merging discards the most informative event in the system: the case where the machine flag and the self-report screen disagree. There is no standing production check that compares the two channels, learns from their disagreement, or re-validates subgroup calibration as the population drifts; the genuine ex-ante subgroup validation was a launch answer, not a standing question, and no clinician acceptance, override, or workload rate was ever measured post-deployment. The lesson this case adds is that integrating a new sensor into a working workflow is a workload-and-deference problem before it is an accuracy problem: the careful ex-ante design — community engagement, subgroup checks, an advisory-only flag — is real and rare, but it governs the model's entry, not its continued life inside a workflow that sums rather than reconciles and that no one is watching absorb the load. The honest boundary throughout: none of this models suicide, and the patients being scored are not in the diagram — the false-positive burden, the outcome gap, and the subgroup differences are documented outside any system map, and a flag is an institutional signal, never a life.
The concepts used in this reading are defined in the Field Guide; the governance responses live in the Practice Library. The model organization for this case can be stress-tested in the PAN Lab.