Domain Atlas / Behavioral-health & crisis triage
Crisis Text Line & Loris.ai
Work with this case in the PAN Lab ↗
Crisis Text Line, a national nonprofit crisis service, built an in-house machine-learning severity-triage model that reorders which texters volunteer counselors reach first; from about 2017 to 2020 the same anonymized crisis-conversation corpus was routed to Loris.ai, a for-profit spinoff CTL held an ownership stake in — reported by Politico-derived reporting at roughly 53% — which used it to train commercial customer-service software. After a January 28, 2022 Politico exposé, CTL ended the arrangement within three days and requested that the data be deleted; an FCC commissioner referred the matter to the FTC in March 2022, and no public FTC enforcement action is documented. CTL states the shared data was anonymized and never sold as personally identifiable information, and the exact number of records shared has not been made public.[4]
What happened
Crisis Text Line (CTL) is a national nonprofit that has offered free, 24/7 text-based crisis support since 2013. To route scarce counselor attention under heavy volume, its data-science team built an in-house machine-learning severity-triage system: CTL describes it as an ensemble of deep neural networks trained on about 65 million messages that scores the opening messages of a conversation for severity and reorders the counselor queue by predicted risk rather than first-in-first-out. CTL says the model identifies 86% of people at severe imminent risk in their first conversations and aims to serve 94% of high-risk texters in under five minutes — figures that are self-reported blog and marketing claims, never independently evaluated. The model allocates human attention; it does not reply to texters or dispatch emergency services. It is retrained through a human-in-the-loop feedback loop: volunteer Crisis Counselors tag conversations (for example, "Suicide") and rate risk in post-conversation surveys, and those labels are used to retrain the model. A separate escalation channel exists for imminent danger: a paid clinical supervisor can authorize an "active rescue," contacting a local 911 center using the texter's phone number and carrier. A 2020 CTL self-report put active rescues at about 0.82% of conversations — roughly 28 a day — with about 40% of imminent-risk conversations ending in a rescue and about 60% de-escalated without one; those are 2020 figures, and the voluntary-versus-involuntary split of the dispatches is not clearly documented.
The controversy that defines this case is not about the triage's accuracy — it is about what became of the data. In November 2017, CTL incorporated a for-profit spinoff, Loris.ai, co-founded by CTL co-founder Nancy Lublin. CTL held an ownership stake in Loris — reported by Politico-derived reporting at roughly 53% — together with a revenue-sharing agreement that, per that reporting, had not activated as of January 2022. Loris used the anonymized CTL crisis-conversation corpus to train commercial customer-service and de-escalation software; Loris's own website cited a "sentiment-rich" corpus of "62 million messages" drawn from CTL, a marketing figure rather than an audited count. The exact number of records shared has never been made public. Consent for the underlying data collection was obtained at the moment of acute crisis: an automated reply pointed texters to a lengthy Terms of Service — described by the Markkula Center as a "50-page agreement" and by the Reform Crisis Text Line campaign as a "4,000+ word" document — from users who, critics noted, include many minors. A former CTL board chair, danah boyd, who voted for the arrangement, later said that "knowing what I know now, I would not have"; a terminated volunteer, Tim Reierson, argued publicly that a Terms of Service accepted mid-crisis cannot be meaningful informed consent. An external data-ethics committee of academics and technologists reviewed research proposals — but was reportedly not looped in on the Loris data-sharing decision.
On January 28, 2022, Politico reported the arrangement, and public backlash followed quickly. Three days later, on January 31, 2022, CTL announced it had ended its data-sharing relationship with Loris and requested that Loris delete the data; CTL stated the shared data had been anonymized and scrubbed of personally identifiable information, was never sold as PII, and had not been accessed by Loris since the beginning of 2020. FCC Commissioner Brendan Carr publicly criticized the sharing and, in March 2022, wrote to FTC Chair Lina Khan urging the FTC to investigate CTL's practices for collecting, retaining, and sharing crisis-conversation data and how it obtains consent; CTL said it had engaged "in good faith" and that its practices comply with the law. No public FTC enforcement action is documented. In its 2025 response to a Journal of Medical Internet Research commentary, CEO Dena Trujillo said texters must consent to the privacy policy to use the service and can request deletion by texting the word DELETE, that Loris was required to return or delete previously accessed data, and that since 2023 in-house research has been overseen by an Institutional Review Board. CTL reported nearly 10 million conversations and about 300 million messages collected since 2013, with more than 71,000 trained volunteers as of early 2025, and said it supported more than 1.5 million conversations in 2025 — its highest-volume year. CTL continues to use machine-learning risk-triage; whether the identical 2018 ranker persists unchanged is not documented, and its error and override rates have never been published. The nearest peer-reviewed crisis-text triage error benchmarks — about 62% sensitivity and a ~38% false-negative rate on a neural model — come from SafeUT, a separate Utah/Idaho/Nevada service, and cannot be attributed to CTL.
The sociotechnical reading
Most cases in this Atlas turn on a model that is wrong, or a human loop that fails to catch it. Crisis Text Line is neither. The triage's accuracy is genuinely unaudited — but accuracy is not where the harm in this case lives, and the human loop around the model is one of the better ones in the collection: the ranker only reorders a queue, counselors keep full discretion within a conversation, and a supervisor exercises real clinical judgment before any emergency dispatch. This is the case that moves the governable surface off the model and the operator entirely, onto the data: its provenance, the consent it rests on, and who is allowed to connect to it.
Three things make the shape distinctive. The first is a fiduciary loyalty conflict. The most sensitive corpus a service could hold — millions of crisis conversations, self-described as the largest mental-health dataset in the world — was routed to a for-profit the nonprofit itself owned a stake in, with a revenue-sharing agreement attached. When the deploying organization profits from reusing the data it gathered, "should we reuse this?" is no longer a neutral question; a financial incentive answers it by default unless a governance structure is built to say no. The second is consent that cannot be given. The data was collected from people in acute crisis, many of them minors, via a link to a Terms of Service in the middle of a conversation about staying alive. A Terms of Service is a legal instrument; it is not informed consent from someone texting a crisis line, and the case is the clearest illustration in the Atlas of the difference. The third, and the reason the first two went unchecked, is a review body that existed but was bypassed. CTL had a data-ethics committee — and it reviewed research proposals while reportedly never being asked about the commercial arrangement. A check that is not invoked is not a check; the fastest-moving governance failure here (the relationship ended in three days once it was public) is precisely the one that had no standing gate to catch it beforehand.
Read on the system map, the lesson is that the levers are all on the data pathways, not the human loop. Connection authorization decides who may reach the corpus at all, before an incentive to reuse it can decide for you. A vendor gate is the audited data-processing agreement and deletion terms that a financially entangled recipient makes more, not less, necessary. An oversight cadence turns "we had a committee" into "the committee is asked, every time, before the data moves." Provenance labels carry with each record the consent it actually rests on — and the honest fact that "anonymized" was claimed rather than shown achieved. Structured dissent is what would have let the board member who had misgivings, or the volunteer who objected, force the question onto the table before an outside reporter did. Data minimization is the only lever that shrinks the asset rather than guarding it — and here the asset is the risk. Even purge, the after-the-fact deletion CTL reached for, teaches its own limit: you can request a copy back, but you cannot un-share what a partner has already trained on. The distinct lesson this case adds to the Atlas is that when the data itself is the product and the deployer has a stake in its reuse, the whole of governance moves upstream of both the model and the human — to consent at the door and authorization at the gate — and a governance body that is never asked is the same as no governance body at all. The honest boundary throughout: nothing here models suicide, crisis, or a texter, and none of the triage's self-reported accuracy claims are treated as established. A conversation, a score, or a rescue on this map is an institutional signal, never a person.
The concepts used in this reading are defined in the Field Guide; the governance responses live in the Practice Library. The model organization for this case can be stress-tested in the PAN Lab.