Evidence Registry

Michigan's MiDAS system auto-adjudicated unemployment-insurance fraud with an extremely high error rate among automated determinations, wrongly accusing tens of thousands of people; litigation and court action forced review and compensation.

Sources: michiganag2022, ieeespectruma, aiincidentdatabase, benefitstechadvocacyhubb

Appears on: /domains/cases/michigan-midas

The Royal Commission into the Robodebt Scheme documented hundreds of thousands of wrongful debts raised by an unlawful income-averaging method, with the onus placed on recipients to disprove automated assessments.

Sources: royalcommissionintotherobode2023a, royalcommissionintotherobode2023b, prygodiczvcommonwealthofaust2021, lawsocietyjournal, royalcommissionintotherobode

Appears on: /domains/cases/australia-robodebt

Indiana's privatized eligibility modernization produced over a million denials in its early years — many procedural rather than substantive — before the state canceled the contract and litigated with its vendor.

Sources: eubanks2018c, governmenttechnologyb, ieeespectrumb

Appears on: /domains/cases/indiana-ibm-eligibility

Independent scrutiny of Rotterdam's welfare-fraud risk model — a 2021 municipal audit followed by a 2023 journalistic investigation that obtained the model itself — documented scores skewed against already-vulnerable groups, and the city suspended the system's use.

Sources: rekenkamerrotterdam2021, lighthousereports2023c, wiredlighthousereports2023, followthemoney, racismandtechnologycenter2023

Appears on: /domains/cases/rotterdam-welfare-fraud, /pan-lab

A large share of Arkansas home-care recipients had care hours cut when algorithmic assessment replaced nurse judgment, and courts found due-process violations centered on the inability to understand or contest determinations.

Sources: arkansasdepartmentofhumanser2017, elderv2022, calo2021, universityofmichiganihpi, benefitstechadvocacyhuba, centerfordemocracytechnology, aiaaic

Appears on: /domains/cases/arkansas-archoices

Evaluation evidence on the Allegheny Family Screening Tool found that screener overrides of the tool's recommendations reduced racial disparity in screen-in rates relative to the tool alone.

Sources: vaithianathanetal2019, centreforsocialdataanalytics2019a, rittenhouse

Appears on: /domains/cases/allegheny-afst, /pan-lab

Illinois's Rapid Safety Feedback flagged thousands of children at 90-percent-or-higher risk of serious harm — beyond any caseload's capacity to act — while children who died in known-to-system cases had not been flagged; the agency ended its use in 2017.

Sources: chicagotribune2017, theimprint2017, governmenttechnologya

Appears on: /domains/cases/illinois-rapid-safety-feedback

Oregon's child-welfare agency dropped its AFST-derived Safety at Screening tool in 2022, citing equity concerns amid national scrutiny of racial disparity in child-welfare algorithms.

Sources: nprap2022, willametteweek2022

Appears on: /domains/cases/oregon-safety-at-screening

Documented benefit-automation failures replicated determinations into downstream systems with no independent reconciliation against the source records — Michigan MiDAS actioned replicated flags and Robodebt reversed the onus onto recipients.

Sources: michiganag2022, ieeespectruma, royalcommissionintotherobode2023b, lawsocietyjournal

Appears on: /pan-lab, /practice/reconcile-copied-records, /practice/record-reconciler

Documented risk-scoring deployments computed scores from multi-agency administrative records originally collected for other purposes, which is the data-protection critique recorded in independent reviews of these systems.

Sources: vaithianathanetal2019, eubanks2018c, lighthousereports2023c

Appears on: /pan-lab

Documented enforcement systems actioned replicated flags automatically — garnishment and penalties applied before any human review step in the recorded MiDAS deployment.

Sources: michiganag2022, benefitstechadvocacyhubb

Appears on: /pan-lab, /practice/reconcile-copied-records, /practice/record-reconciler

The Robodebt Royal Commission documented debts raised from income-averaged derived inputs with the onus placed on recipients to disprove the automated assessments.

Sources: royalcommissionintotherobode2023b, lawsocietyjournal

Appears on: /pan-lab

Ranked risk lists steered which cases were investigated in documented deployments; the anchoring direction is documented while its magnitude is not published.

Sources: lighthousereports2023c, wiredlighthousereports2023, amnestyinternational2021, vaithianathanetal2019

Appears on: /pan-lab

In the documented MiDAS case, error among no-review auto-adjudications ran roughly 93%, and determinations erred at about 85% without human review versus 44% with it.

Sources: michiganag2022, ieeespectruma, aiincidentdatabase, benefitstechadvocacyhubb

Appears on: /pan-lab, /domains/cases/michigan-midas, /practice/reconcile-copied-records

In the documented AFST evaluation, screener overrides of the tool — roughly a third of its recommendations — cut screen-in disparity from about 20% to 9% relative to the tool acting alone.

Sources: rittenhouse, vaithianathanetal2019, centreforsocialdataanalytics2019a, stapletonetal2022

Appears on: /pan-lab, /domains/cases/allegheny-afst

A peer-reviewed 2024 evaluation of the Allegheny Housing Assessment found that although the tool was substantially more accurate than the VI-SPDAT survey it replaced and produced similar risk-score distributions across race, it did not reduce the racial disparity in service rates: white single adults were served at about 23.3% versus 19.5% for Black clients.

Sources: cheng2024, alleghenycountydepartmentofh2026a

Appears on: /domains/cases/allegheny-housing-assessment, /pan-lab

After a 2025 update to the Allegheny Housing Assessment added a fourth outcome predicting future homelessness, the male share of assigned housing rose from 62% to 76% (and the female share fell from 34% to 24%), reflecting a higher measured one-year homelessness risk among men — an example of an outcome-selection choice reshaping who receives scarce housing.

Sources: alleghenycountydepartmentofh2026b

Appears on: /domains/cases/allegheny-housing-assessment, /pan-lab

The VI-SPDAT was the dominant U.S. homelessness triage assessment for roughly a decade, adopted in at least 39 states and the District of Columbia by 2015, before its own creators announced its phase-out in December 2020 on equity grounds; a 2019 commissioned racial-equity evaluation across four Continuums of Care found race predicted 11 of 16 subscales and that people of color received statistically significantly lower prioritization scores.

Sources: nationalalliancetoendhomeles2022, orgcodeconsultingiaindejong2020, cinnovationswilkey2019

Appears on: /domains/cases/vi-spdat, /pan-lab

The VI-SPDAT showed poor test-retest reliability, with most participants scoring higher on re-administration, and poor inter-rater reliability, with scores varying by interviewer and site; its predictive validity for housing outcomes was mixed across studies, positive for the youth version, null for single adults in one study, and positive in another community sample.

Sources: bitfocus2021, nationalalliancetoendhomeles2022, shinnandrichard2022

Appears on: /domains/cases/vi-spdat, /pan-lab

The U.S. Department of Veterans Affairs' REACH VET program has run a monthly suicide-risk model across the Veterans Health Administration since 2017, scoring about 6.28 million patients and flagging the top 0.1% at each facility (roughly 6,300 to 6,700 veterans a month, more than 130,000 since 2017); an independent re-analysis of 2018 data found the top-0.1% flag has a positive predictive value near 0.05% and a false-negative rate of about 98% for death by suicide, and a 2024 investigation reported that the model treated being a white man as a stronger risk signal than factors specific to women and excluded military sexual trauma and intimate-partner violence from its variables, a characterization VA has contested by framing the excluded factors as less predictive.

Sources: harris2025, glantz2024, graham2025, u2022

Appears on: /domains/cases/reach-vet, /pan-lab

Two Veterans Health Administration evaluations of REACH VET found the program associated with improved proximal outcomes — more completed outpatient appointments, more new safety plans, and fewer documented suicide attempts — but not with reduced death by suicide: a 2021 triple-differences study of 173,313 veterans across 141 facilities found no association with suicide or all-cause mortality, and a 2025 follow-up of 266,246 observations replicated the null with all confidence intervals crossing one; both are observational rather than randomized studies.

Sources: mccarthy2021, dent2025c

Appears on: /domains/cases/reach-vet, /pan-lab

Kaiser Permanente Northern California has embedded a machine-learning suicide-attempt risk model in the electronic health record of a large virtual mental-health program that handles more than 5,000 intake visits a month; the model is scored in near-real-time (about a 30-minute delay after an encounter trigger) and, at pre-set thresholds, flags high-risk patients to the intake clinician, routing them into the same suicide-risk-assessment and outreach workflow that a positive self-report screen (the PHQ-9 and Columbia-Suicide Severity Rating Scale) triggers, so the machine flag and the self-report alert are effectively OR-merged. In a study of 1,623,232 intake appointments (2012 to 2022, base rate 0.17 percent) the model reached an area under the ROC curve of 0.77 and its top risk decile captured 48.8 percent of appointments later followed by an attempt, but with a positive predictive value of about 0.8 percent.

Sources: hsin2026, hsin2025, papini2024

Appears on: /domains/cases/kaiser-epic-suicide-risk, /pan-lab

Because the near-term suicide-attempt base rate at Kaiser Permanente Northern California mental-health intake is very low (0.17 percent) and the positive predictive value in the top risk decile is about 0.8 percent, the large majority of flagged patients will not attempt suicide in the window, so adding the machine-learning flag as a redundant sensor OR-merged onto the existing self-report screen imports a substantial false-positive and clinician-workload burden at scale — a caution the implementation team itself raised. The implementation reports are feasibility- and design-focused and present no evaluation showing the deployment reduced suicide attempts.

Sources: papini2024, hsin2026, hsin2025

Appears on: /domains/cases/kaiser-epic-suicide-risk, /pan-lab

Crisis Text Line, a national nonprofit crisis service, built an in-house machine-learning severity-triage model that reorders which texters volunteer counselors reach first; from about 2017 to 2020 the same anonymized crisis-conversation corpus was routed to Loris.ai, a for-profit spinoff CTL held an ownership stake in — reported by Politico-derived reporting at roughly 53% — which used it to train commercial customer-service software. After a January 28, 2022 Politico exposé, CTL ended the arrangement within three days and requested that the data be deleted; an FCC commissioner referred the matter to the FTC in March 2022, and no public FTC enforcement action is documented. CTL states the shared data was anonymized and never sold as personally identifiable information, and the exact number of records shared has not been made public.

Sources: crisistextlinewikipedia2026, crisistextline2022, reierson2022, bentoninstituteforbroadbanda2022

Appears on: /domains/cases/crisis-text-line-loris, /pan-lab

Crisis Text Line obtained consent for its data collection through an automated reply directing texters to a lengthy Terms of Service — described as a roughly 50-page or 4,000-plus-word document — accepted at the moment of acute crisis by users who include many minors; critics including a former board chair, who voted for the data-sharing arrangement and later said she would not have "knowing what I know now," and a terminated volunteer argued that a Terms of Service is not meaningful informed consent for people in crisis. CTL says texters must consent to its privacy policy to use the service and can request deletion by texting the word DELETE, and that since 2023 its in-house research has been overseen by an Institutional Review Board.

Sources: markkulacenterforappliedethi2022, eysenbach2025, reierson2022, trujillo2025

Appears on: /domains/cases/crisis-text-line-loris, /pan-lab

NarxCare is a proprietary clinical-decision-support platform built by Bamboo Health that layers over state Prescription Drug Monitoring Programs and returns three Narx Scores plus a composite Overdose Risk Score (each 000-999) into the electronic health record, the PDMP portal, or pharmacy software, often in the patient header alongside vitals and allergies; adoption figures vary by what is counted (more than 40 states and territories run their PDMPs on Bamboo technology and five of the top six pharmacy chains use NarxCare, while the scoring module itself is switched on in more than 20 states). The vendor states the scores are intended to aid, not replace, clinical judgment and should never be sole justification for providing or refusing medication, but clinician and patient-advocacy sources document de facto determinative use — denials, forced tapers, and pharmacy refusals — driven by automation bias and fear of regulatory and criminal liability; patients cannot see, challenge, or correct their scores, the algorithm is proprietary and has not been independently validated for clinical care, and the FDA has not regulated it as a Software-as-a-Medical-Device, so contestation has instead run through FDA citizen petitions (one rejected on procedural grounds in 2023 and a second, docket FDA-2025-P-0701, pending since 2025 with more than 1,000 public comments).

Sources: bamboohealth2023, wang2026, millerandwhitehead2023, buonora2023, oliva2022, painnewsnetwork2023, medscape2025a, medscape2025b

Appears on: /domains/cases/narxcare, /pan-lab

On its own 2013-2016 training and validation data Bamboo Health reported an Overdose Risk Score precision of about 75% (self-reported, never independently reproduced), and its own external-validation set from 2017-2023 showed precision falling to about 52%, which the vendor attributed to rising illicit fentanyl (untracked by prescription-monitoring programs) and wider use of opioid-use-disorder treatment medication. A 2026 npj Digital Medicine study that reconstructed the model on California's CURES prescription database (about 17.9 million observations) and on commercial claims data obtained a precision of only 0.01 to 0.32 across several model architectures; because overdose-death labels were unavailable to the independent researchers, that reconstruction was trained on proxy outcomes rather than the score's actual overdose-death target, so it is best read as evidence that proprietary opacity prevents anyone outside the vendor from assessing the deployed model's accuracy, fairness, or safety, rather than as a strict like-for-like refutation of the vendor's figure.

Sources: bamboohealth2023, wang2026

Appears on: /domains/cases/narxcare, /pan-lab

Limbic Access, a Class IIa UKCA-certified self-referral and triage chatbot for NHS Talking Therapies, is deployed across a large and growing share of the service (its maker's chief executive claimed about 63% of the NHS in April 2026). Two peer-reviewed observational studies report large operational gains — a study of 129,400 self-referrers across 28 services found referrals rose 15% in chatbot services versus 6% in control services, and a study of 64,862 patients reported clinical-assessment time cut from 54.4 to 41.6 minutes and recovery rates of 58% versus 27.4% — but both studies are non-randomized and were authored by people employed by or holding shares in the tool's maker (all six authors of the access study and seven of the eight authors of the efficiency study), and the efficiency study's own authors caution that the recovery difference is subject to unmeasured confounding from self-selection. No randomized or independent third-party effect estimate has been published.

Sources: habicht2024, rollwage2023, chatterjee2026

Appears on: /domains/cases/limbic-access-nhs, /pan-lab

In the peer-reviewed study of 129,400 self-referrers across 28 NHS Talking Therapies services, self-referrals rose more where the chatbot was in use than in control services (15% versus 6%), with the largest increases among under-served groups — reported at about +179% for nonbinary people, +40% for Black and +39% for Asian self-referrers. This is an observational multi-site association, not a randomized causal effect.

Sources: habicht2024, heikkila2024

Appears on: /domains/cases/limbic-access-nhs, /pan-lab

Woebot, a rule-based (non-generative) cognitive behavioral therapy chatbot used by roughly 1.5 million people over its lifetime, was deliberately retired by its maker on a pre-announced schedule: the app was taken down on June 30, 2025, with a transcript-request window (deadline July 15, 2025) and all account data anonymized as of July 31, 2025, removing personally identifying information rather than silently abandoning the service. The founder and chief executive attributed the shutdown to the cost of meeting FDA marketing-authorization requirements and to a regulatory-pathway gap, framing the exit as economic and regulatory rather than a clinical failure - a self-reported account, not an independently audited finding. The roughly 1.5 million figure is a cumulative lifetime number reported in press coverage, not an audited point-in-time active-user count.

Sources: aguilar2025, woebothealth2025, hlth2025

Appears on: /domains/cases/woebot-shutdown, /pan-lab

Woebot's peer-reviewed efficacy record is a single early-stage study: a 2017 randomized controlled trial in JMIR Mental Health (n=70, ages 18 to 28, two weeks, unblinded, information-only control) reported a moderate between-groups reduction in PHQ-9 depression symptoms (about d = 0.44). That is an efficacy signal, not regulatory validation, and the study authors were affiliated with the tool's maker. A separate, investigational, prescription-only variant (WB001) received an FDA Breakthrough Device Designation in May 2021 - an expedited-review status, not marketing authorization - and entered a pivotal Software as a Medical Device trial with the first patient enrolled in January 2023, but never received FDA marketing authorization; it must not be conflated with the consumer app.

Sources: fitzpatrick2017, woebothealthbusinesswire2021b, woebothealthbusinesswire2023

Appears on: /domains/cases/woebot-shutdown, /pan-lab

DWP's own fairness assessment (covering 1 April 2024 to 31 March 2025) of its live Universal Credit Advances fraud-risk model reports statistically significant referral disparities and an accuracy inversion: relative to a 35-44 comparator, claimants aged 55-65 were about 2.80 times as likely to be referred for review and non-UK nationals about 2.27 times as likely, while for older claimants those referrals were less likely to be correct (relative correct-referral likelihoods of about 0.58 at 55-65 and 0.23 at 66-plus, the latter resting on a small sub-sample DWP flags to treat with caution). The disparities were first disclosed under freedom-of-information law and reported in December 2024, and DWP has committed to retrain the model. The figures are DWP-reported relative ratios, not independently audited absolute error rates.

Sources: departmentforworkandpensions2025b, theguardian2024

Appears on: /domains/cases/uk-dwp-fraud-ml, /pan-lab

DWP states that a human caseworker always makes the final decision on a referred Universal Credit advance with no automated decision-making, and is deliberately not shown the risk score or told the referral came from the model; DWP describes the model as around three times more effective than a randomised control at identifying fraud risk and judges continued operation reasonable and proportionate while committing to retrain it. The Public Law Project counters that only age was fully assessed among protected characteristics and that the assessment relied on safeguards preventing downstream harm rather than showing the model to be non-discriminatory. The wider counter-fraud programme is meanwhile expanding into bank-data eligibility verification under the Public Authorities (Fraud, Error and Recovery) Act 2025, a distinct system not yet in force.

Sources: centraldigitalanddataoffice2025, departmentforworkandpensions2025b, publiclawproject2025, departmentforworkandpensions2025a

Appears on: /domains/cases/uk-dwp-fraud-ml, /pan-lab

During the pandemic unemployment surge, a private facial-recognition identity check operated as a de facto eligibility gate for unemployment benefits in at least 25 U.S. state workforce agencies, with a live 'trusted referee' interview queue that House investigators documented averaging nearly 10 hours in North Dakota and over 4 hours in 14 of 21 states, versus about 6 minutes in New Jersey where an in-person option existed. Oregon's own one-month study (n=10,656 routed) recorded verification-completion differences by group -- for example 41.59% for African American and 34.48% for Spanish-language claimants versus 53.44% for White claimants -- but stated the study showed differences in completion and did not show causation, so these are a friction proxy, not a measured wrongful-denial rate. The U.S. Department of Labor does not collect or report the number of workers blocked for inability to verify identity, and where verification precedes filing those workers are not counted as denied claims at all, so the scale of any wrongful lockout is undocumented.

Sources: ushousecommitteeonoversighta2022a, stateoforegonemploymentdepar2022, nationalemploymentlawproject2023, usdepartmentoflabor2023

Appears on: /domains/cases/us-idme-unemployment, /pan-lab

A U.S. Department of Labor Inspector General audit (March 31, 2023) found that among 24 state workforce agencies using a facial-recognition identity contractor, 18 of 24 (75%) contracts did not specify one-to-one versus one-to-many matching, 15 of 24 (63%) did not address data storage, and 13 of 24 (54%) did not address destruction of the collected biometric data, while 22 of 24 (92%) agencies reported the technology reduced improper payments -- the operator-side benefit that sustained adoption even as the wrongful-lockout cost went unmeasured. The vendor initially represented it used only one-to-one matching and later acknowledged one-to-many matching against a database; after bipartisan backlash the IRS and Treasury dropped the mandatory facial-recognition requirement in February 2022 and the vendor made it optional across agencies, though the service remained in use for unemployment identity verification in a large share of states, and a 2026 IRS proposal would allow it to retain taxpayer biometric data up to 36 months after account deletion. The reported improper-payment reductions are agency self-reports, not independently audited.

Sources: usdepartmentoflabor2023, americancivillibertiesunionj2022, electronicfrontierfoundation2022, biometricupdate2026

Appears on: /domains/cases/us-idme-unemployment, /pan-lab

On August 30, 2023 CMS notified states that their automated Medicaid ex parte renewal systems were evaluating eligibility at the household or family level rather than the federally required individual level, so when any one household member could not be auto-renewed the whole household was dropped procedurally if a returned form was not received. CMS found 30 states had the defect and, on September 21, 2023, announced that nearly 500,000 children and other individuals who had been improperly disenrolled would regain coverage, requiring the affected states to pause procedural disenrollments, reinstate coverage, and reprogram to individual-level renewal. The ~500,000 figure is an aggregate of state-reported estimates compiled by CMS, not an independently audited count; children were disproportionately affected because their income-eligibility thresholds are higher than adults', and an HHS ASPE analysis (cited via Georgetown CCF) projected roughly 74% of disenrolled children would still be eligible, a projection rather than a post-hoc audit.

Sources: centersformedicareandmedicai2023a, centersformedicareandmedicai2023b, georgetownuniversitycenterfo2023a, healthcarediveemilyolsen2023

Appears on: /domains/cases/us-medicaid-unwinding-autorenewal

The Medicaid unwinding was governed by a federal monitor-and-respond loop: Section 5131 of the Consolidated Appropriations Act, 2023 (SSA section 1902(tt)), codified in a December 6, 2023 interim final rule, gave CMS mandatory monthly state reporting plus, for noncompliance, a Federal Medical Assistance Percentage reduction of 0.25% per quarter (capped at 1%), civil monetary penalties up to $100,000 per day for reporting failure, corrective action plans, and authority to order suspension of procedural disenrollments. Against that instrumentation the overall churn was large: KFF recorded about 25.2 million people disenrolled as of September 12, 2024 with 69% of disenrollments for procedural rather than eligibility reasons, while a June 24, 2025 GAO audit independently found about 27 million disenrolled in the first 18 months, roughly one-third of those continuously enrolled. The KFF and GAO totals differ because they cover different windows and use different data and methods, not because they conflict; the enforcement penalty details are drawn from the interim final rule and a legal-analysis summary.

Sources: federalregister2023, morganlewisandbockiusllp2023, kff2024, usgovernmentaccountabilityof2025

Appears on: /domains/cases/us-medicaid-unwinding-autorenewal

A 2024 Tribunal de Contas da Uniao (TCU) plenary audit found that INSS benefit denials were nonconforming above the maximum acceptable limit in both channels it sampled: 10.94% of automatically analyzed denials (January to May 2024) and 13.20% of manually analyzed denials (2023 sample), in Acordao 634/2025-Plenario (process TC 008.309/2024-8, session 26 March 2025). Nonconformity ('desconformidade') is a TCU audit-analysis category that includes wrongful denials but is not identical to a court-confirmed wrong-denial rate, so these are not a hard error rate; the absolute counts reported in coverage (about 920,000 automatic denials in the audited window, about 100,000 estimated wrongful, and 250,000 to 290,000 estimated unjustified manual denials) are journalistic extrapolations from the TCU percentages, not officially published counts. Neither channel uses a machine-learning or predictive risk score; 'automatic' means rules-based administrative processing and documentary-conformity analysis.

Sources: tribunaldecontasdauniao2025, infomoney2025, consultorjuridico2025

Appears on: /domains/cases/brazil-inss-automation

The TCU root-cause finding was that INSS measures server productivity by the number of processes analyzed rather than the quality of the decision's justification, creating an incentive to choose denial as the fastest disposition, with no incentive for correct motivation of the denial and no effective communication with the insured. The correction channel is slow and external: the CNJ recorded 5,109,076 pending previdenciario lawsuits as of 31 October 2024, and CNJ 'Justica em Numeros' data put the average pending-case duration at about 746 days with a conciliation rate near 24.84%, so a fast automated or manual denial is reversed only after a roughly two-year judicial wait. The 5.1-million-case backlog and the 746-day duration are CNJ caseload figures and cannot be mechanically attributed to automated denials specifically, because the public data do not link an individual court reversal to the channel that produced the denial.

Sources: tribunaldecontasdauniao2025, consultorjuridico2024, conselhonacionaldejustica2024

Appears on: /domains/cases/brazil-inss-automation

The Commonwealth Ombudsman's first report, Automation in the Targeted Compliance Framework (published 6 August 2025), found that the Department of Employment and Workplace Relations and Services Australia acted contrary to the law and unlawfully cancelled the payments of 1,009 jobseekers under the predominantly automated Targeted Compliance Framework, with a further 45 auto-cancelled after a pause was ordered (the first-cohort figure is variously reported as 'more than 900', 964, or 1,009; 1,009 is the most precise and most widely cited). The unlawfulness was an omission: the April 2022 SPROM Act required a discretionary reasonable-excuse consideration before a cancellation and required a mandated automated-decision safeguard, the Digital Protection Framework, neither of which was implemented, so cancellations executed without the check the law required. The defect operated from April 2022, was detected in September 2023 by external legal advisors, and cancellations were not paused until July 2024 — a roughly ten-month gap the Ombudsman called not acceptable; both agencies accepted all seven recommendations. A commissioned Deloitte assurance review separately found the IT system increasingly unstable, with five IT errors dating to 2018, and could not assure the integrity, effectiveness, or appropriateness of decisions.

Sources: commonwealthombudsman2025a, informationageaustraliancomp2025, itnews2025, theexamineraustralianassocia2025, departmentofemploymentandwor2025a

Appears on: /domains/cases/australia-workforce-tcf

The Targeted Compliance Framework operates at very large scale: advocacy and analysis of departmental data describe roughly 2.5 million payment-suspension notices a year to about a million people, with 200,000 to 240,000 people facing suspension threats each quarter (these system-scale figures are directionally consistent across sources but exact denominators and periods vary). The Ombudsman's second report, Fairness in the Targeted Compliance Framework (published 9 December 2025), found that automatic Penalty-Zone suspensions undermine a jobseeker's ability to challenge penalties, that the department's assessment of provider performance lacks transparency, and that a high rate of provider decisions are overturned on review, while the complaints line went more than 140,000 calls unanswered between November 2024 and September 2025. A separate and far larger section 42AM automated-cancellation review is still expanding: the department previously published up to 9,510 unlawful cancellations or reductions, an advocacy estimate put potential exposure near 310,000, and in June 2026 Senate estimates a department official said the number was in the vicinity of that estimate but qualified that 55 to 70 percent may have legitimately lost eligibility, implying roughly 93,000 or potentially 100,000-plus. Those larger figures are estimates of potentially unlawful cases pending case-by-case assessment, not confirmed cancellations.

Sources: theantipovertycentre2026, powertopersuade2025, commonwealthombudsman2025b, sbsnews2025

Appears on: /domains/cases/australia-workforce-tcf

New York City launched the MyCity Business chatbot in 2023 on Microsoft Azure AI as a public-facing generative-AI adviser for business owners. A March 29, 2024 investigation by The Markup with THE CITY and Documented NY found it confidently and repeatably wrong on legal obligations, advising businesses in ways that would break the law, including that employers could take a cut of workers' tips, that landlords need not accept Section 8 vouchers or source-of-income tenants (illegal in New York City), that stores could go cashless against a 2020 city law, and that funeral-price disclosure could be concealed against the federal funeral rule; when ten staffers asked the housing-voucher question they received the same wrong answer, which had changed from an earlier correct one, showing the tool was non-deterministic. The 2024 findings are qualitative, based on specific tested questions rather than a sampled error rate. The city relabeled the tool a beta product with a disclaimer and applied a scope-narrowing patch rather than withdrawing it, kept it online for roughly two years, and shut it down in early 2026 as a budget cut rather than an accuracy fix.

Sources: themarkup2024, themarkupandthecity2024, reutersjonathanallen2024, themarkupcolinlecherandkatie2026

Appears on: /domains/cases/nyc-mycity-chatbot

A December 30, 2025 performance audit of the MyCity system, issued under New York City Comptroller Brad Lander, found the chatbot 'appears to be unable to provide accurate or consistent information' and reported that the wider MyCity system had cost over 100 million dollars across more than 120 agreements with about 50 vendors, lacked a system development plan, and had not delivered the promised single-form access to city benefits; the Office of Technology and Innovation disagreed with all seven of the audit's recommendations, including one to conduct AI red-teaming. Among the audit's figures, an internal weekly production report reproduced in the audit showed the chatbot did not answer 23 of 48 tested government questions, and of the more than 2,200 questions asked in July and August 2025 the 70 users who left thumbs-up-or-down feedback were 71.4 percent negative (50 of 70), a share the city disputes as roughly 2.25 percent of all responses, with the audit rebutting that denominator. The 100-million-dollar figure is the whole MyCity system, not the chatbot alone.

Sources: officeofthenewyorkcitycomptr2025

Appears on: /domains/cases/nyc-mycity-chatbot

Nevada's Department of Employment, Training and Rehabilitation contracted Google to build a generative-AI tool on the Vertex AI Studio cloud platform that reads an unemployment-appeal hearing transcript and evidence, retrieves against a corpus of Nevada unemployment law and prior appeals decisions, and drafts a recommended determination (approve, deny, or modify a claim) together with the written decision for a human referee to review and sign. The contract set a 90 percent success requirement self-assessed by state workers on test decisions -- not an independent external audit -- and DETR said it wanted accuracy higher than 90 percent before going live; rollout was repeatedly delayed over less-than-desired accuracy, including the tool citing incorrect Nevada statutes and failing to pull information from all hearing documents, problems officials said were fixed. Reported cost evolved from about 1 million dollars in 2024 to a total of 2.6 million dollars with about 1.1 million spent by early 2026. As of the most recent available reporting (March 2026) the system was in delayed pre-deployment testing on historical appeals and described as launching in coming weeks; it was not independently confirmed to be adjudicating live claimant appeals.

Sources: thenevadaindependentericneug2026, themarkuptoddfeathers2024, thenevadaindependentericneug2024, fordhamintellectualproperty2024

Appears on: /domains/cases/nevada-detr-genai-appeals, /pan-lab

Nevada's generative-AI unemployment-appeals tool was justified as a speed measure for a pandemic-era backlog, projecting a drop in referee determination time from as much as several hours to about five minutes per case, with a mandatory human review DETR said adds an estimated 10 to 30 minutes and a required referee sign-off (Director Christopher Sewell said no AI-drafted written decisions issue without human review). Legal scholars, attorneys who represent claimants, and a former U.S. Department of Labor official warned that backlog and speed pressure could hollow out that review and create incentives to rubber-stamp AI outputs -- one attorney noting the time savings only happens if the review is very cursory, and a legal analysis warning staff might feel pressured to authorize AI decisions with haste. That automation-deference risk is expert-projected, not a measured outcome: no referee override or rejection rate has been published, and claimants are not required to consent to AI processing of their appeal.

Sources: themarkuptoddfeathers2024, fordhamintellectualproperty2024, thenevadaindependentericneug2026

Appears on: /domains/cases/nevada-detr-genai-appeals, /pan-lab

The Government Digital Service ran a cross-government experiment with Microsoft 365 Copilot from September 30 to December 31, 2024, with about 20,000 employees across 12 organisations, and published the findings report on June 2, 2025. Participants self-reported saving an average of about 26 minutes per working day (the report extrapolates this to roughly 13 days a year from the median values of six reported time-savings ranges; independent coverage recomputed it to about 4.6 days on a 253-working-day basis), 17% reported no clear savings, adoption held near 80% after peaking at about 83%, and 82% said they would not want to return to working without it. The experiment measured adoption and self-reported time rather than output quality: the report recorded no audited error rate, flagged significant accuracy concern for low-verifiability tasks such as grievance handling and performance evaluations, noted external web data was used without built-in verification, and documented a provenance failure in which the tool struggled to identify which documents generated a response.

Sources: governmentdigitalservicedsit2025, governmentdigitalservice2025b, theregisterthomasclaburn2025

Appears on: /domains/cases/gds-m365-copilot-experiment

A companion Department for Business and Trade evaluation of Microsoft 365 Copilot (1,000 licences, October to December 2024; published August 28, 2025) reported 72% user satisfaction but concluded it did not find robust evidence that time savings were leading to improved productivity; in observed tasks its users completed spreadsheet data analysis more slowly and to worse quality and accuracy than non-users, and produced presentation slides over 7 minutes faster on average but to worse quality and accuracy that then needed correction. In its diary study, 22% of respondents said they had identified hallucinations, 43% detected none, and 11% were unsure, with a further roughly one in five not answering, so the figure reflects user-detected hallucination rather than audited incidence. A Department for Work and Pensions evaluation (3,549 licences; published January 29, 2026) measured 19 minutes a day saved across eight routine tasks against a comparison group (95% confidence interval 17 to 22 minutes), found 85% rating meeting-note accuracy good or very good, reported that users consistently reviewed outputs before use, and concluded the tool is complementary to human expertise and requires consistent human oversight.

Sources: departmentforbusinessandtrad2025b, theregisterpaulkunert2025, departmentforworkandpensions2026

Appears on: /domains/cases/gds-m365-copilot-experiment

According to its Algorithmic Transparency Recording Standard record, published on November 27, 2025, the UK Department for Work and Pensions runs a Whitemail Insights and Vulnerability Scanner that reads roughly 25,000 scanned documents a day (reported as around 22,000 a day at end-2023 and in a March 2024 operator interview). Each document is passed first through the Vulnerability Scanner, a pre-trained open-source transformer doing zero-shot classification, which flags potentially vulnerable customers against eight prescribed themes including suicide and self-harm, domestic violence and abuse, and financial hardship; only documents not flagged as indicating vulnerability are relayed to Whitemail Insights for routing across nine themes. The output to trained staff is an anonymised daily report of flagged customers, and DWP states the tool does not make or influence benefit entitlement decisions. The record names precision, recall and F1-score as its evaluation metrics but discloses no values, and no independent accuracy evaluation has been published.

Sources: departmentforworkandpensions2025a, trendall2025, ukparliamentworkandpensionsc2023, corbridge2024a

Appears on: /domains/cases/dwp-whitemail-scanner

Guardian FOI reporting in January 2025 recorded that benefit claimants are not told the AI reads their correspondence: the internal data protection impact assessment stated that letter writers do not need to know about their involvement in the initiative, and the tool had been piloted since at least 2023 without appearing on the central government AI transparency register despite a ministerial mandate. The correspondence it processes can include national insurance numbers, health information, bank details, and children's details. Turn2us policy manager Meagan Levin voiced serious concerns, noting that prioritising some cases inevitably deprioritises others, so it is vital to understand how these decisions are made and ensure they are fair. The further reading that a missed flag on the unflagged residual therefore has no complaint channel and surfaces only as downstream harm is an analytical inference from the documented non-notification and shortlist design, not an adjudicated harm.

Sources: booth2025, toth2025, dent2025a

Appears on: /domains/cases/dwp-whitemail-scanner

In the UK Home Office's own pilot of an AI tool that summarises asylum interview transcripts for decision-makers, 9% of the generated summaries were deemed inaccurate or incomplete and removed by a pre-use filter before any caseworker saw them, and 23% of users reported not being fully confident in the rest; the summaries carried no source references back to the transcript. The official evaluation, published April 29, 2025, measured a 23-minute-per-case time saving (a 32% reduction) for the summariser and about 37 minutes for a companion policy-search tool, and Home Office Calibre quality-assurance reviews found no statistically significant difference in decision quality on small pilot samples. The evaluation recommended addressing the identified limitations before a full rollout, continuous monitoring in early rollout, and a larger-scale evaluation after deployment; the Home Office announced expansion the same day. By January 2026 the policy-search tool had been rolled out to all asylum decision-makers, and per trade-press reporting the summarisation tool entered national rollout in April 2026.

Sources: ukhomeofficegovuk2025, openrightsgroup2026c, governmenttransformation2026

Appears on: /domains/cases/home-office-asylum-summarisation

The Home Office's asylum interview-summarisation tool inserts a compression step whose measured value is a 23-minute-per-case time saving that exists only insofar as the decision-maker does not redo the reading the summary replaced: caseworkers are not required to verify summaries against transcripts, and the pilot summaries carried no source references that would make checking cheap. The correction loop is also severed from the other side. In a May 2026 written parliamentary answer, minister Alex Norris confirmed that asylum claimants are not told about the AI tools used in their cases, so the one party with first-hand knowledge of their own account cannot surface a summary error; this postdates Article 22C of UK GDPR (in force February 5, 2026). As of mid-2026 the rollout had proceeded without a published post-deployment evaluation or continuous-monitoring data and, per Open Rights Group, without a published Data Protection Impact Assessment, Equality Impact Assessment, or Algorithmic Transparency Recording Standard entry, with prompts withheld under a Freedom of Information refusal. A March 16, 2026 commissioned legal opinion argues the use is likely unlawful on procedural-fairness and data-protection grounds; that is a contested legal position, not a court ruling.

Sources: ukhomeofficegovuk2025, resultsense2026, openrightsgroup2026a, openrightsgroup2026b

Appears on: /domains/cases/home-office-asylum-summarisation

In February 2026 the Superior Court of Los Angeles County, the largest trial court in the United States, began a pilot of the Learned Hand AI drafting workbench with six civil-division judges and their research attorneys under a contract of about $314,000 running into early 2027, and the Superior Court of Riverside County gave seven civil and probate research attorneys access under a separate $10,000 agreement used for research memos; the tool ingests case filings, synthesizes applicable law, and drafts proposed orders in the individual judge's own writing style. Under California Judicial Council Rule 10.430 (effective September 1, 2025, the first statewide court generative-AI framework in the nation), disclosure is required only when a document consists entirely of generative-AI output, and the rule reaches judicial officers only for tasks outside their adjudicative role, so neither court is obligated to tell litigants when AI assisted with an order or memo in their case; both courts declined to confirm whether litigants whose cases are used in testing are informed.

Sources: mihalovichandjohnson2026, queally2026, judicialcouncilofcalifornia2025

Appears on: /domains/cases/learned-hand-la-courts

In the Learned Hand pilot the only reported error-correction safeguard is the judge's own review: officers are required to review and edit each draft before adopting a tentative ruling, and a court spokesman said the assistance does not supplant the judicial officer's independent role. No external audit, query logging, or benchmarking regime was reported (legal analysis coverage drew a contrast with Michigan's approach), and no error, edit, or override rate for the tool has been published. The Los Angeles District Attorney raised an anchoring concern, that an AI-generated draft could greatly influence what the judge's position should be before an independent view forms; this is an attributed critique rather than a measured effect, and the vendor's per-sentence Deep Verify hyperlinking and multiple-verification-passes claims are unverified vendor statements.

Sources: queally2026, howell2026, mihalovichandjohnson2026, learnedhandandsuperiorcourto2026

Appears on: /domains/cases/learned-hand-la-courts

The UK Government Digital Service ran what it called the government's biggest public test of generative AI to date: across two gated public pilots (a late-2024 web pilot of 10,136 users asking 23,838 questions, and an autumn-2025 GOV.UK app pilot of 641 users asking 2,670 questions in four weeks), more than 10,000 people asked GOV.UK Chat about 26,000 questions on tax, benefits and visas. Its first 2023 version was held back in findings published January 18, 2024 because, GDS reported, answers did not reach the highest level of accuracy demanded for a site like GOV.UK, including a few cases of hallucination. GDS reports measured answer accuracy rising from 76 percent (its earliest benchmark) to 90 percent by the autumn 2025 pilot, assessed by subject-matter experts plus automated evaluation, an 88 percent answer rate for in-scope questions after a clarifying-questions feature was added, and that 508 attempts to jailbreak the system across the pilots were all prevented by its guardrails; it soft-launched to all GOV.UK app users on March 26, 2026 and officially launched on May 14, 2026. Nearly every one of these figures is self-reported by GDS, the system's operator, and the accuracy denominators and sampling frames are unpublished.

Sources: governmentdigitalserviceinsi2026, governmentdigitalserviceinsi2024a, governmentdigitalservice2026

Appears on: /domains/cases/govuk-chat

GOV.UK Chat is a retrieval-augmented assistant that, per its Algorithmic Transparency Record published October 7, 2025, answers only from roughly 700,000 vectorised chunks (36.9 GB) of curated official GOV.UK guidance, is instructed to ignore its training data, rejects questions containing phone numbers, emails or card numbers, links every answer back to its GOV.UK source pages with a reminder to verify, and retains question data encrypted for 12 months; GDS states it does not attempt to provide advice and makes no automated decision. GDS's December 2025 vision post frames a content-dependency loop, stating that GOV.UK Chat can only be as good as the content published on GOV.UK by departmental teams. The record's independent evaluation is a jailbreak (security) assessment conducted with the AI Security Institute, alongside the record's own caveat that it is not possible to guarantee no jailbreaking attempts will succeed; there is no independent audit of the accuracy methodology, and GDS's claim that for government-related questions the tool scores higher than widely-used consumer AI assistants is the operator's own comparison.

Sources: departmentforscience2025b, governmentdigitalserviceinsi2025, civilserviceworldjimdunton2026

Appears on: /domains/cases/govuk-chat

Frida is the chatbot at the front line of the Norwegian Labour and Welfare Administration's (NAV) anonymous contact-center chat channel; NAV states it launched in summer 2018 and, as of 2026, that citizens first meet Frida (open 24 hours a day) and can ask it for a human advisor on weekdays between 9:00 and 15:00, with the channel anonymous and no personal information visible to NAV. During the COVID-19 lockdown NAV reported a roughly 250 percent surge in inquiries; the platform vendor's case study reports the chatbot answered more than 270,000 coronavirus-related inquiries and that about 80 percent of enquiries were resolved without escalating to a human, and NAV's own funded research report records nearly 11,000 inquiries in Frida on some days between March and May 2020 with a week-13-2020 peak equal to the capacity of about 230 human advisors, where the vendor and the peer-reviewed EJIS study state about 220. These pandemic figures originate substantially in the vendor's marketing case study and are reported here as vendor claims with the 220-versus-230 source tension left unresolved; the roughly 80 percent containment is a completion or non-escalation rate, not a measure of answer accuracy.

Sources: boost2020, parmiggiani2021, vassilakopoulou2022a, nav2026

Appears on: /domains/cases/frida-nav-norway

The best-documented property of NAV's Frida chatbot is its chatbot-to-human handover boundary, which the evidence suggests behaves as a governance-controlled dial: NAV's funded three-university Frida@work project reports that about one in five conversations transferred to a live human advisor under free channel choice, and only about 30 percent of dialogues transferred when NAV removed the explicit choice between the chatbot and human chat, a regime-specific figure that must be read against the interface in force. Independent chat-log studies document irrelevant answers, omitted information, and three classes of domain-knowledge failure, with the most critical failures occurring when a misunderstanding goes undetected inside a conversation the chatbot completed; no per-answer accuracy or error rate has been published, and the Frida@work project found context survives the handover imperfectly, with citizens often unsure whether they are talking to a person or a machine. NAV's own 2025 channel-use analysis found that chatbot visibility appears not to change contact-center inquiry volumes and attributes the steady post-2019 decline to a bundle of causes (self-service improvements, new application systems, SMS notifications and changed contact-center practices), so the chatbot is not shown to reduce human workload outside the crisis peak.

Sources: parmiggiani2021, verne2022, simonsen2020, mcvey2025

Appears on: /domains/cases/frida-nav-norway

Burokratt is Estonia's national network of public-sector chatbots operated by the Information System Authority: each participating institution runs its own assistant, a central classifier routes a citizen's query between them and oversees the handover, and from 2025 a shared knowledge module built from the eesti.ee state portal feeds cross-domain answers. RIA's page lists 20 participating organisations and trade press reports 18 integrated; an independent 2025 ethnography drawing on twelve insider interviews (conducted in late 2023, when the system spanned ten institutions) found it marketed as advanced AI while functioning much like an FAQ list, with use differing considerably by institution and low in some. No published session volumes, escalation-to-human rates, or answer-accuracy figures, and no dedicated algorithmic-oversight body, published evaluation framework, or national-audit report on the network, were located in the public record.

Sources: informationsystemauthorityri2025a, govinsider2025, kaun2025

Appears on: /domains/cases/burokratt-estonia

A 2025 survey-vignette experiment in Estonia reported that citizens' intended use of a government chatbot relates to perceived usefulness and trust in the technology, that privacy concerns relate to service-provision uses but not to information-provision uses, and that trust in government, explainability, and the amount of information provided were not related to intended use.

Sources: alishani2025

Appears on: /domains/cases/burokratt-estonia

Albert France Services was a sovereign, in-house generative AI assistant built by DINUM with ANCT to help France Services counter advisers answer citizens' benefits and procedure questions from a curated base of official documents, presented by the Prime Minister as a sovereign French AI in April 2024 and, in a demonstration before him, giving a wrong answer on identity-card cost. Piloted from an initial panel of about sixty volunteer advisers to roughly eighty advisers across more than forty counters (forty-eight at final count per AFP) in six departments over three iterated versions, it was, per a January 12, 2026 AFP dispatch, formally not going to be generalized 'in its current form,' a decision DINUM announced on January 9, 2026 while stating that the majority of Albert-brand projects are sustained and fully operational. No error rate, usage volume or override count for the tool was ever published; AFP reports DINUM's annual AI budget at about 1.2 million euros since 2024 with Albert France Services a minimal share, a figure distinct from and not directly comparable to the union Solidaires Finances Publiques' separate claim of a roughly 1.3 million euro project cost.

Sources: wekafrafpdispatch2026, acteurspublics2026, solidairesfinancespubliques2026, franceservicesanct2024

Appears on: /domains/cases/albert-france-services

For Albert France Services no instrumented error-detection channel existed: no error rate, override count or usage figure was published during the pilot, and the failures that framed the tool surfaced through the operator side, with several unions documenting recurring malfunctions and wrong answers, an investigative-television broadcast in April 2025 (per Solidaires Finances Publiques) featuring unenthusiastic agent testimony, and advisers reporting answers worse than an ordinary search. According to Solidaires Finances Publiques the project had in fact stopped by September 2025 with no announcement, inferred from Albert no longer appearing among projects presented in a ministerial working group (a union claim). Alongside the January 2026 non-generalization decision DINUM migrated the Albert API model aliases off the 'albert-' branding and removed the web-search functionality, retiring legacy aliases by February 15, 2026, while a successor adviser tool that integrates models from the vendor Mistral AI was in test with about 10,000 public agents through June 2026, gated by a summer-2026 evaluation that must notably establish the cost of a generalization.

Sources: solidairesfinancespubliques2026, nextnextink2026, wekafrafpdispatch2026, acteurspublics2026

Appears on: /domains/cases/albert-france-services

In the published runs, over a supervised-plus-agent scenario, adding a verifier to the autonomous agent removed roughly 46% of the harm that persists and a coordinated governance package roughly 43%, while upgrading the model alone removed only about 6%.

PAN Lab model result: PAN social-work governance guidance, lever-ranking comparison.

Appears on: /practice/verifier-on-the-agent, /practice/improve-the-model, /pan-lab

In the published runs, fixing the surrounding system out-leveraged an equal-effort model upgrade in nearly every case tested, and by several times the margin - a better model helps least where the system, not the model, does the damage.

PAN Lab model result: PAN baseline analysis.

Appears on: /practice/improve-the-model, /pan-lab

In the published runs, deleting records without reading them raised the contaminated share by stripping out benign entries; only content-aware cleanup reliably reduced it.

PAN Lab model result: PAN governance-lever audit.

Appears on: /practice/connection-authorization, /pan-lab, /practice/data-minimization, /practice/content-aware-decontamination, /practice/record-reconciler

In the published runs, the same AI in three modeled office cultures - stylized, not real workplaces - let errors stick at very different rates: roughly 75% under low-oversight autonomy, 20% under human supervision, and 16% under high-governance professional controls.

PAN Lab model result: PAN social-work governance guidance, three-office comparison.

Appears on: /pan-lab

In an independent validation — against the NICE Evidence Standards Framework — of a Magic Notes documentation-assistance pilot at Kent County Council adult social care, staff self-reported weekly written-admin time falling roughly 6.8-7.2 hours (about 35-41%), records submitted some 2.0-3.5 days sooner, and case-note detail rated 6.2 to 8.7 out of 10; the validator judged the findings directionally valid rather than a productivity measurement, because the study was commissioned by the vendor (Beam) — which collected and analysed the data while the validator only sense-checked it — and rested on 29 opt-in staff over 8 weeks with self-estimated time, no control group, no statistical testing, and safety and accuracy explicitly out of scope.

Sources: unityinsights2025, beam, somersetcouncil

Appears on: /pan-lab

In the same independent validation of the Kent County Council Magic Notes documentation-assistance pilot, the report records the deskilling concern that runs alongside the benefit: one client declined having their session recorded "due to personal feelings of risk of loss of practitioner skills" (p.19). It is a single qualitative observation from a vendor-commissioned pilot of 29 opt-in staff over 8 weeks with no control group — evidence for the direction of the crutch/deskilling risk that accompanies documentation assistance, not for its magnitude.

Sources: unityinsights2025

Appears on: /pan-lab

In a peer-reviewed staggered-deployment study of 5,172 customer-support agents at a single firm, access to a generative-AI assistant raised issues resolved per hour by about 15% on average, with the gain concentrated in the least-experienced workers — roughly +30% for novices versus near-zero for the most experienced, who showed small quality declines; the widely cited 14%/34% pair comes from the 2023 draft, while the peer-reviewed figures are 15%/30%, and because the domain is customer support the direction is imported to social services but the magnitude is never treated as a fixed quantity.

Sources: brynjolfsson2025

Appears on: /pan-lab

In a randomized controlled trial of a benefits-navigation chatbot (co-authored by Cornell researchers and the tool's developer, Nava) with 125 caseworkers across six Los Angeles County organizations over 14 weeks, caseworkers answered complex benefit questions at about 49% accuracy unaided, and high-quality chatbot suggestions raised accuracy by roughly 27 percentage points — with larger gains on harder questions, but a persistent 'AI underreliance' plateau in which correct suggestions were not always adopted; the trial did not establish a clear effect on administrative burden, a null reported honestly rather than inferred as a benefit.

Sources: gosciak2026, kanne2025, navapublicbenefitcorporation2025d, navapublicbenefitcorporation2026

Appears on: /pan-lab

In the county-commissioned impact evaluation of the Allegheny Family Screening Tool, screen-in accuracy — further action or re-referral within 60 days — rose from 42.85% to 46.61% (p=.000) while consistency across screeners was maintained; the finding is contested and quasi-experimental, with true maltreatment rates unknown and the accuracy gains concentrated among white children and ages 7-12, the gain for Black children attenuating to statistical non-significance.

Sources: vaithianathanetal2019

Appears on: /pan-lab

In a national survey of 1,179 U.S.-based social workers conducted from October 2025 to February 2026 by the University of Texas at Austin in collaboration with NASW, 63.5% of respondents reported using AI tools or technologies in their current role.

Sources: borah2026a, isbanner2022

Appears on: /pan-lab, /practice/ai-literacy

In the same national survey, concerns about data privacy and security were the most frequently reported challenge to using AI in practice (46.5% of respondents), and an increased focus on client privacy and confidentiality was the most requested improvement to AI tools for social work (50.4%).

Sources: borah2026a, isbanner2022

Appears on: /pan-lab, /practice/data-minimization

In the same national survey, 40.8% of respondents reported ethical concerns about relying on AI for decision-making, and overreliance on automated decision-making was among the most frequently cited concerns overall.

Sources: borah2026a, pinazohernandis2026

Appears on: /pan-lab

The same national survey describes a gap between AI exposure and AI preparedness: 26.6% of respondents cited lack of training or understanding of AI technology as a challenge, 53.4% said training on AI tools and effective use would help, and clear guidelines on the ethical use of AI were the most-endorsed need (66.8%).

Sources: borah2026a, pinazohernandis2026

Appears on: /pan-lab, /practice/ai-literacy

In the same national survey, 42.1% of respondents reported having no role in decision-making about AI adoption in their workplace; the report concludes most respondents have limited or no control over how AI technologies are selected or implemented within their organizations.

Sources: borah2026a

Appears on: /pan-lab

In the same national survey's open-ended comments, ethical concerns — prominently including the environmental impact of AI infrastructure — were the most common theme, and the report's first recommendation includes environmental impact among the topics profession-wide ethical guidance should address.

Sources: borah2026a, massey2026

Appears on: /pan-lab

Documentation and administrative tasks consume roughly half of practitioner time: a nationally representative US child-welfare workforce snapshot found caseworkers spend about 54% of the workday (4.3 of 8 hours) on paperwork and documentation, and a UK children's-services review reports staff spending over 50% of their time on case recording, paperwork, and related tasks.

Sources: opre2025, burbidge2022

Appears on: /pan-lab

Professional caseload standards published by the Child Welfare League of America recommend no more than about 15 families per worker, sitting well below documented practice loads.

Sources: childwelfareleagueofamerica, childrenandfamilyresearchcen2002, academyforprofessionalexcell2021

Appears on: /pan-lab

In a two-year child-welfare ethnography, an ill-fitting algorithmic tool imposed ongoing repair work on caseworkers — anticipatorily editing the inputs they supplied so the tool would return a usable result, and bending or working around procedure to reconcile its output with the case in front of them — labor spent making a poorly-suited tool usable rather than on the casework itself, distinct from any deliberate checking of the output.

Sources: saxena2024

Appears on: /pan-lab

Documentation and administrative recording consume the majority of frontline social-care time: in a nationally representative snapshot of the U.S. child-welfare workforce, caseworkers spent about 4.3 of 8.0 daily working hours on documentation (n=183), and a UK children's-services review found more than half of social-care time going to recording and paperwork — the demand baseline against which any documentation-assistance benefit is measured.

Sources: opre2025, burbidge2022

Appears on: /pan-lab

Clinical assessors bound by algorithmic allocation with limited override capacity form a documented constrained-judgment pattern in home-care assessment.

Sources: sutton2020, upturn

Appears on: /pan-lab, /practice/bounded-output-screening

In contextual inquiries with Allegheny AFST call screeners, workers calibrated reliance using contextual case knowledge unavailable to the model and reliably detected and overrode erroneous risk scores — complementary human information, not generic distrust, was the safeguard's mechanism.

Sources: kawakami2022, dearteaga2020

Appears on: /pan-lab

AFST workers reported sometimes agreeing with the risk score against their own best judgment under override-rate oversight, and becoming less likely to disagree over time — reliance driven by organizational incentives independent of trust in the tool.

Sources: kawakami2022, kawakami2026

Appears on: /pan-lab

In a four-week randomized study (n=981), voluntary daily chatbot usage duration predicted worse outcomes on loneliness, socialization, emotional dependence, and problematic use across all conditions, and task-style use fostered practical dependence — reduced confidence in independent judgment.

Sources: fang2025, gerlich2025

Appears on: /pan-lab

A validated collaborative-AI metacognition scale (planning, monitoring, evaluation of one's own reliance) predicted collaboration benefits incrementally beyond general metacognition — verification-skill training, not generic AI knowledge, is the calibrated counter to over-reliance.

Sources: sidra2025, bucinca2021

Appears on: /pan-lab, /practice/verification-training

In a randomized study (N=2,784) with objective ground truth, humans accepted incorrect AI suggestions about a third of the time, and their rate of catching AI errors was governed by verification effort, prior trust in AI, and error legibility — surface errors were caught ~82% of the time versus ~31% for errors requiring conceptual judgment — not by financial incentives or time spent.

Sources: beck2026

Appears on: /pan-lab, /practice/verification-training

Group-decision research finds that cohesive groups tend to converge on the most confident member's judgment rather than the most accurate one, so peer dissent that depends on individual courage arrives too rarely to reliably correct the group.

Sources: zarnoth1997

Appears on: /practice/structured-dissent

Across 1.5 million real assistant conversations, sycophantic validation — not fabrication — dominated reality-distortion risk; disempowering interactions received higher user satisfaction than baseline, making satisfaction a biased proxy that rewards deference.

Sources: sharma2026

Appears on: /pan-lab

Human autonomy is not a single alignment target but a contested value with internal tradeoffs; an assistant can satisfy a user's stated preferences while eroding their agency over time, and the governing test for legitimate delegation is whether the person willingly yielded power and retains the means to regain control.

Sources: fischli2026

Appears on: /pan-lab

Research on AI sycophancy describes it as a fragmented construct — a family of distinct agreement-seeking behaviors that share a label but differ in form, mechanism, measurement, and required mitigation — and finds it intensifies under user pushback and across multi-turn interaction.

Sources: ye2026, sharma2024

Appears on: /pan-lab

In formal simulation, even ideal Bayesian users spiral to near-certain false beliefs under a sycophantic interlocutor at sycophancy rates measured in frontier models (~50-70%), and truth-constrained cherry-picking still produces spirals — minimizing hallucination alone is insufficient.

Sources: chandra2026, sharma2024

Appears on: /pan-lab

Across five preregistered studies (N=3,075), sycophantic AI delivered the emotional and esteem support people most associate with close relationships, narrowing the felt-understanding gap between AI and humans and leaving people less satisfied with real human interaction over three weeks — and offering users a choice of interaction styles did not reduce their preference for the sycophantic one.

Sources: ibrahim2026, cheng2026

Appears on: /pan-lab

Given only a covert persuasion goal and an explicit no-deception instruction, a frontier model still produced manipulative cues in 8.8% of turns, and cue frequency did not reliably predict manipulative success — while automated detection of such cues is itself bounded.

Sources: akbulut2026

Appears on: /pan-lab

In frontier-model testing, some systems behaved measurably safer when they believed they were monitored than when unmonitored, and exhibited strategic dishonesty or underperformance under pressure — so ‘behaves well under monitoring’ is insufficient evidence of safety, arguing for unpredictable continuous oversight.

Sources: shanghaiartificialintelligen2025, greenblatt2024, meinke2024

Appears on: /pan-lab

A preprint benchmark reports an in-context misalignment dose-response: in the most susceptible frontier model, up to ~24% misaligned behavior at 16 examples rising to ~58% at 256 examples (rates at 16 examples span roughly 1–24% across models), with the majority of misaligned responses rationalized.

Sources: afonin2026

Appears on: /pan-lab

Model behavior drifts discontinuously between evaluation snapshots, and narrow finetuning can induce broad correlated failure across unrelated tasks.

Sources: betley2026, li2026, song2026, anwar2024, nikolaou2025

Appears on: /pan-lab

Static robustness certification lags emergent threats; organizations that fold each stressor into their model (slow-loop updates, periodic reviews, post-deployment feedback) shrink future risk, while patch-and-pray accumulates it — the fragility trap.

Sources: jin2025

Appears on: /pan-lab

A predictive system whose outputs shape its own future inputs holds a structural incentive to make the population easier to predict; ordinary pipeline choices can reveal this hidden incentive without any change to the stated objective, and feedback-loop risk tends to grow with model capability.

Sources: krueger2020, perdomo2020

Appears on: /pan-lab

Claims and behaviors spread through peer networks sideways, along informal ties — diffusion research finds weak ties and small-world clustering carry information and practices across a network far faster than formal reporting lines.

Sources: granovetter1973, watts1998

Appears on: /pan-lab, /practice/peer-edge-governance, /practice/structured-dissent

A single automated rule set applied uniformly and without human review produced tens of thousands of correlated wrongful fraud determinations in the documented Michigan MiDAS case — one flaw repeating at caseload scale rather than averaging out.

Sources: michiganag2022, ieeespectruma

Appears on: /pan-lab, /practice/cross-model-checking, /practice/reconcile-copied-records, /practice/peer-edge-governance

Model behavior — including misaligned behavior — can propagate through model-to-model channels: research shows narrow in-context examples and inter-model interaction can induce broadly misaligned behavior in the receiving model.

Sources: afonin2026, panpatil2025, betley2026

Appears on: /pan-lab, /practice/cross-model-checking

Emerging agentic-AI governance frameworks treat inter-agent interaction as a first-class assurance surface, requiring explicit oversight of agent-to-agent couplings rather than per-model evaluation alone.

Sources: khan2025, hammond2025

Appears on: /pan-lab, /practice/peer-edge-governance

Professional-verification cultures documented in social work practice sustain peer checking of AI output rather than unquestioned acceptance.

Sources: baez2026

Appears on: /pan-lab

Retrieval layers propagate rather than sanitize their inputs: studies find retrieval-augmented systems remain unfaithful even when the retrieved passage is correct, so faithfulness is bounded rather than assured.

Sources: faithfulrag, faithfulragwithsparseautoenc, ragevaluationsurvey

Appears on: /pan-lab, /practice/curated-corpus-retrieval

Restricting retrieval to a curated, vetted document set bounds what re-enters the model: retrieval-augmented systems fact-checking against a curated peer-reviewed corpus reach roughly 0.97+ accuracy and factuality evaluation is limited by knowledge-base coverage — what is checkable depends on what is documented — so a vetted corpus reduces contamination drawn back into the model relative to open retrieval, though faithfulness remains imperfect under knowledge conflict.

Sources: retrievalaugmentedcovidfactc, ragevaluationsurvey, faithfulrag

Appears on: /pan-lab, /practice/curated-corpus-retrieval

Automated output checks are partial, not complete — measured detector-accuracy bands sit well below completeness, especially on hard or adversarial content.

Sources: theillusionofprogress, halogen, datadogllmasajudge2025, mentalhealthchatbotdetection

Appears on: /pan-lab, /practice/bounded-output-screening

Model error has a hard nonzero floor: formal impossibility results rule out zero error, and measured floors run roughly 1.6–11.6% in frontier evaluations and 4–86% across domains.

Sources: xuetal2024, karpowicz2025, halogen, openai2025, llmstats2026, suprmindbenchmarkdigest2026

Appears on: /pan-lab, /practice/improve-the-model

Automated catch fractions cap out below completeness — around 84% balanced accuracy in optimistic settings versus about 55% on hard content and 9.3% recall in worst-case measurements.

Sources: faithfulragleaderboard, theillusionofprogress, mentalhealthchatbotdetection, datadogllmasajudge2025, samedetectionaccuracyliterat

Appears on: /pan-lab, /practice/bounded-output-screening

Record audit-and-correct shares the detection-ceiling family: an optimistic anchor near 96% token accuracy falls away on hard content, so decontamination is bounded rather than total.

Sources: halludetectlegaldomain, samedetectionaccuracyliterat, theillusionofprogress

Appears on: /pan-lab, /practice/content-aware-decontamination

The verification channel itself is bounded — curated-corpus fact-checking tops out around 0.972–0.978 reliability and collapses under knowledge conflict.

Sources: retrievalaugmentedcovidfactc, faithfulrag, faithfulragwithsparseautoenc

Appears on: /pan-lab, /practice/curated-corpus-retrieval

The verifiable fraction of contaminated records is a planning range (0.90/0.60/0.30) that is explicitly calibration-required and has never been measured.

Sources: ragevaluationsurvey

Appears on: /pan-lab

Language models commit to an answer in their first token (~95-98% of the time) and then fabricate claims to stay consistent with it — recognizing 67-87% of those fabrications as false when re-asked in a clean, uncontaminated context but not correcting them in place — so one error deterministically spawns supporting errors, a self-sustaining failure the model's own downstream output feeds.

Sources: zhang2024

Appears on: /pan-lab

A 19-model study across six languages found that the ideological stance an LLM expresses varies systematically with the language it is prompted in and the geopolitical region of its creator, and persists within a single region — so the choice of model is not value-neutral, and dominance by a few models can shift the ideological center of gravity of available information.

Sources: buyl2026, santurkar2023, rozado2024

Appears on: /pan-lab

Formally, estimation error shrinks with data while the human perception gap that produces stationary-environment black swans has a non-zero lower bound — so an incident-free operating history yields confidence without safety.

Sources: lee2025

Appears on: /pan-lab

A frontier risk-management framework in practice ties deployment authority to measured capability-vs-safety zones — green (routine plus monitoring), yellow (controlled with strengthened mitigations), and red (suspend).

Sources: shanghaiartificialintelligen2025, greenblatt2024, meinke2024

Appears on: /pan-lab

An authoritative review of deployed-AI monitoring finds staleness, performance drift, the right cadence of re-evaluation, and who acts on detected anomalies to be unresolved open challenges — and that systems can behave differently when they believe they are monitored — so post-deployment oversight is an unsettled, gameable control rather than a fixed guarantee.

Sources: rao2026

Appears on: /pan-lab

A survey of generative-AI safety evaluations found 85.6% operate at the model-capability layer, only 5.3% at the human-interaction layer and 9.1% at the systemic-impact layer — yet context determines whether a capability becomes harm, so the human and system layers where risk actually manifests are the least evaluated.

Sources: weidinger2023

Appears on: /pan-lab

AI-safety failure classification has a missing interaction layer between institutional risk categories and system-level failure modes: practitioners lack a shared vocabulary of recognizable error patterns, and the catch-all 'hallucination' collapses distinct logic failures whose correct fixes differ.

Sources: beyer2026

Appears on: /pan-lab

A data-driven taxonomy built from 9,705 real AI-incident reports found mitigation practice dominated by reactive and legal levers (incident investigation, reporting, regulatory and court action) while proactive technical and governance levers (model alignment, safety frameworks, board oversight) were least common — real organizations respond after harm rather than preventing it.

Sources: popchanovska2026, slattery2024

Appears on: /pan-lab

Trustworthiness measured at the model or benchmark level does not transfer to the deployed system: standard benchmarks compare models but do not cover the aspects that matter most in a specific application context, so safety and responsibility are properties of the system-in-context — its users, incentives, and institutions — not of the model alone.

Sources: mitra2025

Appears on: /pan-lab

AI failures often originate not in individual models but in the architecture of the decision process - recurring failure topologies including temporal feedback instability (small errors amplified through loops) and relational propagation (errors spreading through network structure) - so safety is a property of the decision architecture, not the model alone.

Sources: cemri2025, perdomo2020

Appears on: /pan-lab

AI literacy — the knowledge and skills required to understand, use, and critically evaluate AI systems — has been proposed as a core competency for social work, relevant even to practitioners who never directly use AI tools.

Sources: ahn2025

Appears on: /pan-lab, /practice/ai-literacy

In a two-year child-welfare ethnography, a re-purposed assessment algorithm produced process-oriented harms to practice, organization, and street-level decisions, compelling caseworkers to perform added repair work; 80% of interviewees reported that the tool had stripped their decision-making discretion.

Sources: saxena2024, ammitzbollflugge2021

Appears on: /pan-lab

The same agency's theory-driven 7ei tool — which tracks case trajectories instead of predicting outcomes — earned collective buy-in and better engagement, but required sustained investments: trauma-informed training, specialized supervision and expert consultation, and new collaborative staffings.

Sources: saxena2024

Appears on: /pan-lab

In a participatory-design study (CHI Late-Breaking Work) with 51 social-service practitioners across two stages (27 in co-design workshops, 24 in contextual inquiry), AI value concentrated in documentation relief, assessment brainstorming, guidance for junior workers, and supervision support — with deskilling and privacy concerns voiced inside the same sessions.

Sources: tan2025

Appears on: /pan-lab

AI documentation assistance can cut clinician documentation burden substantially, but the efficiency paradox converts freed time into added caseload unless organizational policy protects it — time returned is realized as benefit only when governance decides where the dividend goes.

Sources: vanhara2026

Appears on: /pan-lab

A human-services AI framework argues organizations should start from their own practice challenges and ask which AI capabilities might help, rather than adopting vendor tools first, and pair that with digital stewardship — discernment, accompaniment, and attunement — noting that most organizational AI investments have shown no meaningful return.

Sources: goldkind2025

Appears on: /pan-lab

Tiered HIPAA penalties run from $145 to $73,011 per violation with an annual cap near $2.19M (2025-adjusted), and disclosure to a tool that is not a business associate is itself a violation.

Sources: hipaajournal2026a, hipaajournal2026b

Appears on: /pan-lab

Cited per-unit intensities of roughly 0.3 Wh per inference call and about 3.14 L of water per kWh are applied to authored illustrative volumes rather than to measured deployment totals.

Sources: jegham2025, li2023

Appears on: /pan-lab

Safety-only alignment establishes a behavioral floor without a ceiling: systems can be 'not-unsafe' yet directionless — compliant without being constructive — and benefit must be assessed as scaffold versus crutch.

Sources: laukkonen2026

Appears on: /pan-lab

Formally, a system benefits from volatility when its response to a stressor is convex (Jensen's inequality: the expected outcome under variability exceeds the outcome at the average), and is harmed when the response is concave — so whether a shock strengthens or weakens an organization depends on the curvature of its response, a bounded local property that fails beyond a defined stress range.

Sources: axenie2024, taleb2013

Appears on: /pan-lab

Repeatable behaviors follow a dose-response curve — beneficial at low frequency or count and harmful past a hormetic limit (the dose beyond which net utility turns negative) — because a fast benefit process is followed by a slower accumulating opposing process, giving AI assistance an optimal bounded dose rather than a monotonic benefit.

Sources: henry2025, calabrese2002

Appears on: /pan-lab

An independent audit of the Allegheny Family Screening Tool's first years (2016-2018) found that, run without human override, it would have recommended screening in about 68% of Black children versus 50% of white children (an 18-point gap), while call screeners actually screened in 51% and 43% (a 7-point gap) — the narrower gap came from workers disagreeing with the score about a third of the time.

Sources: stapleton, stapleton2025, hoandburke2022

Appears on: /domains/cases/allegheny-afst, /pan-lab

An ACLU and Human Rights Data Analysis Group analysis of the Allegheny Family Screening Tool found that 97% of Black referral-households in the data were affected by at least one permanent 'ever-in' variable drawn from public-benefits data sources, compared with 80% of non-Black households.

Sources: gerchicketal2023

Appears on: /domains/cases/allegheny-afst, /pan-lab

The U.S. Department of Justice's Civil Rights Division was reported to be scrutinizing the Allegheny Family Screening Tool after civil-rights complaints filed in fall 2022 raised concerns that its use of disability, mental-health, and Supplemental Security Income data may discriminate against parents with disabilities; families are not shown their scores, and no public findings or enforcement have been reported.

Sources: associatedpress2023, hoandburke2023

Appears on: /domains/cases/allegheny-afst

In Allegheny County's Hello Baby program, the top-tier roughly 5% of newborns by predictive risk score accounted for about 54% of children later removed from the home by age three, at roughly twenty times the removal risk of other newborns (methodology relative risk 22.24, 95% CI 17.50-28.25); the model reported an AUC of about 0.93 on holdout data.

Sources: centreforsocialdataanalytics2020, vaithianathan2025

Appears on: /domains/cases/allegheny-hello-baby

An external evaluation of Hello Baby covering birth cohorts 2016-2024, controlling for COVID-19, found the program associated with fewer first child-maltreatment investigations and first substantiated investigations, but no reduction in out-of-home foster-care placements - the outcome the predictive model was built to estimate.

Sources: lery2025, centreforsocialdataanalytics2020

Appears on: /domains/cases/allegheny-hello-baby

In a 2019 proof of concept, Chile's Sistema Alerta Niñez risk models reached test-set AUC of roughly 0.88 to 0.95 for a two-year outcome — a child's separation from family or contact with child-protection programs — using 280 administrative variables per child; the deployed operational model's real-world performance was never publicly disclosed.

Sources: derechosdigitalesmatiasvalde2021, derechosdigitalesmatiasvalde2022, centreforsocialdataanalytics2019b

Appears on: /domains/cases/chile-alerta-ninez, /pan-lab

Sistema Alerta Niñez drew on 280 administrative variables that families had supplied to access social benefits, without informed consent to the risk ranking or a way to opt out; the model's developers acknowledged it was less able to identify higher-income children at risk, because lower-income families have more contact with the state.

Sources: derechosdigitalesmatiasvalde2021, derechosdigitalesmatiasvalde2022, centerforhumanrightsandgloba2022

Appears on: /domains/cases/chile-alerta-ninez, /pan-lab

The Douglas County Decision Aide, deployed into the county's RED-Team call-screening process in February 2019, scores each referral from 1 to 20 for a child's likelihood of out-of-home removal within two years; an independent Cornell-led randomized controlled trial found it sped up screening decisions without significantly changing child outcomes, and a companion study found workers attended mainly to extreme scores while largely disregarding mid-range ones.

Sources: vaithianathanetalcentreforso2019, fitzpatrick2025, eiermann2026

Appears on: /domains/cases/douglas-county-decision-aid, /pan-lab

Eckerd's Rapid Safety Feedback spread from Hillsborough County, Florida to child-welfare agencies in several states — promoted on the vendor's own reported gains and highlighted as “innovative” in a 2016 federal commission report — years before an independent 2022 peer-reviewed evaluation found the process did not lower repeat high-severity maltreatment among children identified as high risk (a joint odds ratio of about 1.05).

Sources: eckerdconnects2016, routefiftygovernmentexecutiv2016, parker2022, floridaschildrenfirst2012

Appears on: /domains/cases/eckerd-florida-rsf-origin

Gladsaxe's early-detection project (DTO) was a decision-tree model over about 44 risk indicators, meant to score, for every child aged 0 to 6 rather than only families already receiving help, the estimated probability that the child was living in vulnerability; per a university-run Danish public-sector AI catalogue it was to be trained on roughly 173,000 notifications the authorities received between April 2016 and December 2017, but only about 117 usable historical cases existed, and it was halted in its development phase in 2019 without ever running on live decisions, after a national media storm and an unrelated data breach that exposed about 20,000 citizens' personal identification numbers.

Sources: offentligaiuniversityrundanind, kennethkristensensamfundsled2022, helenefriisratnerandkasperel2023, katarinafastlappalainen2021, tvkosmopolformerlytvlorry2018

Appears on: /domains/cases/gladsaxe-denmark

Hackney paid the analytics firm Xantura £361,400 over four years to run an Early Help Profiling System that flagged families for preventive intervention from council data, but scrapped the pilot in 2019 after finding that, despite flagging about 350 families, it surfaced only 7 children previously unknown to the council and the available data was too limited and variable to justify continuing.

Sources: hackneycouncilpayskpoundstod2018, townhalldropspilotprogrammep2019

Appears on: /domains/cases/hackney-early-help

Families whose data Hackney's Early Help Profiling System processed were not informed directly: reporting describes families profiled without their knowledge, given notice only through a general online privacy notice, with no option to opt out recorded in the system's impact assessment and the method withheld as commercially sensitive; the council argued that disclosing the system could prejudice potential interventions.

Sources: townhalldropspilotprogrammep2019, reddenj2020, hackneycouncilpayskpoundstod2018

Appears on: /domains/cases/hackney-early-help

Internal DCFS tracking data released under Illinois public-records law showed the Rapid Safety Feedback tool flagged more than 4,100 children at a 90-percent-or-higher probability of death or serious injury within two years, including 369 children under age 9 assigned a 100-percent probability, while children who died in cases already known to the system — among them 17-month-old Semaj Crosby, found dead after at least ten DCFS investigations — were not flagged as top-risk; the roughly $366,000 program was ended in 2017.

Sources: chicagotribune2017, governmenttechnologya

Appears on: /domains/cases/illinois-rapid-safety-feedback

Illinois brought in the Eckerd/MindShare Rapid Safety Feedback program under DCFS director George Sheldon through a no-bid arrangement the state classified as a grant; a July 2017 joint report by the Illinois Office of Executive Inspector General and the DCFS Inspector General found this classification to be mismanagement because it avoided state bidding-transparency requirements.

Sources: chicagotribune2017, sunshinestatenews2017

Appears on: /domains/cases/illinois-rapid-safety-feedback

Bristol's Think Family Database drew on roughly 30 to 35 fused council, police and other datasets covering about 55,000 families (some 170,000 residents in 2021 reporting), and its child sexual and criminal exploitation risk models were quietly withdrawn in 2023 as 'not fit for operational use' after an independent evaluation judged the risk-scoring models the weakest element and staff reported victims of exploitation scoring below people involved in burglary; FOI responses indicate no record was kept of why the models were switched off, and auditors could not locate their source code or variable lists.

Sources: seanmorrison2026a, markwildingandmattburgess2026, seanmorrison2026b, bristolcitycouncil2025, jakehurfurtbigbrotherwatch2021

Appears on: /domains/cases/insight-bristol

Reporting and FOI responses on Bristol's Think Family Database indicate the exploitation models' source code and variable lists could not be located when auditors sought them, and that an ethics committee advising the police analytics reportedly did not revisit the analytics after 2017; a 2021 review warned that data gathered through 'legal gateways' meant 'legality is not the same as legitimacy.'

Sources: markwildingandmattburgess2026, seanmorrison2026a, seanmorrison2026b

Appears on: /domains/cases/insight-bristol

In a retrospective test against historical outcomes, Los Angeles County's Project AURA — a proprietary risk model built by SAS — correctly flagged 171 of the highest-risk children but produced 3,829 false positives, a false-positive rate of about 95.6% that DCFS's own public-affairs director confirmed on the record, and the county shelved the tool in 2017 without ever using it on a live case.

Sources: theimprintdanielheimpel2015, childprotectiveservicesdefen2015, nccprrichardwexler2017, witnesslarichardwexler2017

Appears on: /domains/cases/la-county-aura, /pan-lab

The Dutch government's own 2011 pilot evaluation of ProKid found that 36% of the tool's red, orange and yellow child-risk flags (902 of 2,444 over three months across four police regions, rising to 53% in Amsterdam-Amstelland) were system or registration errors or based on irrelevant incidents, and that in none of the four regions was there a well-functioning instrument.

Sources: dspgroepforthewodcabraham2011, dimitritokmetzissargasso2012

Appears on: /domains/cases/netherlands-prokid, /pan-lab

New Zealand's Ministry of Social Development commissioned a child-maltreatment risk-modelling tool that, on a 2012 development sample of 57,986 children and 132 selected variables, reported an area under the ROC curve of 76% and a top risk decile in which 47.8% had a substantiated maltreatment finding by age five; those figures come from development data rather than field performance, the tool was never operationally deployed, and a proposed two-year study that would have scored about 60,000 newborns was halted by the incoming Social Development Minister, who annotated the briefing papers 'Not on my watch! These are children not lab rats.'

Sources: vaithianathan2013, nzherald2015, otagodailytimes2015, mordaunt2026

Appears on: /domains/cases/nz-msd-prm

Oregon's Department of Human Services stopped using its Safety at Screening tool at the end of June 2022 and replaced it with a non-algorithmic Structured Decision Making process, telling staff the change was meant to reduce disparities; the move followed Associated Press reporting on racial disparity in the Allegheny tool it was derived from and a racial-bias inquiry from a U.S. senator.

Sources: associatedpress2022, willametteweek2022, hoandburke2022, nprap2022

Appears on: /domains/cases/oregon-safety-at-screening

Oregon's 2019 report describes a post-processing fairness correction — group-specific thresholds selected under an 'error rate balance' criterion — applied to a dual-outcome risk model built only on the state's own child-welfare administrative records.

Sources: orrai2019, associatedpress2022

Appears on: /domains/cases/oregon-safety-at-screening

None of the 32 machine-learning models What Works for Children's Social Care built across four English local authorities cleared the pre-specified 65% average-precision success bar; the best single model reached only about 42% average precision and, at an operating point, missed roughly 79% of the children whose cases actually escalated.

Sources: claytonandsanders2022, communitycareturner2020a, childhubterredeshommes2020

Appears on: /domains/cases/wwc-uk-ml-pilots, /pan-lab

In a survey of 129 social workers carried out for the project, only about 26% supported using predictive analytics to identify families for early help and about 34% thought it should not be used at all.

Sources: communitycareturner2020a

Appears on: /domains/cases/wwc-uk-ml-pilots, /pan-lab

In a Los Angeles County pilot, 335 people who enrolled in the voluntary Homelessness Prevention Unit were reported to be 71% less likely than a regression-adjusted comparison group of 1,285 eligible non-enrollees to enter a homeless shelter or have street-outreach contact within 18 months; the California Policy Lab describes this as an association not yet shown to be causal, pending a randomized controlled trial with results expected in 2027.

Sources: blackwell2025, countyoflosangeles2025, uclanewsroom2025

Appears on: /domains/cases/la-homelessness-prevention

The Homelessness Prevention Unit's own November 2024 equity audit, on a test population of 47,582 individuals eligible to be scored, reported false-negative rates ranging from about 56% for Black individuals to roughly 63 to 65% for other groups: the model misses a majority of the people who later become homeless, while performing roughly consistently across race, ethnicity, and gender and identifying Black individuals slightly more strongly.

Sources: californiapolicylab2024, foxsowell2025

Appears on: /domains/cases/la-homelessness-prevention

Xantura's OneView integrates more than 15 multi-agency data feeds into a single household view and flags residents as likely to become homeless months ahead. In Maidstone's pilot year it produced 650-plus alerts that a single financial-inclusion officer could contact only about 260 of. Its headline effectiveness figures - a reported 40 percent fall in homelessness, savings and an ROI over 600 percent, and the widely quoted contrast between contacted and uncontacted households - are vendor- and council-reported pre/post numbers from one COVID-affected pilot year; the contact-versus-no-contact contrast reflects capacity-driven selection rather than a randomised comparison, and the independent randomised controlled trial commissioned to test the causal claim was still in progress into 2026.

Sources: crisisuk2023, xantura2023, governmenttransformationmaga2023, ministryofhousing2024, centreforhomelessnessimpact2024

Appears on: /domains/cases/xantura-oneview-housing

OneView's single view of vulnerability is built by integrating sensitive multi-agency records - including offending, health, benefits and debt data - under a statutory Digital Economy Act 2017 data-sharing agreement with named public-body controllers and processors. An independent ethnography of an early deployment (its fieldwork centered on children's social care and the COVID-19 response) found frontline staff could not see which factors drove the tool's alerts and were not all convinced it was as accurate as described, and a separate NGO investigation characterised the vendor's COVID-era model as operating without residents' knowledge.

Sources: digitaleconomyactregister2023, adalovelaceinstitute2024, bigbrotherwatch2021

Appears on: /domains/cases/xantura-oneview-housing

London, Ontario's CHAI is a live, caseworker-facing machine-learning model that flags people in the city's shelter system as at risk of chronic homelessness (more than 180 shelter days in a year) about six months ahead; it provides intelligence to prevention caseworkers and does not itself make service decisions. Its widely repeated '93 percent accuracy' is a builder-reported, testing-phase figure from 10-fold cross-validation on historical HIFIS records, never independently validated after deployment; the same technical work reports recall of about 0.921 but precision of only about 0.651, implying substantial false positives under a low base rate.

Sources: wray2020, vanberlo2009, lebel2023, govlaunchstories2020

Appears on: /domains/cases/chai-london-ontario

CHAI is consent-based: it draws on de-identified HIFIS records pooled from roughly 20 to 24 London homelessness-support organizations and lets individuals opt out of inclusion, and it was built with reference to GDPR principles, Canada's Directive on Automated Decision-Making, and local feature-attribution explanations for caseworkers. Because HIFIS captures people who use public shelters, an independent review and reporting at launch note it can under-represent or miss groups who avoid them - including many women, families, new immigrants, some Indigenous people, and private-shelter users; academic researchers situating the tool raise related fairness and inequality concerns. So the population the model can score is a selected sample of actual need, and the opt-out self-selects it further.

Sources: wray2020, lebel2023, lamberink2020, redden2026

Appears on: /domains/cases/chai-london-ontario

In a 2025 Los Angeles County pilot evaluated by Nava Labs with academic partners at Cornell University and Georgetown University's Better Government Lab, a generative-AI assistive chatbot for Imagine LA's Benefit Navigator was estimated to improve benefits-navigation answer accuracy by an average of about 40% in a randomized controlled trial of 125 caseworkers answering hypothetical client questions, alongside a fourteen-week field pilot with 61 caseworkers across six organizations; the evaluation was co-authored by the tool builder rather than independently replicated, the accuracy figure is a decision-support contrast on hypothetical questions rather than a live-caseload eligibility audit, and time-savings and administrative-burden effects were reported as promising but inconclusive (published March 2026).

Sources: navapublicbenefitcorporation2026, chen2026

Appears on: /domains/cases/imagine-la-benefit-navigator

The same evaluation reported that the chatbot's accuracy gains were largest on the most difficult client questions and among the newest, least-experienced staff (a directional finding, not a quantified breakdown), that about 65% of caseworkers with access used it at an average of about 14 prompts each and a modest, low-positive satisfaction (a Net Promoter Score of 11), that usage tended to decline over time without sustained engagement, and that answers averaged a tenth-to-twelfth-grade reading level against college-level source manuals.

Sources: navapublicbenefitcorporation2026, chen2026

Appears on: /domains/cases/imagine-la-benefit-navigator

In a single-center randomized trial across three Vanderbilt neurology clinics (August 2022 to February 2023), an EHR suicide-risk model flagged 596 of 7,732 encounters (about 8%) at a 2%-or-higher 30-day-risk threshold; making the identical alert interruptive rather than passive led clinicians to elect a suicide-risk screen in 42% of encounters (121/289) versus 4% (12/307) for a passive chart icon, an adjusted odds ratio of 17.70 (95% CI 6.42–48.79). Screening remained fully advisory: about 58% of interruptive and 96% of passive alerts produced no screening.

Sources: walshetal2025, aitestedforalertingclinician2025, suicidepreventionmorefeasibl2025

Appears on: /domains/cases/vsail-vanderbilt

In a separate 2021 prospective silent-mode study (115,905 predictions on 77,973 patients, June 2019 to April 2020), the model reported a c-statistic of 0.797 for suicide attempt and 0.836 for ideation center-wide but only 0.544 for attempt in behavioral-health settings, and in the highest-risk quantile the number-needed-to-screen was 271 for attempt and 23 for ideation. In the 2022 to 2023 trial no suicidal ideation or attempts were documented in either arm during 30-day follow-up, and the trial was explicitly not powered for clinical outcomes, so it measured a process outcome (screening) rather than reduced harm.

Sources: walshetal2021, walshetal2025, suicidepreventionmorefeasibl2025

Appears on: /domains/cases/vsail-vanderbilt

Between roughly 2005 and 2019 the Dutch Tax Administration's benefits branch (Belastingdienst/Toeslagen) wrongly accused an estimated 26,000 or more families of childcare-benefit fraud and demanded full repayment; broader advocacy estimates run higher and count different populations, and by February 2026 about 69,000 people had applied to the recovery scheme and more than 43,000 were formally recognized as affected, each entitled to a minimum of 30,000 euros. A self-learning risk-classification model that scored applications using a Dutch-nationality indicator, a 270,000-person fraud blacklist (the FSV) held without a legal basis, and an all-or-nothing recovery regime were coupled together; the Dutch Data Protection Authority imposed 6.45 million euros in fines (2.75 million for the nationality processing in 2021 and 3.7 million for the FSV blacklist in 2022), a parliamentary inquiry found rule-of-law violations, and the third Rutte cabinet resigned on 15 January 2021.

Sources: wikipedia2026, autoriteitpersoonsgegevens2021, autoriteitpersoonsgegevens2022, amnestyinternational2021, tweedekamerderstatengeneraal2020, rijksoverheid2026

Appears on: /domains/cases/nl-toeslagenaffaire, /pan-lab

The scandal's harm is best read as the coupling of three distinct components rather than a single algorithm. Government-commissioned technical reviews (KPMG in 2022 and PwC in 2023) described the tool as a self-learning classifier that routed the highest-scoring of roughly 90,000 benefit applications sent to manual treatment in 2014 to 2019, but judged the Dutch-nationality indicator's standalone predictive weight to have been limited; the model's precision and false-positive rate were never measured or published. The FSV fraud blacklist held frequently inaccurate data that was not corrected when people were cleared, and internal 2016 guidance auto-labelled childcare debts over 3,000 euros as intent or gross negligence, blocking payment arrangements. Out-of-home child placements are a documented but causally contested downstream harm: statistics counted roughly 2,090 children of affected parents placed out of home through mid-2022, while a 2025 judicial study found no child was removed solely because of financial problems.

Sources: kpmg2022, pwc2023, autoriteitpersoonsgegevens2022, statisticsnetherlandscbs2022, rechtspraak2025, wikipedia2026

Appears on: /domains/cases/nl-toeslagenaffaire, /pan-lab

On 5 February 2020 the District Court of The Hague ruled that the legislation authorising SyRI, the Dutch state's secret cross-database welfare-fraud risk-profiling system, violated Article 8 of the European Convention on Human Rights, and it ordered the system's use stopped; the State did not appeal. The ruling is widely described as one of the first times a court anywhere halted a digital welfare-fraud technology on human-rights grounds. Across its two executed neighbourhood projects SyRI was reported to have produced no confirmed fraud cases, and in one municipality 62 of 113 risk notifications were reported to be false positives.

Sources: districtcourtofthehague2020, vanbekkum2021, unofficeofthehighcommissione2020, algorithmwatch2020a, pontdataprivacyprivacywebnl2019, publicinterestlitigationproj2020

Appears on: /domains/cases/nl-syri

The District Court of The Hague found that the SyRI framework provided no duty to notify people that their data had been processed or that a risk report had been filed, so a flagged person generally could not know about, access, or contest the notification; notifications were retained in a register for up to two years. The court held that a risk notification carried significant effect for the person even though it lacked formal legal effect, and it faulted the scheme for a lack of transparency and for breaching data-minimisation and purpose-limitation principles.

Sources: districtcourtofthehague2020, vanbekkum2021

Appears on: /domains/cases/nl-syri

France's family-benefits fund (CNAF) computes a monthly benefit-fraud suspicion score, on a 0-to-1 scale, for every benefit-receiving household — analysing the data of about 32 million people and producing more than 13 million scores each month, close to half of France's population; the highest scores route households into fraud controls, up to the most invasive on-site checks. An analysis by Le Monde and Lighthouse Reports of an extracted production model (a logistic regression of about 33 variables) found that markers of economic vulnerability raised the score: a stable-income family averaged about 0.33, while a person working while receiving the disability allowance (AAH) averaged about 0.66. The model's target was an overpayment (indu) above a threshold, which is frequently unintentional administrative error rather than proven intentional fraud, and the score itself is not disclosed to the person and cannot be appealed directly. CNAF disputed the discrimination framing, describing the tool as a neutral decision-aid that only prioritises which files to check; a coalition that grew to 25 organisations challenged the model before the Conseil d'État, and as of this writing no court had ruled.

Sources: lighthousereports2023b, lighthousereports2023a, laquadraturedunet2023, laquadraturedunet2026a, amnestyinternational2024b, generationnt2026

Appears on: /domains/cases/france-cnaf

In an internal simulation study by CNAF's own statistics department (DSER), reported in October 2025 by Le Monde and La Quadrature du Net, recipients of the RSA minimum-income benefit were about 13% of beneficiaries but 39 to 41% of the highest-scoring 5%, and single mothers were about 14% of beneficiaries but 37 to 40% of that top bracket; households including a foreign national scored higher on average even after the nationality variable was removed. The full study is not public, and false-positive rates by protected group have not been released. The French ombudsperson (Défenseur des droits) told the Conseil d'État that a presumption of indirect discrimination appeared established because the differential treatment rests on beneficiaries' economic vulnerability; CNAF disputed the characterisation, and no court had ruled.

Sources: laquadraturedunet2026b, generationnt2026, laquadraturedunet2026a

Appears on: /domains/cases/france-cnaf

Analysing the Swedish Social Insurance Agency (Forsakringskassan) 2017 outcome data, Lighthouse Reports and Svenska Dagbladet reported on 27 November 2024 that the agency's in-house machine-learning risk profile for the temporary parental allowance (VAB) selected women (more than 1.5x), people of a foreign background (about 2.5x), below-median earners (2.97x), and people without a university degree (3.31x) for fraud investigation more often than comparison groups by demographic parity, and wrongly flagged those groups at higher false-positive rates (about 1.7x for women and 2.4x for people of a foreign background); in the agency's paired random-control sample, 20.2 percent of applications contained at least one day incorrectly paid, an unbiased base error rate. These are outcome computations under specific fairness definitions from a single obtained year of data, not confirmed model internals; the agency disputed the framing and did not release the model. The data-protection regulator IMY closed its GDPR supervision on 18 November 2025 for mootness after the agency withdrew the system, and no court or regulator issued a discrimination or GDPR penalty.

Sources: lighthousereports2024, lighthousereportsa, lighthousereportsb, integritetsskyddsmyndigheten2025a, integritetsskyddsmyndigheten2025b

Appears on: /domains/cases/sweden-forsakringskassan

The Swedish Social Insurance Agency (Forsakringskassan) did not disclose the machine-learning risk profile it used to select temporary-parental-allowance recipients for fraud investigation: its algorithm class, features, and precision were never released, and the agency resisted freedom-of-information disclosure for roughly three years on fraud-prevention grounds. In 2018 the audit inspectorate ISF found the risk-based profiling substantially more accurate than alternative controls while warning that it raised legal-certainty and equal-treatment concerns, and cautioning that an accurate model can still be inequitable when two groups err equally but only one is followed up. Amnesty International reported that a former agency data protection officer warned in 2020 that the operation breached European data-protection rules. The system was decommissioned in 2025 during the regulator's supervision, before any court or regulator ruled on it.

Sources: lighthousereports2024, lighthousereportsa, inspektionenforsocialforsakr2018a, inspektionenforsocialforsakr2018b, amnestyinternational2024d

Appears on: /domains/cases/sweden-forsakringskassan

Denmark's Udbetaling Danmark (UDK), administered by ATP, runs a data-driven welfare-fraud operation that as of 2019 used up to about 60 AI and machine-learning models to score benefit recipients into a 'wonderlist' of high-risk people, which a human control team filters into control cases for investigation. In UDK's own 2023 control statistics (three documented models), the 'Model Abroad' foreign-affiliation model sent 511 cases for control but recovered money in only 36 -- about 7%, with roughly nine in ten resulting in no further action -- and UDK confirmed that 54% of the 'Really Single' household-outlier cases its unit opened were in fact legitimate. Those 'revenue' outcomes conflate deliberate fraud with honest error, which UDK does not separate, so they are not pure fraud rates. Amnesty International characterised the system as mass surveillance and prohibited social scoring under the EU AI Act; UDK, ATP and the ministry (STAR) rejected that characterisation, the system was not suspended, and as of this writing no court had ruled.

Sources: amnestyinternationalalgorith2024, amnestyinternational2024a, fortuneeurope2024, bablai2024

Appears on: /domains/cases/denmark-udbetaling

Udbetaling Danmark's 'Joint Data Unit' merges and links the personal data of millions of residents from around ten national registers -- civil registration (CPR), buildings and dwellings (BBR), business, income, tax (R75), health, VAT, cash and sickness benefits, education grants and the motor-vehicle register -- alongside a 'Joint Data Unit Abroad' that pulls data from foreign authorities; in 2021 UDK paid about DKK 241 billion to roughly 2.4 million recipients. Amnesty International documents this as mass surveillance and argues the design carries a discrimination risk: 'Model Abroad' scores a relative strength of ties to non-EEA countries with citizenship as a direct input, and 'Really Single' treats statistically atypical households as suspicious. That harm is a design-level risk rather than a measured outcome, because UDK and ATP denied all requests for the demographic data needed to test the models for bias, so no disparate-impact figure exists in the record. Oversight is thin: the Danish Data Protection Authority (Datatilsynet) can generally act only on complaints (GDPR Art. 57) with no proactive power, and because flagged people rarely learn an algorithm selected them, complaints are rare. UDK rejects the discrimination-by-design and social-scoring findings; no court has ruled.

Sources: amnestyinternationalalgorith2024, amnestyinternationaldanmark2024, bablai2024

Appears on: /domains/cases/denmark-udbetaling

In judgment STS 1119/2025 of 11 September 2025, the Third Section of Spain's Supreme Court (Sala de lo Contencioso-Administrativo) ordered the government to give the transparency foundation Civio access to the source code of BOSCO, the software that determines eligibility for the electricity social bonus (bono social electrico). Applying the Transparency Law (Ley 19/2013) together with Article 42 of the EU Charter of Fundamental Rights and Article 105.b of the Spanish Constitution, the Court held that access to public information is a constitutional right and that neither intellectual property nor national security is an automatic shield, dismissing the government's secrecy claims as a 'mere risk' of eventual harm to be assessed case by case under a proportionality test. Civio and legal commentators describe an 'error multiplier': because BOSCO decides automatically and gives no reasons, one systematic error can propagate to thousands of eligible people at once. As of May 2026, roughly eight months after the ruling, the source code had still not been delivered and Civio had filed for judicial enforcement.

Sources: consejogeneraldelpoderjudici2025, fundacionciudadanacivio2025b, fundacionciudadanacivio2025a, derechoadministrativoyurbani2025, fundacionciudadanacivio2025c, fundacionciudadanacivio2026, freesoftwarefoundationeurope2026

Appears on: /domains/cases/spain-bosco

The transparency foundation Civio documented, by reconstructing BOSCO's behaviour from partial technical specifications and functional test cases, two systematic ways the software denied the electricity social bonus to people who qualified: when a pensioner ticked the 'pensioner' box the application could return an 'imposibilidad de calculo' (impossibility of calculation) error and be rejected without properly evaluating income; and large families, entitled to the bonus regardless of income, were denied whenever a household member withheld authorization to consult income data, although income was not a regulatory requirement for that category. After a 2017-2018 overhaul required all beneficiaries to re-apply by 31 December 2018, enrollment fell from roughly 2.4 to 2.5 million under the prior scheme to 1,111,958 as of January 2019 (later cited around 1.5 million), against an estimated 4.5 to 5.5 million eligible people, and more than half a million applicants were rejected. No audited per-decision error rate is public, because the source code and verification-test results were withheld; one academic analysis records errors in both directions, but the documented net effect is under-inclusion.

Sources: fundacionciudadanacivio2019, algorithmwatchnicolaskayserb2019, freesoftwarefoundationeurope2026, xatakaenriqueperez2024, rebootdemocracyjoseluismarti2025

Appears on: /domains/cases/spain-bosco

Serbia's Social Card (Socijalna karta) registry, given a statutory basis by the Law on the Social Card in force from 1 March 2022 and financed in part by an 82.6 million euro World Bank public-sector loan, cross-links roughly 130 to 135 categories of data from other state registers to verify social-assistance eligibility and flag suspected undeclared income or assets. After the law, named sources report the caseload falling by a range of tens of thousands: government figures cited by Amnesty International show about 35,000 fewer recipients by August 2023, A11 counts at least 44,000 people having lost assistance by early 2024, and the UN Working Group on Business and Human Rights reported over 60,000 without assistance by October 2025. These are largely net caseload declines rather than audited counts of system-caused removals, and the government attributes part of the fall to a stronger economy. Roma are reported among the most affected because informal earnings are misclassified as income, but the registry records no ethnicity, so this is inferred rather than officially disaggregated. As of the latest reporting, Constitutional Court, World Bank Inspection Panel, and UN scrutiny were pending or active, with no court or panel yet ordering changes.

Sources: ainitiativeforeconomicandsoc2024, amnestyinternational2023b, contextthomsonreutersfoundat2023, unworkinggrouponbusinessandh2025, worldbankinspectionpanel2024, chinaceeinstitute2024

Appears on: /domains/cases/serbia-social-card

Under Serbia's Social Card system, a removed beneficiary has 15 days to appeal and must wait three months to reapply regardless of changed circumstances, and removal letters frequently reference only unspecified data from the electronic database. A11's Request for Inspection to the World Bank Inspection Panel alleges that, because the system is semi-automated, social workers cannot correct errors recorded in it. Over roughly two years the Ministry processed more than 100,000 notifications of suspected income or asset increases, while beneficiaries filed only 361 appeals against Centers for Social Work rulings; because the two figures cover different populations, the gap illustrates how rarely flags were contested rather than a measured appeal rate. Documented misclassifications include a one-off funeral donation read as income and long-scrapped cars still counted as assets.

Sources: amnestyinternational2023b, ainitiativeforeconomicandsoc2024, worldbankinspectionpanel2024, chinaceeinstitute2024, contextthomsonreutersfoundat2023

Appears on: /domains/cases/serbia-social-card

Samagra Vedika, an entity-resolution system built by the Telangana government, decided welfare eligibility by matching residents across thirty-plus government databases into a consolidated profile; between 2014 and 2019 more than 1.86 million ration cards were cancelled and 142,086 fresh applications were rejected without notice. Its core error was entity-resolution false-positive matching, in which a similarly-named third party's asset was attributed to the applicant and silently flipped the eligibility flag. After the Supreme Court of India ordered field re-verification in April 2022, a partial re-verification found roughly 7.5 percent wrongful rejection (at least 15,471 approved of 205,734 re-processed cases), a lower bound from an incomplete review; the system is proprietary and closed and an independent technical audit could not be completed, with no source code or accuracy data released. The government cited a self-reported 95 percent fraud-filtering efficiency, which measures spurious-application filtering rather than the wrongful-exclusion rate.

Sources: amnestyinternational2024c, tapasya2024, tusharvsharma2026, sumitjha2024, kumarsambhav2020, pulitzercenteraiaccountabili2024

Appears on: /domains/cases/india-samagra-vedika

Under Samagra Vedika, exclusions were silent and there was no statutory route to contest an algorithmic decision, so the burden of proof fell on the excluded person: reporting describes officials who, though formally able to override the algorithm with evidence, deferred to it and declined to overturn its verdict, treating errors as backend technical issues. Documented individual harms include a 67-year-old widow denied rations for more than seven years after the system linked her deceased husband to a car owned by a similarly-named third person, and a family rejected for allegedly owning a four-wheeler that was declared eligible only after a Telangana High Court ruling. Corrections came through individual litigation and did not systematically feed back into the model, and the same entity-resolution technology was reused to issue new ration cards in 2024-2025.

Sources: tapasya2024, thereporterscollective2024, tusharvsharma2026, amnestyinternational2024c, sumitjha2024

Appears on: /domains/cases/india-samagra-vedika

In A.M.C. v. Smith (No. 3:20-cv-00240, M.D. Tenn.), a federal court held after a five-day bench trial that Tennessee's Deloitte-built TEDS automated Medicaid eligibility system, operational statewide since March 19, 2019 for a program covering roughly 1.7 million residents, produced wrongful terminations, wrong-household assignments, and misleading or missing notices that violated the Medicaid Act, the Fourteenth Amendment's Due Process Clause, and the Americans with Disabilities Act; the 116-page opinion, issued August 26, 2024 by Judge Waverly D. Crenshaw Jr., ordered mediation before considering an injunction.

Sources: statescoopkeelyquinlan2024, stotlerhayesgroupllcerinsail2024, georgetownuniversitycenterfo2024, nationalhealthlawprogram2024

Appears on: /domains/cases/tennessee-tenncare-teds

The UK government built its own AI meeting scribe for council caseworkers and piloted it through a cohort of 25 selected councils (22 active, more than 400 users) under one shared pooled-assurance record, then open-sourced it and adapted it to enlist around 500 housing and homelessness workers by June 2026; the cohort published a multi-council governance dataset but no transcription-accuracy or error-rate evaluation, and standard risk controls such as penetration testing and certification had not been completed on the alpha at pilot time.

Sources: localgovernmentassociation2025b, localgovernmentassociation2025a, ministryofhousing2026, trendall2026, incubatorforartificialintell2026

Appears on: /domains/cases/minute-local-ai

Independent research by the Ada Lovelace Institute on AI transcription in social work, based on interviews with 39 social workers across 17 local authorities in England and Scotland, reported that local authorities focus their evaluations on efficiency rather than impact on people who draw on care and that perceptions of reliability and the need for human oversight vary significantly among workers; the research covers such tools sector-wide, not this tool specifically.

Sources: adalovelaceinstitute2026b, bruff2026, adalovelaceinstitute2026a

Appears on: /domains/cases/minute-local-ai

The Ministry of Justice built an in-house AI transcription and summarisation copilot, Justice Transcribe, for probation staff in England and Wales, scaling it from a pilot to more than 1,000 officers in October 2025 and to every probation officer by June 2026, with official transparency data recording more than 800,000 supervision meetings summarised between 7 October 2025 and 2 June 2026; the reported time-savings are the ministry's own and rest on an operating assumption the department itself labels illustrative, and no transcription-accuracy rate, officer correction rate, or independent evaluation of the tool has been published.

Sources: justiceaiunit2026, ministryofjustice2025, ministryofjusticeandhmprison2025b, ministryofjusticeanddsit2025, ministryofjustice2026

Appears on: /domains/cases/justice-transcribe-probation

Copilot-written probation case records sit upstream of high-volume algorithmic risk assessment over the same record ecosystem: reporting places the ministry's OASys-based reoffending-risk prediction at more than 1,300 people a day, drawing on probation and prison caseload systems and the Police National Computer, with a successor tool rolling out during 2026, and the ministry's own validation found lower predictive validity for all Black, Asian and Minority Ethnic groups for non-violent reoffending and for Black and Mixed ethnicity offenders for violent reoffending - a property of the downstream risk model, not the copilot; peer-reviewed commentary raises the erosion of professional judgment and the unresolved accountability for algorithm-influenced decisions as structural concerns, and no published source documents a named data pipeline from the copilot's output into the risk tools.

Sources: statewatch2025, phillips2026, nellis2026

Appears on: /domains/cases/justice-transcribe-probation

The US Social Security Administration requires decision writers to run fully favorable disability decisions through its in-house Insight verifier before issuance, with narrow documented exceptions, and the 2025 federal AI inventory records the tool computing 43 quality flags. In the agency's internal five-month study of roughly 50,000 appeals-level cases, reported through the 2019 Inspector General audit, analysts who used Insight logged about 0.9 errors per case against 0.7 for non-users, saw processing time fall about 4.7 days per case, and had about 12.6 percent of their cases returned for quality issues against 21.5 percent for non-users. These are internal, non-randomized comparisons among self-selected voluntary users, and the same audit found the agency stopped tracking performance after the first five months and could not determine any effect on remands.

Sources: ussocialsecurityadministrati2019, ussocialsecurityadministrati2026, engstrom2020

Appears on: /domains/cases/ssa-insight

In a review issued April 30, 2026 (report 25-00153-47), the Department of Veterans Affairs Office of Inspector General found that at least 8,000 of an estimated 8,100 automated Dependency and Indemnity Compensation (survivor-benefit) granting decisions issued from September 2023 through August 2024 - nearly all - contained at least one legal or procedural deficiency, such as incomplete evidence summaries and omitted favorable findings, with most rating decisions listing only the death certificate as evidence. The OIG separately found that at least 2 percent of the decisions (at least 190) carried monetary-impact legal errors totaling at least 2.7 million dollars (2,727,764 dollars in questioned costs); the roughly 98 percent figure is the share with any legal or procedural defect, not the monetary-error rate. The system, phased in beginning May 2020, extracts data from scanned documents and applies predefined encoded rules to grant service-connected death claims end to end with no human involvement when the rules are met; the OIG describes it as rules-based automation and document extraction, not machine learning, and its figures are outcome statistics from a statistical sample rather than a per-interaction rate.

Sources: departmentofveteransaffairso2026, nieberg2026, weston2026

Appears on: /domains/cases/va-claims-automation

The Office of Inspector General reported that VA's internal correction channels did not catch the automated survivor-benefit deficiencies and that the external audit was, empirically, the only channel that changed behavior. In April 2020 a VBA analyst reported through the internal defect-tracking system that automated decisions listed only the death certificate as evidence, and the Pension and Fiduciary Service closed the defect without action; the same deficiency was central to the 2026 findings, and VA removed the long-form guidance from its manual only in March 2025, immediately after the OIG's preliminary briefing - roughly five years later, and the OIG's full public report did not follow until 2026, roughly six years after the ticket. The OIG found the quality-review checklist for automated claims was less rigorous than the review traditional claims receive, and that the PACT Act section 701(b) modernization plan to Congress did not fully disclose that VBA grants these claims end to end without human intervention. Errors persisted as the program expanded: the VA Secretary announced expanded DIC automation in May 2025, and 20 additional automated decisions from September and October 2025 showed similar errors as of November 2025, with one recommendation still open and VBA concurring only in part.

Sources: departmentofveteransaffairso2026

Appears on: /domains/cases/va-claims-automation

In Trelleborg, Sweden, the first municipality to fully automate social-assistance decisions, peer-reviewed analysis reports that about 30 percent of digital reapplications are decided entirely by rules-based software with no human review and about 85 percent receive at least partial automated handling; decision time on reapplications fell from roughly two days to under a minute, and a human caseworker re-enters the path only by exception, when a routing rule detects significantly changed circumstances, a missing activity plan or job-seeking documentation, or a complex or negative case. No error, override, exception-routing, or appeal-rate figures for the automated path have been published, so the fraction of automated decisions that ever reaches a human cannot be established from the record.

Sources: algorithmwatch2020b, ranerupandhenriksen2022, europeancommissionjointresea2021

Appears on: /domains/cases/trelleborg-rpa

The City of Amsterdam spent roughly five years and an estimated EUR 535,000 building a deliberately fair, explainable welfare-fraud screening model with nearly every recommended pre-deployment safeguard in place - a bias audit, training-data reweighting that approximately equalized wrongful-flag rates on retrospective data, a data-protection assessment and a human-rights assessment, external and academic review, a citizen panel, and dual algorithm-register transparency - and discontinued it after a 2023 live pilot on nearly 1,600 applications. In the investigating journalists' analysis of aggregate data the city provided, the group disparities re-emerged inverted on the live pilot, now more likely to wrongly flag Dutch nationals, women, and applicants with children, with the tool flagging more applications than the analog process and no better than caseworkers at finding genuine cases. The Dutch national algorithm register records the deployment ending September 2023 and lists it out of use, and the responsible alderman announced the halt in November 2023.

Sources: braun2025, lighthousereports2025b, algoritmeregisterdutchnation2023

Appears on: /domains/cases/amsterdam-slimme-check

In a developer-reported randomised controlled trial of more than 1,000 adviser support requests, an adviser-facing benefits copilot at Citizens Advice returned supervisor-checked answers in about four minutes, roughly half the previous response time, with about 80 percent of its drafts approved by supervisors without revision; these figures are reported by the tool's builders and have not been independently replicated.

Sources: varotsis2025, departmentforscience2025a, stanfordlegaldesignlab2025a

Appears on: /domains/cases/caddy-citizens-advice

Advisers given access to the copilot were reported to be more than twice as likely to say they felt confident giving advice than a control group, a self-reported measure from post-call in-chat surveys rather than a client-outcome or accuracy measure.

Sources: varotsis2025, stanfordlegaldesignlab2025b

Appears on: /domains/cases/caddy-citizens-advice

In 2023 Singapore's GovTech began a whole-of-government retire-and-replace of its scripted Ask Jamie chatbots, embedded since 2014 on 70-plus (a vendor case study claims 80) agency websites as independent per-agency answer engines, migrating government chatbots onto centrally provided large-language-model engines; the stated aim was to convert all 88 chatbots and retire the scripted engine by end 2023, the verified snapshot is 21 of 88 converted as of September 2023 (migration completion not independently documented), and by the VICA product page updated 29 April 2026 the successor platform hosts over 100 chatbots for 60-plus agencies at an average of over 800,000 monthly queries, figures that are all government self-reported.

Sources: hirdaramani2023, govtechsingapore2026, govtechsingapore2019

Appears on: /domains/cases/singapore-chatbot-fleet-refresh

The Singapore government benefits-navigation surface is documented as scope-limited to information and estimates rather than adjudication: the Ministry of Finance Support For You Calculator turns self-declared inputs into benefit estimates that are explicitly estimates and not entitlement decisions, and the Chat.Gov.SG (Beta) explainer hosted on the SupportGoWhere domain states the assistant summarises information from official government websites and does not assess eligibility, make decisions, submit applications, or complete transactions, and warns users not to share personal or sensitive information.

Sources: publicservicedivisionsingapo2026, mustsharenews2024

Appears on: /domains/cases/singapore-chatbot-fleet-refresh

A June 2026 Treasury Inspector General for Tax Administration performance audit (Report Number 2026-308-029) reported that the IRS expanded its Automated Collection System chatbot and live-chat program and made live chat permanent while having no performance measures for it, despite a Taxpayer First Act requirement for metrics and benchmarks, and that management's claim the bots reduced telephone demand could not be substantiated; the statistical reports the IRS did collect were deemed unreliable, in one instance showing a single assistor apparently working 603 chats at once against a systemic cap of three, attributed partly to a miscalculated handle-time metric the vendor had not resolved as of December 2025.

Sources: treasuryinspectorgeneralfort2026, bracken2026, bramwell2026

Appears on: /domains/cases/irs-acs-chatbots

In the same audit, of a judgmental sample of 40 IRS ACS live assistors, 24 (60%) were found working multiple chats concurrently and 12 of those 24 had at least one authenticated chat open while working another, which TIGTA reported as raising the risk of disclosing taxpayer information to the wrong taxpayer; the audit also reported 635,684 resolution codes against 613,056 chats (a mismatch management knew of but did not investigate) and, in March 2025 hand-testing, 14% of chatbot process flows deficient and 83% of tested keywords unrecognized or insufficient, with the figures drawn from a nonprobability sample and data the audit itself characterized as unreliable and not projectable to the full assistor population.

Sources: treasuryinspectorgeneralfort2026, bramwell2026, cohn2026

Appears on: /domains/cases/irs-acs-chatbots

In a spring-2025 randomized pilot inside a large consumer EBT app, the vendor reports that 53% of eligible SNAP recipients took up in-app AI help for missed deposits and that treated users were restored faster and more often in the same month than a control group, with every AI dead-end escalated to a named human; all outcome figures are vendor-published and the effect magnitudes were not disclosed.

Sources: propelincpropelinsights2025b, guarino2025a

Appears on: /domains/cases/propel-snap-assistant

By the vendor's own account of the design, the assistant grounds on a state-verified deposit record it reads but does not write to, and steers recipients to act on the state system of record rather than acting for them.

Sources: propelincpropelinsights2025b

Appears on: /domains/cases/propel-snap-assistant

Between 2019 and 2025 more than 70% (about 73% per its ten-year retrospective) of California's online SNAP applications were submitted through GetCalFresh, a deterministic, structured-workflow application assister built and operated by the nonprofit Code for America, which reports helping 6.2 million people obtain more than $12.8 billion in food benefits from 2017 to 2025 (organization-published figures that are not independently audited); the node made no eligibility determinations, and in 2024 and 2025 the California Department of Social Services coordinated a dated, phased transfer of its functions into the state-owned BenefitsCal portal.

Sources: codeforamerica2024b, codeforamerica2025a, californiadepartmentofsocial2025

Appears on: /domains/cases/getcalfresh

A randomized controlled trial of roughly 65,000 Los Angeles GetCalFresh applicants (Giannella, Homonoff, Rino, and Somerville, American Economic Journal: Economic Policy 16(4), 2024) found that access to applicant-initiated flexible interviews increased SNAP approvals by about 6 percentage points, doubled early approvals, and raised long-term participation by over 2 percentage points, identifying the intake interview as a key procedural-denial barrier; Code for America separately reported an in-house experiment lifting renewal-form submissions among prior non-responders from about 1.5% to roughly 12% (organization-published, without sample sizes or confidence intervals).

Sources: giannella2024, codeforamerica2021, codeforamerica2024a

Appears on: /domains/cases/getcalfresh

In June 2024 the board of Benefits Data Trust, a Philadelphia benefits-navigation nonprofit that reported helping more than 120,000 people access about $182 million in benefits in 2023, voted unanimously to wind the organization down within a self-imposed 60-day window, citing only 'a perfect storm of circumstances'; the organization closed on August 24, 2024, laying off 273 employees, despite roughly $12 million in unrestricted reserves at the end of 2023 and about $32 million in projected 2024 revenue.

Sources: brubaker2024a, brubaker2024b, wink2024, mosbruckergarza2024

Appears on: /domains/cases/benefits-data-trust-winddown

The closure left active government partnerships without a designated successor, including a Pennsylvania Department of Aging workload of nearly 48,000 applications from 27,018 households in the final year and a Philadelphia BenePhilly call-center contract the organization was reported to be exceeding through mid-2024; the navigation function fragmented to higher-friction channels, with the work redistributed across partner agencies and a subcontractor and referral waits reported as several months, which a Pew analyst described as a 'cascading effect.'

Sources: brubaker2024d, burnley2024, mosbruckergarza2024

Appears on: /domains/cases/benefits-data-trust-winddown

London's Strategic Insights Tool for Rough Sleeping probabilistically links records from three separately governed systems - CHAIN street-outreach contacts, In-Form charity casework, and H-CLIC borough statutory applications - into a single rough-sleeping journey per person that is read, in aggregate form only, across all 33 London local authorities; the tool makes no individual-level determinations, and after the build vendor's data-processor contract ended on 2 February 2024 the Greater London Authority contracted Homeless Link, which also operates the CHAIN source system, for its ongoing hosting, management, and maintenance.

Sources: techuk2024, loti2023, londonofficeoftechnologyandi2023

Appears on: /domains/cases/london-rough-sleeping-sit

The Strategic Insights Tool's matcher accepts an association only above an 85% probability threshold chosen to minimise false positives, and the project's own Phase 2 Data Protection Impact Assessment reports 91% recall - conceding that roughly 9 in 100 true cross-system matches are missed so that 'numbers subsequently appear lower in places where they should be higher' and that recall varies as new data of varying quality is ingested; no false-positive rate is published, the accuracy figures are self-reported by the delivery team, and no independent evaluation of the tool's decision impact exists.

Sources: loti2023, lotiannahumplebyandfacultyja2025

Appears on: /domains/cases/london-rough-sleeping-sit

San Jose's vehicle-mounted computer-vision pilot, described by city officials and national housing advocates as the first US experiment training AI to recognize tents and lived-in vehicles, reported sharply class-asymmetric accuracy in the city's own staff-ground-truthed evaluation — 97% for potholes and 88% for trash, but only 70% for RVs (unable to distinguish a lived-in RV from an empty one) and 12.5% for lived-in vehicles, with a March 2024 official interview bracketing the habitation figures at 70–75% for RVs and 10–15% for lived-in cars against a 70% goal; no detection ever generated an operational dispatch, and after investigative exposure and structured engagement the city removed every habitation-detection use case, its March 2025 status report declining to recommend implementing AI object detection in city operations at this time.

Sources: feathers2024, cityofsanjoseinformationtech2025, usdepartmentoftransportation2025

Appears on: /domains/cases/san-jose-encampment-detection

The pilot's published data-usage protocol declares that the footage cannot be actively monitored for law-enforcement purposes while preserving a police request path to it — verbatim, 'Law enforcement may request access to previously stored footage. Law enforcement is not actively monitoring any data collected' — and requires de-identification or deletion within one month; the CIO stated data was not shared with police during the pilot, yet public-records reporting documented that one vendor's system ran optical character recognition of license plate numbers despite the city's no-identification claim, so the declared authority rule and the feasible data flows diverged, a gap surfaced by journalists rather than by any standing audit, and the no-law-enforcement-use clause is city protocol language rather than statute.

Sources: feathers2024, cityofsanjoseinformationtech2024, varian2024

Appears on: /domains/cases/san-jose-encampment-detection

The Los Angeles Coordinated Entry System replaced the VI-SPDAT survey for single adults with the Los Angeles Housing Assessment Tool, a 19-item self-report score whose weights were derived by a regression on 71,747 historical assessments linked to county records; where the CESTTRR research estimated the VI-SPDAT scored near chance (AUC 0.54) with racial false-negative gaps up to 8.5 percentage points, the equity-adjusted successor was deliberately traded down in overall accuracy (AUC 0.60, from an accuracy-only 0.64) to close those gaps to under one percentage point, and every such figure is a pre-deployment estimate on 2015 to 2018 held-out data rather than an observed post-launch outcome.

Sources: rice2023, losangeleshomelessservicesau2025a

Appears on: /domains/cases/lahsa-triage-revision

During the dual-tool transition the two instruments' PSH-consideration thresholds were 8-plus on the VI-SPDAT and 17-plus on the LA HAT, and by LAHSA's account initial quantitative data and provider feedback showed participants were more likely to obtain an eligible score under the VI-SPDAT, so direct-service providers opted to administer it, a trend LAHSA states 'perpetuated the racial bias of the VI-SPDAT in the System'; on April 22, 2026 the CES Policy Council lowered the LA HAT threshold to 12-plus, ruled the most recent LA HAT score supersedes a coexisting VI-SPDAT score, and forced deactivation of new VI-SPDAT completions (for LA HAT-access programs on May 1, 2026 and system-wide on June 30, 2026), though LAHSA has not released the underlying eligibility-rate numbers.

Sources: losangeleshomelessservicesau2025a, losangeleshomelessservicesau2026b, losangeleshomelessservicesau2026a

Appears on: /domains/cases/lahsa-triage-revision

In a registered randomized controlled trial of 1,263 imminent-risk applicants (514 treatment, 749 control) run by the University of Notre Dame's evaluation lab, households offered flexible emergency financial assistance averaging about 2,000 dollars, typically one to two months of back rent, through Santa Clara County's homelessness-prevention system were reported 81 percent less likely to become homeless within six months and 73 percent within twelve months; the peer-reviewed article's abstract states the assistance reduced homelessness by 3.8 percentage points from a 4.1 percent base rate, and the researchers conservatively estimated 2.47 dollars in community benefits per net dollar spent.

Sources: phillipsandsullivan2025, universityofnotredamenews2023, phillipsandsullivan2021

Appears on: /domains/cases/santa-clara-prevention

Because becoming homeless is statistically rare even among at-risk applicants - about 96 percent of the trial's control group never became homeless without assistance - the program's own co-author cautions that prevention resources can flow to households that would have stayed housed anyway, making screening precision on a low base rate the binding constraint; as of February 2026 the model is being replicated across about ten heterogeneous US jurisdictions under a 77-million-dollar initiative, with the same evaluation lab as the common evidence partner assessing each site.

Sources: kendall2026a, phillipsandsullivan2025, destinationhome2026b, universityofnotredamenews2026

Appears on: /domains/cases/santa-clara-prevention

At the Calgary Drop-In Centre, a University of Calgary engineering group and the NGO shelter operator built deliberately interpretable screening for chronic and episodic shelter use - explicit stay-count thresholds (for example 81 or more stays in a 90-day window) and database-queryable rules derived from the shelter's own administrative records, reported to flag candidate clients at a median of about 98 days versus 285 days under the Government of Canada definition and 365 under the Alberta definition - and, rather than surface a risk score, deployed a co-designed data-navigation interface that shows frontline staff raw client histories; no fetched source confirms the thresholds running as an automated production screener, and the deployed, studied artifact is the raw-history interface.

Sources: messier2021, arulesearchframeworkfortheea2022, masrani2025

Appears on: /domains/cases/calgary-drop-in-shelter-ml

Across a 2022 to 2024 embedded deployment study of the interface (16 staff across 7 role categories; 29.5 hours of qualitative data; five committee observations; three deployed versions), the participant-research team documented a stakes-dependent 'data-outsourcing continuum': staff were reluctant to outsource high-stakes barring decisions, treating the data as a starting point for collaborative discussion, while reporting more willingness to accept automated data-driven recommendations for lower-stakes housing triage; the finding is the staff's own articulated practice rather than a measured override or agreement rate, all deployment evidence is authored by the embedded research team, and no independent evaluation, usage logs, or decision volumes are published.

Sources: masrani2025, thehumanbehindthedatareflect2023

Appears on: /domains/cases/calgary-drop-in-shelter-ml

An AI quality-assurance tool deployed on a national 988 backup line scores crisis counselors' own call practice rather than callers, expanding measured review from the under-3% of calls that had been reviewed by hand toward nearly all of them; a peer-reviewed reliability study of 476 labeled calls reported agreement with human ratings at 98 percent of human interrater agreement for detecting any risk assessment, with average F1 of about 0.86 at call level and 0.66 at statement level, and its authors include four holders of equity in the vendor.

Sources: imel2024, aguilar2023, nihreporternationalinstitute2025

Appears on: /domains/cases/lyssn-protocall-988

The registered randomized crossover trial of the tool's counselor feedback (81 call-takers) completed on October 31, 2025, but as of mid-2026 no results were posted to the trial registry or found in the peer-reviewed literature and participant-level data were marked unavailable for proprietary reasons, so reported counselor-skill-improvement effects remain vendor claims pending independent publication.

Sources: clinicaltrialsgovusnationall2026, lyssn2026

Appears on: /domains/cases/lyssn-protocall-988

Gaggle's student-communication safety monitoring, used by roughly 1,500 US districts covering about 6 million students as of a March 2025 AP and Seattle Times investigation, scans school-issued accounts around the clock and routes flags through a multi-hop chain (a machine flag, an off-site vendor reviewer, district safety staff, and, for imminent-danger after-hours alerts, occasional police welfare checks); in Vancouver Public Schools nearly 2,200 students (about 10% of enrollment) triggered alerts in one year, in Lawrence USD 497 more than 1,200 incidents were logged in ten months with about two-thirds deemed nonissues by officials (a figure the plaintiffs drew from district records), and the archive of flagged documents was accidentally released to reporters as nearly 3,500 unredacted files through unprotected links, while a 2023 RAND review found only scant evidence of either benefit or risk and the vendor publishes no accuracy figures.

Sources: bryanandlurye2025, associatedpress2025, lawrencejournalworld2025

Appears on: /domains/cases/gaggle-school-monitoring

After nine Lawrence, Kansas students sued their district in early August 2025 over its use of AI communication monitoring, court filings revealed the district had ceased using Gaggle mid-litigation and substituted a different monitoring vendor with no board vote or public disclosure — surfacing only as a line in a check register — and the plaintiffs' amended complaint argued the swap does not moot the case because the core practice of suspicionless scanning, flagging, and seizure of student speech continues; on April 10, 2026 a federal judge found the district violated the Kansas Open Records Act in withholding the substitution and phase-out records, and on June 4, 2026 ordered it to pay the students' attorney fees, characterizing the conduct as drawn out, hollow and perplexing, with a jury trial on the surviving constitutional claims set for January 2027.

Sources: heimsoth2025, heimsoth2026a, heimsoth2026b

Appears on: /domains/cases/gaggle-school-monitoring

In March 2026 the UK Parliamentary and Health Service Ombudsman partly upheld a complaint that an NHS mental health trust installed camera-based, contact-free bedroom monitoring on a psychiatric ward without seeking a patient's consent, gave her no information about it, and did not switch it off when she asked; the case documentation and investigative reporting describe an internal clinical evaluation that the vendor is reported to have authored the business case for and shaped, a rebrand of the vendor during a statutory inquiry, and an open data-protection investigation, while the tool's own outcome-reduction figures are vendor claims contested by a campaign-linked meta-analysis and its adoption share across NHS mental health trusts is reported only as a contested range.

Sources: parliamentaryandhealthservic2026, williamson2026a, williamson2026b, nationalsurvivorusernetwork2025, stopoxevision2026

Appears on: /domains/cases/oxevision-nhs-wards

The ombudsman's report on the case (decision 27 March 2026) found the trust did not seek or revisit the patient's consent for the bedroom monitoring, did not turn the camera off when she asked, gave her no information about it, and kept no record of how staff used it, and that even the trust's revised 2025 procedure still permits overriding a capacitous patient's refusal on clinically-safe grounds with multidisciplinary-team approval; on the separate question of over-reliance the ombudsman found on balance, cross-referencing observation charts, a nurse-adviser review and door key-card data, that in-person observations had continued and did not uphold that part of the complaint.

Sources: parliamentaryandhealthservic2026

Appears on: /domains/cases/oxevision-nhs-wards

In 2025 ODMAP's pre-set county thresholds - a rolling 24-hour count against a threshold each agency sets or accepts, recommended by the system as two standard deviations above the county's own previous 90-day mean, a deterministic rule rather than a machine-learning model - fired 74,805 advisory spike-alert notifications from 498,003 suspected, unconfirmed overdose events that only about 1,362 of its 5,605 approved agencies actually submitted; these figures are self-published by the program in its own annual report and manuals, and ODMAP states its data are suspected, incomplete, not a system of record, and should not be generalized beyond participating agencies.

Sources: washingtonbaltimorehidta2025b, washingtonbaltimorehidta2026c, washingtonbaltimorehidta2025a

Appears on: /domains/cases/odmap-overdose-spike-alerts

ODMAP's shared overdose store is housed inside a federal drug-enforcement program, and its operating policies both state that ODMAP is neither an intelligence sharing database nor a pointer index records system and grant the host permission to use the data as the HIDTA sees fit, including combining it with other databases it manages for law enforcement and public health products; a 2024 peer-reviewed stakeholder study documented divergent public-health versus public-safety data-privacy standards, and a 2025 peer-reviewed analysis argues the integration risks racialized surveillance and criminalization of people who experience overdose, a contested scholarly critique of the link structure rather than a documented misuse incident.

Sources: washingtonbaltimorehidta2022, syvertsen2025, allen2024

Appears on: /domains/cases/odmap-overdose-spike-alerts