Other534

An independent audit of the Allegheny Family Screening Tool's first years (2016-2018) found that, run without human override, it would have recommended screening in about 68% of Black children versus 50% of white children (an 18-point gap), while call screeners actually screened in 51% and 43% (a 7-point gap) — the narrower gap came from workers disagreeing with the score about a third of the time.

Sources: stapleton, stapleton2025, hoandburke2022

Appears on: /domains/cases/allegheny-afst, /pan-lab

An ACLU and Human Rights Data Analysis Group analysis of the Allegheny Family Screening Tool found that 97% of Black referral-households in the data were affected by at least one permanent 'ever-in' variable drawn from public-benefits data sources, compared with 80% of non-Black households.

Sources: gerchicketal2023

Appears on: /domains/cases/allegheny-afst, /pan-lab

The U.S. Department of Justice's Civil Rights Division was reported to be scrutinizing the Allegheny Family Screening Tool after civil-rights complaints filed in fall 2022 raised concerns that its use of disability, mental-health, and Supplemental Security Income data may discriminate against parents with disabilities; families are not shown their scores, and no public findings or enforcement have been reported.

Sources: associatedpress2023, hoandburke2023

Appears on: /domains/cases/allegheny-afst

In Allegheny County's Hello Baby program, the top-tier roughly 5% of newborns by predictive risk score accounted for about 54% of children later removed from the home by age three, at roughly twenty times the removal risk of other newborns (methodology relative risk 22.24, 95% CI 17.50-28.25); the model reported an AUC of about 0.93 on holdout data.

Sources: centreforsocialdataanalytics2020, vaithianathan2025

Appears on: /domains/cases/allegheny-hello-baby

An external evaluation of Hello Baby covering birth cohorts 2016-2024, controlling for COVID-19, found the program associated with fewer first child-maltreatment investigations and first substantiated investigations, but no reduction in out-of-home foster-care placements - the outcome the predictive model was built to estimate.

Sources: lery2025, centreforsocialdataanalytics2020

Appears on: /domains/cases/allegheny-hello-baby

In a 2019 proof of concept, Chile's Sistema Alerta Niñez risk models reached test-set AUC of roughly 0.88 to 0.95 for a two-year outcome — a child's separation from family or contact with child-protection programs — using 280 administrative variables per child; the deployed operational model's real-world performance was never publicly disclosed.

Sources: derechosdigitalesmatiasvalde2021, derechosdigitalesmatiasvalde2022, centreforsocialdataanalytics2019b

Appears on: /domains/cases/chile-alerta-ninez, /pan-lab

Sistema Alerta Niñez drew on 280 administrative variables that families had supplied to access social benefits, without informed consent to the risk ranking or a way to opt out; the model's developers acknowledged it was less able to identify higher-income children at risk, because lower-income families have more contact with the state.

Sources: derechosdigitalesmatiasvalde2021, derechosdigitalesmatiasvalde2022, centerforhumanrightsandgloba2022

Appears on: /domains/cases/chile-alerta-ninez, /pan-lab

The Douglas County Decision Aide, deployed into the county's RED-Team call-screening process in February 2019, scores each referral from 1 to 20 for a child's likelihood of out-of-home removal within two years; an independent Cornell-led randomized controlled trial found it sped up screening decisions without significantly changing child outcomes, and a companion study found workers attended mainly to extreme scores while largely disregarding mid-range ones.

Sources: vaithianathanetalcentreforso2019, fitzpatrick2025, eiermann2026

Appears on: /domains/cases/douglas-county-decision-aid, /pan-lab

Eckerd's Rapid Safety Feedback spread from Hillsborough County, Florida to child-welfare agencies in several states — promoted on the vendor's own reported gains and highlighted as “innovative” in a 2016 federal commission report — years before an independent 2022 peer-reviewed evaluation found the process did not lower repeat high-severity maltreatment among children identified as high risk (a joint odds ratio of about 1.05).

Sources: eckerdconnects2016, routefiftygovernmentexecutiv2016, parker2022, floridaschildrenfirst2012

Appears on: /domains/cases/eckerd-florida-rsf-origin

Gladsaxe's early-detection project (DTO) was a decision-tree model over about 44 risk indicators, meant to score, for every child aged 0 to 6 rather than only families already receiving help, the estimated probability that the child was living in vulnerability; per a university-run Danish public-sector AI catalogue it was to be trained on roughly 173,000 notifications the authorities received between April 2016 and December 2017, but only about 117 usable historical cases existed, and it was halted in its development phase in 2019 without ever running on live decisions, after a national media storm and an unrelated data breach that exposed about 20,000 citizens' personal identification numbers.

Sources: offentligaiuniversityrundanind, kennethkristensensamfundsled2022, helenefriisratnerandkasperel2023, katarinafastlappalainen2021, tvkosmopolformerlytvlorry2018

Appears on: /domains/cases/gladsaxe-denmark

Hackney paid the analytics firm Xantura £361,400 over four years to run an Early Help Profiling System that flagged families for preventive intervention from council data, but scrapped the pilot in 2019 after finding that, despite flagging about 350 families, it surfaced only 7 children previously unknown to the council and the available data was too limited and variable to justify continuing.

Sources: hackneycouncilpayskpoundstod2018, townhalldropspilotprogrammep2019

Appears on: /domains/cases/hackney-early-help

Families whose data Hackney's Early Help Profiling System processed were not informed directly: reporting describes families profiled without their knowledge, given notice only through a general online privacy notice, with no option to opt out recorded in the system's impact assessment and the method withheld as commercially sensitive; the council argued that disclosing the system could prejudice potential interventions.

Sources: townhalldropspilotprogrammep2019, reddenj2020, hackneycouncilpayskpoundstod2018

Appears on: /domains/cases/hackney-early-help

Internal DCFS tracking data released under Illinois public-records law showed the Rapid Safety Feedback tool flagged more than 4,100 children at a 90-percent-or-higher probability of death or serious injury within two years, including 369 children under age 9 assigned a 100-percent probability, while children who died in cases already known to the system — among them 17-month-old Semaj Crosby, found dead after at least ten DCFS investigations — were not flagged as top-risk; the roughly $366,000 program was ended in 2017.

Sources: chicagotribune2017, governmenttechnologya

Appears on: /domains/cases/illinois-rapid-safety-feedback

Illinois brought in the Eckerd/MindShare Rapid Safety Feedback program under DCFS director George Sheldon through a no-bid arrangement the state classified as a grant; a July 2017 joint report by the Illinois Office of Executive Inspector General and the DCFS Inspector General found this classification to be mismanagement because it avoided state bidding-transparency requirements.

Sources: chicagotribune2017, sunshinestatenews2017

Appears on: /domains/cases/illinois-rapid-safety-feedback

Bristol's Think Family Database drew on roughly 30 to 35 fused council, police and other datasets covering about 55,000 families (some 170,000 residents in 2021 reporting), and its child sexual and criminal exploitation risk models were quietly withdrawn in 2023 as 'not fit for operational use' after an independent evaluation judged the risk-scoring models the weakest element and staff reported victims of exploitation scoring below people involved in burglary; FOI responses indicate no record was kept of why the models were switched off, and auditors could not locate their source code or variable lists.

Sources: seanmorrison2026a, markwildingandmattburgess2026, seanmorrison2026b, bristolcitycouncil2025, jakehurfurtbigbrotherwatch2021

Appears on: /domains/cases/insight-bristol

Reporting and FOI responses on Bristol's Think Family Database indicate the exploitation models' source code and variable lists could not be located when auditors sought them, and that an ethics committee advising the police analytics reportedly did not revisit the analytics after 2017; a 2021 review warned that data gathered through 'legal gateways' meant 'legality is not the same as legitimacy.'

Sources: markwildingandmattburgess2026, seanmorrison2026a, seanmorrison2026b

Appears on: /domains/cases/insight-bristol

In a retrospective test against historical outcomes, Los Angeles County's Project AURA — a proprietary risk model built by SAS — correctly flagged 171 of the highest-risk children but produced 3,829 false positives, a false-positive rate of about 95.6% that DCFS's own public-affairs director confirmed on the record, and the county shelved the tool in 2017 without ever using it on a live case.

Sources: theimprintdanielheimpel2015, childprotectiveservicesdefen2015, nccprrichardwexler2017, witnesslarichardwexler2017

Appears on: /domains/cases/la-county-aura, /pan-lab

The Dutch government's own 2011 pilot evaluation of ProKid found that 36% of the tool's red, orange and yellow child-risk flags (902 of 2,444 over three months across four police regions, rising to 53% in Amsterdam-Amstelland) were system or registration errors or based on irrelevant incidents, and that in none of the four regions was there a well-functioning instrument.

Sources: dspgroepforthewodcabraham2011, dimitritokmetzissargasso2012

Appears on: /domains/cases/netherlands-prokid, /pan-lab

New Zealand's Ministry of Social Development commissioned a child-maltreatment risk-modelling tool that, on a 2012 development sample of 57,986 children and 132 selected variables, reported an area under the ROC curve of 76% and a top risk decile in which 47.8% had a substantiated maltreatment finding by age five; those figures come from development data rather than field performance, the tool was never operationally deployed, and a proposed two-year study that would have scored about 60,000 newborns was halted by the incoming Social Development Minister, who annotated the briefing papers 'Not on my watch! These are children not lab rats.'

Sources: vaithianathan2013, nzherald2015, otagodailytimes2015, mordaunt2026

Appears on: /domains/cases/nz-msd-prm

Oregon's Department of Human Services stopped using its Safety at Screening tool at the end of June 2022 and replaced it with a non-algorithmic Structured Decision Making process, telling staff the change was meant to reduce disparities; the move followed Associated Press reporting on racial disparity in the Allegheny tool it was derived from and a racial-bias inquiry from a U.S. senator.

Sources: associatedpress2022, willametteweek2022, hoandburke2022, nprap2022

Appears on: /domains/cases/oregon-safety-at-screening

Oregon's 2019 report describes a post-processing fairness correction — group-specific thresholds selected under an 'error rate balance' criterion — applied to a dual-outcome risk model built only on the state's own child-welfare administrative records.

Sources: orrai2019, associatedpress2022

Appears on: /domains/cases/oregon-safety-at-screening

None of the 32 machine-learning models What Works for Children's Social Care built across four English local authorities cleared the pre-specified 65% average-precision success bar; the best single model reached only about 42% average precision and, at an operating point, missed roughly 79% of the children whose cases actually escalated.

Sources: claytonandsanders2022, communitycareturner2020a, childhubterredeshommes2020

Appears on: /domains/cases/wwc-uk-ml-pilots, /pan-lab

In a survey of 129 social workers carried out for the project, only about 26% supported using predictive analytics to identify families for early help and about 34% thought it should not be used at all.

Sources: communitycareturner2020a

Appears on: /domains/cases/wwc-uk-ml-pilots, /pan-lab

In a Los Angeles County pilot, 335 people who enrolled in the voluntary Homelessness Prevention Unit were reported to be 71% less likely than a regression-adjusted comparison group of 1,285 eligible non-enrollees to enter a homeless shelter or have street-outreach contact within 18 months; the California Policy Lab describes this as an association not yet shown to be causal, pending a randomized controlled trial with results expected in 2027.

Sources: blackwell2025, countyoflosangeles2025, uclanewsroom2025

Appears on: /domains/cases/la-homelessness-prevention

The Homelessness Prevention Unit's own November 2024 equity audit, on a test population of 47,582 individuals eligible to be scored, reported false-negative rates ranging from about 56% for Black individuals to roughly 63 to 65% for other groups: the model misses a majority of the people who later become homeless, while performing roughly consistently across race, ethnicity, and gender and identifying Black individuals slightly more strongly.

Sources: californiapolicylab2024, foxsowell2025

Appears on: /domains/cases/la-homelessness-prevention

Xantura's OneView integrates more than 15 multi-agency data feeds into a single household view and flags residents as likely to become homeless months ahead. In Maidstone's pilot year it produced 650-plus alerts that a single financial-inclusion officer could contact only about 260 of. Its headline effectiveness figures - a reported 40 percent fall in homelessness, savings and an ROI over 600 percent, and the widely quoted contrast between contacted and uncontacted households - are vendor- and council-reported pre/post numbers from one COVID-affected pilot year; the contact-versus-no-contact contrast reflects capacity-driven selection rather than a randomised comparison, and the independent randomised controlled trial commissioned to test the causal claim was still in progress into 2026.

Sources: crisisuk2023, xantura2023, governmenttransformationmaga2023, ministryofhousing2024, centreforhomelessnessimpact2024

Appears on: /domains/cases/xantura-oneview-housing

OneView's single view of vulnerability is built by integrating sensitive multi-agency records - including offending, health, benefits and debt data - under a statutory Digital Economy Act 2017 data-sharing agreement with named public-body controllers and processors. An independent ethnography of an early deployment (its fieldwork centered on children's social care and the COVID-19 response) found frontline staff could not see which factors drove the tool's alerts and were not all convinced it was as accurate as described, and a separate NGO investigation characterised the vendor's COVID-era model as operating without residents' knowledge.

Sources: digitaleconomyactregister2023, adalovelaceinstitute2024, bigbrotherwatch2021

Appears on: /domains/cases/xantura-oneview-housing

London, Ontario's CHAI is a live, caseworker-facing machine-learning model that flags people in the city's shelter system as at risk of chronic homelessness (more than 180 shelter days in a year) about six months ahead; it provides intelligence to prevention caseworkers and does not itself make service decisions. Its widely repeated '93 percent accuracy' is a builder-reported, testing-phase figure from 10-fold cross-validation on historical HIFIS records, never independently validated after deployment; the same technical work reports recall of about 0.921 but precision of only about 0.651, implying substantial false positives under a low base rate.

Sources: wray2020, vanberlo2009, lebel2023, govlaunchstories2020

Appears on: /domains/cases/chai-london-ontario

CHAI is consent-based: it draws on de-identified HIFIS records pooled from roughly 20 to 24 London homelessness-support organizations and lets individuals opt out of inclusion, and it was built with reference to GDPR principles, Canada's Directive on Automated Decision-Making, and local feature-attribution explanations for caseworkers. Because HIFIS captures people who use public shelters, an independent review and reporting at launch note it can under-represent or miss groups who avoid them - including many women, families, new immigrants, some Indigenous people, and private-shelter users; academic researchers situating the tool raise related fairness and inequality concerns. So the population the model can score is a selected sample of actual need, and the opt-out self-selects it further.

Sources: wray2020, lebel2023, lamberink2020, redden2026

Appears on: /domains/cases/chai-london-ontario

In a 2025 Los Angeles County pilot evaluated by Nava Labs with academic partners at Cornell University and Georgetown University's Better Government Lab, a generative-AI assistive chatbot for Imagine LA's Benefit Navigator was estimated to improve benefits-navigation answer accuracy by an average of about 40% in a randomized controlled trial of 125 caseworkers answering hypothetical client questions, alongside a fourteen-week field pilot with 61 caseworkers across six organizations; the evaluation was co-authored by the tool builder rather than independently replicated, the accuracy figure is a decision-support contrast on hypothetical questions rather than a live-caseload eligibility audit, and time-savings and administrative-burden effects were reported as promising but inconclusive (published March 2026).

Sources: navapublicbenefitcorporation2026, chen2026

Appears on: /domains/cases/imagine-la-benefit-navigator

The same evaluation reported that the chatbot's accuracy gains were largest on the most difficult client questions and among the newest, least-experienced staff (a directional finding, not a quantified breakdown), that about 65% of caseworkers with access used it at an average of about 14 prompts each and a modest, low-positive satisfaction (a Net Promoter Score of 11), that usage tended to decline over time without sustained engagement, and that answers averaged a tenth-to-twelfth-grade reading level against college-level source manuals.

Sources: navapublicbenefitcorporation2026, chen2026

Appears on: /domains/cases/imagine-la-benefit-navigator

In a single-center randomized trial across three Vanderbilt neurology clinics (August 2022 to February 2023), an EHR suicide-risk model flagged 596 of 7,732 encounters (about 8%) at a 2%-or-higher 30-day-risk threshold; making the identical alert interruptive rather than passive led clinicians to elect a suicide-risk screen in 42% of encounters (121/289) versus 4% (12/307) for a passive chart icon, an adjusted odds ratio of 17.70 (95% CI 6.42–48.79). Screening remained fully advisory: about 58% of interruptive and 96% of passive alerts produced no screening.

Sources: walshetal2025, aitestedforalertingclinician2025, suicidepreventionmorefeasibl2025

Appears on: /domains/cases/vsail-vanderbilt

In a separate 2021 prospective silent-mode study (115,905 predictions on 77,973 patients, June 2019 to April 2020), the model reported a c-statistic of 0.797 for suicide attempt and 0.836 for ideation center-wide but only 0.544 for attempt in behavioral-health settings, and in the highest-risk quantile the number-needed-to-screen was 271 for attempt and 23 for ideation. In the 2022 to 2023 trial no suicidal ideation or attempts were documented in either arm during 30-day follow-up, and the trial was explicitly not powered for clinical outcomes, so it measured a process outcome (screening) rather than reduced harm.

Sources: walshetal2021, walshetal2025, suicidepreventionmorefeasibl2025

Appears on: /domains/cases/vsail-vanderbilt

Between roughly 2005 and 2019 the Dutch Tax Administration's benefits branch (Belastingdienst/Toeslagen) wrongly accused an estimated 26,000 or more families of childcare-benefit fraud and demanded full repayment; broader advocacy estimates run higher and count different populations, and by February 2026 about 69,000 people had applied to the recovery scheme and more than 43,000 were formally recognized as affected, each entitled to a minimum of 30,000 euros. A self-learning risk-classification model that scored applications using a Dutch-nationality indicator, a 270,000-person fraud blacklist (the FSV) held without a legal basis, and an all-or-nothing recovery regime were coupled together; the Dutch Data Protection Authority imposed 6.45 million euros in fines (2.75 million for the nationality processing in 2021 and 3.7 million for the FSV blacklist in 2022), a parliamentary inquiry found rule-of-law violations, and the third Rutte cabinet resigned on 15 January 2021.

Sources: wikipedia2026, autoriteitpersoonsgegevens2021, autoriteitpersoonsgegevens2022, amnestyinternational2021b, tweedekamerderstatengeneraal2020, rijksoverheid2026

Appears on: /domains/cases/nl-toeslagenaffaire, /pan-lab

The scandal's harm is best read as the coupling of three distinct components rather than a single algorithm. Government-commissioned technical reviews (KPMG in 2022 and PwC in 2023) described the tool as a self-learning classifier that routed the highest-scoring of roughly 90,000 benefit applications sent to manual treatment in 2014 to 2019, but judged the Dutch-nationality indicator's standalone predictive weight to have been limited; the model's precision and false-positive rate were never measured or published. The FSV fraud blacklist held frequently inaccurate data that was not corrected when people were cleared, and internal 2016 guidance auto-labelled childcare debts over 3,000 euros as intent or gross negligence, blocking payment arrangements. Out-of-home child placements are a documented but causally contested downstream harm: statistics counted roughly 2,090 children of affected parents placed out of home through mid-2022, while a 2025 judicial study found no child was removed solely because of financial problems.

Sources: kpmg2022, pwc2023, autoriteitpersoonsgegevens2022, statisticsnetherlandscbs2022, rechtspraak2025, wikipedia2026

Appears on: /domains/cases/nl-toeslagenaffaire, /pan-lab

On 5 February 2020 the District Court of The Hague ruled that the legislation authorising SyRI, the Dutch state's secret cross-database welfare-fraud risk-profiling system, violated Article 8 of the European Convention on Human Rights, and it ordered the system's use stopped; the State did not appeal. The ruling is widely described as one of the first times a court anywhere halted a digital welfare-fraud technology on human-rights grounds. Across its two executed neighbourhood projects SyRI was reported to have produced no confirmed fraud cases, and in one municipality 62 of 113 risk notifications were reported to be false positives.

Sources: districtcourtofthehague2020, vanbekkum2021, unofficeofthehighcommissione2020, algorithmwatch2020a, pontdataprivacyprivacywebnl2019, publicinterestlitigationproj2020

Appears on: /domains/cases/nl-syri

The District Court of The Hague found that the SyRI framework provided no duty to notify people that their data had been processed or that a risk report had been filed, so a flagged person generally could not know about, access, or contest the notification; notifications were retained in a register for up to two years. The court held that a risk notification carried significant effect for the person even though it lacked formal legal effect, and it faulted the scheme for a lack of transparency and for breaching data-minimisation and purpose-limitation principles.

Sources: districtcourtofthehague2020, vanbekkum2021

Appears on: /domains/cases/nl-syri

France's family-benefits fund (CNAF) computes a monthly benefit-fraud suspicion score, on a 0-to-1 scale, for every benefit-receiving household — analysing the data of about 32 million people and producing more than 13 million scores each month, close to half of France's population; the highest scores route households into fraud controls, up to the most invasive on-site checks. An analysis by Le Monde and Lighthouse Reports of an extracted production model (a logistic regression of about 33 variables) found that markers of economic vulnerability raised the score: a stable-income family averaged about 0.33, while a person working while receiving the disability allowance (AAH) averaged about 0.66. The model's target was an overpayment (indu) above a threshold, which is frequently unintentional administrative error rather than proven intentional fraud, and the score itself is not disclosed to the person and cannot be appealed directly. CNAF disputed the discrimination framing, describing the tool as a neutral decision-aid that only prioritises which files to check; a coalition that grew to 25 organisations challenged the model before the Conseil d'État, and as of this writing no court had ruled.

Sources: lighthousereports2023b, lighthousereports2023a, laquadraturedunet2023, laquadraturedunet2026a, amnestyinternational2024b, generationnt2026

Appears on: /domains/cases/france-cnaf

In an internal simulation study by CNAF's own statistics department (DSER), reported in October 2025 by Le Monde and La Quadrature du Net, recipients of the RSA minimum-income benefit were about 13% of beneficiaries but 39 to 41% of the highest-scoring 5%, and single mothers were about 14% of beneficiaries but 37 to 40% of that top bracket; households including a foreign national scored higher on average even after the nationality variable was removed. The full study is not public, and false-positive rates by protected group have not been released. The French ombudsperson (Défenseur des droits) told the Conseil d'État that a presumption of indirect discrimination appeared established because the differential treatment rests on beneficiaries' economic vulnerability; CNAF disputed the characterisation, and no court had ruled.

Sources: laquadraturedunet2026b, generationnt2026, laquadraturedunet2026a

Appears on: /domains/cases/france-cnaf

Analysing the Swedish Social Insurance Agency (Forsakringskassan) 2017 outcome data, Lighthouse Reports and Svenska Dagbladet reported on 27 November 2024 that the agency's in-house machine-learning risk profile for the temporary parental allowance (VAB) selected women (more than 1.5x), people of a foreign background (about 2.5x), below-median earners (2.97x), and people without a university degree (3.31x) for fraud investigation more often than comparison groups by demographic parity, and wrongly flagged those groups at higher false-positive rates (about 1.7x for women and 2.4x for people of a foreign background); in the agency's paired random-control sample, 20.2 percent of applications contained at least one day incorrectly paid, an unbiased base error rate. These are outcome computations under specific fairness definitions from a single obtained year of data, not confirmed model internals; the agency disputed the framing and did not release the model. The data-protection regulator IMY closed its GDPR supervision on 18 November 2025 for mootness after the agency withdrew the system, and no court or regulator issued a discrimination or GDPR penalty.

Sources: lighthousereports2024, lighthousereportsb, lighthousereportsc, integritetsskyddsmyndigheten2025a, integritetsskyddsmyndigheten2025b

Appears on: /domains/cases/sweden-forsakringskassan

The Swedish Social Insurance Agency (Forsakringskassan) did not disclose the machine-learning risk profile it used to select temporary-parental-allowance recipients for fraud investigation: its algorithm class, features, and precision were never released, and the agency resisted freedom-of-information disclosure for roughly three years on fraud-prevention grounds. In 2018 the audit inspectorate ISF found the risk-based profiling substantially more accurate than alternative controls while warning that it raised legal-certainty and equal-treatment concerns, and cautioning that an accurate model can still be inequitable when two groups err equally but only one is followed up. Amnesty International reported that a former agency data protection officer warned in 2020 that the operation breached European data-protection rules. The system was decommissioned in 2025 during the regulator's supervision, before any court or regulator ruled on it.

Sources: lighthousereports2024, lighthousereportsb, inspektionenforsocialforsakr2018a, inspektionenforsocialforsakr2018b, amnestyinternational2024d

Appears on: /domains/cases/sweden-forsakringskassan

Denmark's Udbetaling Danmark (UDK), administered by ATP, runs a data-driven welfare-fraud operation that as of 2019 used up to about 60 AI and machine-learning models to score benefit recipients into a 'wonderlist' of high-risk people, which a human control team filters into control cases for investigation. In UDK's own 2023 control statistics (three documented models), the 'Model Abroad' foreign-affiliation model sent 511 cases for control but recovered money in only 36 -- about 7%, with roughly nine in ten resulting in no further action -- and UDK confirmed that 54% of the 'Really Single' household-outlier cases its unit opened were in fact legitimate. Those 'revenue' outcomes conflate deliberate fraud with honest error, which UDK does not separate, so they are not pure fraud rates. Amnesty International characterised the system as mass surveillance and prohibited social scoring under the EU AI Act; UDK, ATP and the ministry (STAR) rejected that characterisation, the system was not suspended, and as of this writing no court had ruled.

Sources: amnestyinternationalalgorith2024, amnestyinternational2024a, fortuneeurope2024, bablai2024

Appears on: /domains/cases/denmark-udbetaling

Udbetaling Danmark's 'Joint Data Unit' merges and links the personal data of millions of residents from around ten national registers -- civil registration (CPR), buildings and dwellings (BBR), business, income, tax (R75), health, VAT, cash and sickness benefits, education grants and the motor-vehicle register -- alongside a 'Joint Data Unit Abroad' that pulls data from foreign authorities; in 2021 UDK paid about DKK 241 billion to roughly 2.4 million recipients. Amnesty International documents this as mass surveillance and argues the design carries a discrimination risk: 'Model Abroad' scores a relative strength of ties to non-EEA countries with citizenship as a direct input, and 'Really Single' treats statistically atypical households as suspicious. That harm is a design-level risk rather than a measured outcome, because UDK and ATP denied all requests for the demographic data needed to test the models for bias, so no disparate-impact figure exists in the record. Oversight is thin: the Danish Data Protection Authority (Datatilsynet) can generally act only on complaints (GDPR Art. 57) with no proactive power, and because flagged people rarely learn an algorithm selected them, complaints are rare. UDK rejects the discrimination-by-design and social-scoring findings; no court has ruled.

Sources: amnestyinternationalalgorith2024, amnestyinternationaldanmark2024, bablai2024

Appears on: /domains/cases/denmark-udbetaling

In judgment STS 1119/2025 of 11 September 2025, the Third Section of Spain's Supreme Court (Sala de lo Contencioso-Administrativo) ordered the government to give the transparency foundation Civio access to the source code of BOSCO, the software that determines eligibility for the electricity social bonus (bono social electrico). Applying the Transparency Law (Ley 19/2013) together with Article 42 of the EU Charter of Fundamental Rights and Article 105.b of the Spanish Constitution, the Court held that access to public information is a constitutional right and that neither intellectual property nor national security is an automatic shield, dismissing the government's secrecy claims as a 'mere risk' of eventual harm to be assessed case by case under a proportionality test. Civio and legal commentators describe an 'error multiplier': because BOSCO decides automatically and gives no reasons, one systematic error can propagate to thousands of eligible people at once. As of May 2026, roughly eight months after the ruling, the source code had still not been delivered and Civio had filed for judicial enforcement.

Sources: consejogeneraldelpoderjudici2025, fundacionciudadanacivio2025b, fundacionciudadanacivio2025a, derechoadministrativoyurbani2025, fundacionciudadanacivio2025c, fundacionciudadanacivio2026, freesoftwarefoundationeurope2026

Appears on: /domains/cases/spain-bosco

The transparency foundation Civio documented, by reconstructing BOSCO's behaviour from partial technical specifications and functional test cases, two systematic ways the software denied the electricity social bonus to people who qualified: when a pensioner ticked the 'pensioner' box the application could return an 'imposibilidad de calculo' (impossibility of calculation) error and be rejected without properly evaluating income; and large families, entitled to the bonus regardless of income, were denied whenever a household member withheld authorization to consult income data, although income was not a regulatory requirement for that category. After a 2017-2018 overhaul required all beneficiaries to re-apply by 31 December 2018, enrollment fell from roughly 2.4 to 2.5 million under the prior scheme to 1,111,958 as of January 2019 (later cited around 1.5 million), against an estimated 4.5 to 5.5 million eligible people, and more than half a million applicants were rejected. No audited per-decision error rate is public, because the source code and verification-test results were withheld; one academic analysis records errors in both directions, but the documented net effect is under-inclusion.

Sources: fundacionciudadanacivio2019, algorithmwatchnicolaskayserb2019, freesoftwarefoundationeurope2026, xatakaenriqueperez2024, rebootdemocracyjoseluismarti2025

Appears on: /domains/cases/spain-bosco

Serbia's Social Card (Socijalna karta) registry, given a statutory basis by the Law on the Social Card in force from 1 March 2022 and financed in part by an 82.6 million euro World Bank public-sector loan, cross-links roughly 130 to 135 categories of data from other state registers to verify social-assistance eligibility and flag suspected undeclared income or assets. After the law, named sources report the caseload falling by a range of tens of thousands: government figures cited by Amnesty International show about 35,000 fewer recipients by August 2023, A11 counts at least 44,000 people having lost assistance by early 2024, and the UN Working Group on Business and Human Rights reported over 60,000 without assistance by October 2025. These are largely net caseload declines rather than audited counts of system-caused removals, and the government attributes part of the fall to a stronger economy. Roma are reported among the most affected because informal earnings are misclassified as income, but the registry records no ethnicity, so this is inferred rather than officially disaggregated. As of the latest reporting, Constitutional Court, World Bank Inspection Panel, and UN scrutiny were pending or active, with no court or panel yet ordering changes.

Sources: ainitiativeforeconomicandsoc2024, amnestyinternational2023b, contextthomsonreutersfoundat2023, unworkinggrouponbusinessandh2025, worldbankinspectionpanel2024, chinaceeinstitute2024

Appears on: /domains/cases/serbia-social-card

Under Serbia's Social Card system, a removed beneficiary has 15 days to appeal and must wait three months to reapply regardless of changed circumstances, and removal letters frequently reference only unspecified data from the electronic database. A11's Request for Inspection to the World Bank Inspection Panel alleges that, because the system is semi-automated, social workers cannot correct errors recorded in it. Over roughly two years the Ministry processed more than 100,000 notifications of suspected income or asset increases, while beneficiaries filed only 361 appeals against Centers for Social Work rulings; because the two figures cover different populations, the gap illustrates how rarely flags were contested rather than a measured appeal rate. Documented misclassifications include a one-off funeral donation read as income and long-scrapped cars still counted as assets.

Sources: amnestyinternational2023b, ainitiativeforeconomicandsoc2024, worldbankinspectionpanel2024, chinaceeinstitute2024, contextthomsonreutersfoundat2023

Appears on: /domains/cases/serbia-social-card

Samagra Vedika, an entity-resolution system built by the Telangana government, decided welfare eligibility by matching residents across thirty-plus government databases into a consolidated profile; between 2014 and 2019 more than 1.86 million ration cards were cancelled and 142,086 fresh applications were rejected without notice. Its core error was entity-resolution false-positive matching, in which a similarly-named third party's asset was attributed to the applicant and silently flipped the eligibility flag. After the Supreme Court of India ordered field re-verification in April 2022, a partial re-verification found roughly 7.5 percent wrongful rejection (at least 15,471 approved of 205,734 re-processed cases), a lower bound from an incomplete review; the system is proprietary and closed and an independent technical audit could not be completed, with no source code or accuracy data released. The government cited a self-reported 95 percent fraud-filtering efficiency, which measures spurious-application filtering rather than the wrongful-exclusion rate.

Sources: amnestyinternational2024c, tapasya2024, tusharvsharma2026, sumitjha2024, kumarsambhav2020, pulitzercenteraiaccountabili2024

Appears on: /domains/cases/india-samagra-vedika

Under Samagra Vedika, exclusions were silent and there was no statutory route to contest an algorithmic decision, so the burden of proof fell on the excluded person: reporting describes officials who, though formally able to override the algorithm with evidence, deferred to it and declined to overturn its verdict, treating errors as backend technical issues. Documented individual harms include a 67-year-old widow denied rations for more than seven years after the system linked her deceased husband to a car owned by a similarly-named third person, and a family rejected for allegedly owning a four-wheeler that was declared eligible only after a Telangana High Court ruling. Corrections came through individual litigation and did not systematically feed back into the model, and the same entity-resolution technology was reused to issue new ration cards in 2024-2025.

Sources: tapasya2024, thereporterscollective2024, tusharvsharma2026, amnestyinternational2024c, sumitjha2024

Appears on: /domains/cases/india-samagra-vedika

In A.M.C. v. Smith (No. 3:20-cv-00240, M.D. Tenn.), a federal court held after a five-day bench trial that Tennessee's Deloitte-built TEDS automated Medicaid eligibility system, operational statewide since March 19, 2019 for a program covering roughly 1.7 million residents, produced wrongful terminations, wrong-household assignments, and misleading or missing notices that violated the Medicaid Act, the Fourteenth Amendment's Due Process Clause, and the Americans with Disabilities Act; the 116-page opinion, issued August 26, 2024 by Judge Waverly D. Crenshaw Jr., ordered mediation before considering an injunction.

Sources: statescoopkeelyquinlan2024, stotlerhayesgroupllcerinsail2024, georgetownuniversitycenterfo2024, nationalhealthlawprogram2024

Appears on: /domains/cases/tennessee-tenncare-teds

The UK government built its own AI meeting scribe for council caseworkers and piloted it through a cohort of 25 selected councils (22 active, more than 400 users) under one shared pooled-assurance record, then open-sourced it and adapted it to enlist around 500 housing and homelessness workers by June 2026; the cohort published a multi-council governance dataset but no transcription-accuracy or error-rate evaluation, and standard risk controls such as penetration testing and certification had not been completed on the alpha at pilot time.

Sources: localgovernmentassociation2025b, localgovernmentassociation2025a, ministryofhousing2026, trendall2026, incubatorforartificialintell2026

Appears on: /domains/cases/minute-local-ai

Independent research by the Ada Lovelace Institute on AI transcription in social work, based on interviews with 39 social workers across 17 local authorities in England and Scotland, reported that local authorities focus their evaluations on efficiency rather than impact on people who draw on care and that perceptions of reliability and the need for human oversight vary significantly among workers; the research covers such tools sector-wide, not this tool specifically.

Sources: adalovelaceinstitute2026b, bruff2026, adalovelaceinstitute2026a

Appears on: /domains/cases/minute-local-ai

The Ministry of Justice built an in-house AI transcription and summarisation copilot, Justice Transcribe, for probation staff in England and Wales, scaling it from a pilot to more than 1,000 officers in October 2025 and to every probation officer by June 2026, with official transparency data recording more than 800,000 supervision meetings summarised between 7 October 2025 and 2 June 2026; the reported time-savings are the ministry's own and rest on an operating assumption the department itself labels illustrative, and no transcription-accuracy rate, officer correction rate, or independent evaluation of the tool has been published.

Sources: justiceaiunit2026, ministryofjustice2025, ministryofjusticeandhmprison2025b, ministryofjusticeanddsit2025, ministryofjustice2026

Appears on: /domains/cases/justice-transcribe-probation

Copilot-written probation case records sit upstream of high-volume algorithmic risk assessment over the same record ecosystem: reporting places the ministry's OASys-based reoffending-risk prediction at more than 1,300 people a day, drawing on probation and prison caseload systems and the Police National Computer, with a successor tool rolling out during 2026, and the ministry's own validation found lower predictive validity for all Black, Asian and Minority Ethnic groups for non-violent reoffending and for Black and Mixed ethnicity offenders for violent reoffending - a property of the downstream risk model, not the copilot; peer-reviewed commentary raises the erosion of professional judgment and the unresolved accountability for algorithm-influenced decisions as structural concerns, and no published source documents a named data pipeline from the copilot's output into the risk tools.

Sources: statewatch2025, phillips2026, nellis2026

Appears on: /domains/cases/justice-transcribe-probation

The US Social Security Administration requires decision writers to run fully favorable disability decisions through its in-house Insight verifier before issuance, with narrow documented exceptions, and the 2025 federal AI inventory records the tool computing 43 quality flags. In the agency's internal five-month study of roughly 50,000 appeals-level cases, reported through the 2019 Inspector General audit, analysts who used Insight logged about 0.9 errors per case against 0.7 for non-users, saw processing time fall about 4.7 days per case, and had about 12.6 percent of their cases returned for quality issues against 21.5 percent for non-users. These are internal, non-randomized comparisons among self-selected voluntary users, and the same audit found the agency stopped tracking performance after the first five months and could not determine any effect on remands.

Sources: ussocialsecurityadministrati2019, ussocialsecurityadministrati2026, engstrom2020

Appears on: /domains/cases/ssa-insight

In a review issued April 30, 2026 (report 25-00153-47), the Department of Veterans Affairs Office of Inspector General found that at least 8,000 of an estimated 8,100 automated Dependency and Indemnity Compensation (survivor-benefit) granting decisions issued from September 2023 through August 2024 - nearly all - contained at least one legal or procedural deficiency, such as incomplete evidence summaries and omitted favorable findings, with most rating decisions listing only the death certificate as evidence. The OIG separately found that at least 2 percent of the decisions (at least 190) carried monetary-impact legal errors totaling at least 2.7 million dollars (2,727,764 dollars in questioned costs); the roughly 98 percent figure is the share with any legal or procedural defect, not the monetary-error rate. The system, phased in beginning May 2020, extracts data from scanned documents and applies predefined encoded rules to grant service-connected death claims end to end with no human involvement when the rules are met; the OIG describes it as rules-based automation and document extraction, not machine learning, and its figures are outcome statistics from a statistical sample rather than a per-interaction rate.

Sources: departmentofveteransaffairso2026, nieberg2026, weston2026

Appears on: /domains/cases/va-claims-automation

The Office of Inspector General reported that VA's internal correction channels did not catch the automated survivor-benefit deficiencies and that the external audit was, empirically, the only channel that changed behavior. In April 2020 a VBA analyst reported through the internal defect-tracking system that automated decisions listed only the death certificate as evidence, and the Pension and Fiduciary Service closed the defect without action; the same deficiency was central to the 2026 findings, and VA removed the long-form guidance from its manual only in March 2025, immediately after the OIG's preliminary briefing - roughly five years later, and the OIG's full public report did not follow until 2026, roughly six years after the ticket. The OIG found the quality-review checklist for automated claims was less rigorous than the review traditional claims receive, and that the PACT Act section 701(b) modernization plan to Congress did not fully disclose that VBA grants these claims end to end without human intervention. Errors persisted as the program expanded: the VA Secretary announced expanded DIC automation in May 2025, and 20 additional automated decisions from September and October 2025 showed similar errors as of November 2025, with one recommendation still open and VBA concurring only in part.

Sources: departmentofveteransaffairso2026

Appears on: /domains/cases/va-claims-automation

In Trelleborg, Sweden, the first municipality to fully automate social-assistance decisions, peer-reviewed analysis reports that about 30 percent of digital reapplications are decided entirely by rules-based software with no human review and about 85 percent receive at least partial automated handling; decision time on reapplications fell from roughly two days to under a minute, and a human caseworker re-enters the path only by exception, when a routing rule detects significantly changed circumstances, a missing activity plan or job-seeking documentation, or a complex or negative case. No error, override, exception-routing, or appeal-rate figures for the automated path have been published, so the fraction of automated decisions that ever reaches a human cannot be established from the record.

Sources: algorithmwatch2020b, ranerupandhenriksen2022, europeancommissionjointresea2021

Appears on: /domains/cases/trelleborg-rpa

The City of Amsterdam spent roughly five years and an estimated EUR 535,000 building a deliberately fair, explainable welfare-fraud screening model with nearly every recommended pre-deployment safeguard in place - a bias audit, training-data reweighting that approximately equalized wrongful-flag rates on retrospective data, a data-protection assessment and a human-rights assessment, external and academic review, a citizen panel, and dual algorithm-register transparency - and discontinued it after a 2023 live pilot on nearly 1,600 applications. In the investigating journalists' analysis of aggregate data the city provided, the group disparities re-emerged inverted on the live pilot, now more likely to wrongly flag Dutch nationals, women, and applicants with children, with the tool flagging more applications than the analog process and no better than caseworkers at finding genuine cases. The Dutch national algorithm register records the deployment ending September 2023 and lists it out of use, and the responsible alderman announced the halt in November 2023.

Sources: braun2025, lighthousereportsa, algoritmeregisterdutchnation2023

Appears on: /domains/cases/amsterdam-slimme-check

In a developer-reported randomised controlled trial of more than 1,000 adviser support requests, an adviser-facing benefits copilot at Citizens Advice returned supervisor-checked answers in about four minutes, roughly half the previous response time, with about 80 percent of its drafts approved by supervisors without revision; these figures are reported by the tool's builders and have not been independently replicated.

Sources: varotsis2025, departmentforscience2025a, stanfordlegaldesignlabjustic

Appears on: /domains/cases/caddy-citizens-advice, /what-ai-can-do

Advisers given access to the copilot were reported to be more than twice as likely to say they felt confident giving advice than a control group, a self-reported measure from post-call in-chat surveys rather than a client-outcome or accuracy measure.

Sources: varotsis2025, stanfordlegaldesignlab2025

Appears on: /domains/cases/caddy-citizens-advice, /what-ai-can-do

In 2023 Singapore's GovTech began a whole-of-government retire-and-replace of its scripted Ask Jamie chatbots, embedded since 2014 on 70-plus (a vendor case study claims 80) agency websites as independent per-agency answer engines, migrating government chatbots onto centrally provided large-language-model engines; the stated aim was to convert all 88 chatbots and retire the scripted engine by end 2023, the verified snapshot is 21 of 88 converted as of September 2023 (migration completion not independently documented), and by the VICA product page updated 29 April 2026 the successor platform hosts over 100 chatbots for 60-plus agencies at an average of over 800,000 monthly queries, figures that are all government self-reported.

Sources: hirdaramani2023, govtechsingapore2026, govtechsingapore2019

Appears on: /domains/cases/singapore-chatbot-fleet-refresh

The Singapore government benefits-navigation surface is documented as scope-limited to information and estimates rather than adjudication: the Ministry of Finance Support For You Calculator turns self-declared inputs into benefit estimates that are explicitly estimates and not entitlement decisions, and the Chat.Gov.SG (Beta) explainer hosted on the SupportGoWhere domain states the assistant summarises information from official government websites and does not assess eligibility, make decisions, submit applications, or complete transactions, and warns users not to share personal or sensitive information.

Sources: publicservicedivisionsingapo2026, mustsharenews2024

Appears on: /domains/cases/singapore-chatbot-fleet-refresh

A June 2026 Treasury Inspector General for Tax Administration performance audit (Report Number 2026-308-029) reported that the IRS expanded its Automated Collection System chatbot and live-chat program and made live chat permanent while having no performance measures for it, despite a Taxpayer First Act requirement for metrics and benchmarks, and that management's claim the bots reduced telephone demand could not be substantiated; the statistical reports the IRS did collect were deemed unreliable, in one instance showing a single assistor apparently working 603 chats at once against a systemic cap of three, attributed partly to a miscalculated handle-time metric the vendor had not resolved as of December 2025.

Sources: treasuryinspectorgeneralfort2026, bracken2026, bramwell2026

Appears on: /domains/cases/irs-acs-chatbots

In the same audit, of a judgmental sample of 40 IRS ACS live assistors, 24 (60%) were found working multiple chats concurrently and 12 of those 24 had at least one authenticated chat open while working another, which TIGTA reported as raising the risk of disclosing taxpayer information to the wrong taxpayer; the audit also reported 635,684 resolution codes against 613,056 chats (a mismatch management knew of but did not investigate) and, in March 2025 hand-testing, 14% of chatbot process flows deficient and 83% of tested keywords unrecognized or insufficient, with the figures drawn from a nonprobability sample and data the audit itself characterized as unreliable and not projectable to the full assistor population.

Sources: treasuryinspectorgeneralfort2026, bramwell2026, cohn2026

Appears on: /domains/cases/irs-acs-chatbots

In a spring-2025 randomized pilot inside a large consumer EBT app, the vendor reports that 53% of eligible SNAP recipients took up in-app AI help for missed deposits and that treated users were restored faster and more often in the same month than a control group, with every AI dead-end escalated to a named human; all outcome figures are vendor-published and the effect magnitudes were not disclosed.

Sources: propelincpropelinsights2025b, guarino2025a

Appears on: /domains/cases/propel-snap-assistant

By the vendor's own account of the design, the assistant grounds on a state-verified deposit record it reads but does not write to, and steers recipients to act on the state system of record rather than acting for them.

Sources: propelincpropelinsights2025b

Appears on: /domains/cases/propel-snap-assistant

Between 2019 and 2025 more than 70% (about 73% per its ten-year retrospective) of California's online SNAP applications were submitted through GetCalFresh, a deterministic, structured-workflow application assister built and operated by the nonprofit Code for America, which reports helping 6.2 million people obtain more than $12.8 billion in food benefits from 2017 to 2025 (organization-published figures that are not independently audited); the node made no eligibility determinations, and in 2024 and 2025 the California Department of Social Services coordinated a dated, phased transfer of its functions into the state-owned BenefitsCal portal.

Sources: codeforamerica2024b, codeforamerica2025a, californiadepartmentofsocial2025

Appears on: /domains/cases/getcalfresh, /what-ai-can-do

A randomized controlled trial of roughly 65,000 Los Angeles GetCalFresh applicants (Giannella, Homonoff, Rino, and Somerville, American Economic Journal: Economic Policy 16(4), 2024) found that access to applicant-initiated flexible interviews increased SNAP approvals by about 6 percentage points, doubled early approvals, and raised long-term participation by over 2 percentage points, identifying the intake interview as a key procedural-denial barrier; Code for America separately reported an in-house experiment lifting renewal-form submissions among prior non-responders from about 1.5% to roughly 12% (organization-published, without sample sizes or confidence intervals).

Sources: giannella2024, codeforamerica2021, codeforamerica2024a

Appears on: /domains/cases/getcalfresh

In June 2024 the board of Benefits Data Trust, a Philadelphia benefits-navigation nonprofit that reported helping more than 120,000 people access about $182 million in benefits in 2023, voted unanimously to wind the organization down within a self-imposed 60-day window, citing only 'a perfect storm of circumstances'; the organization closed on August 24, 2024, laying off 273 employees, despite roughly $12 million in unrestricted reserves at the end of 2023 and about $32 million in projected 2024 revenue.

Sources: brubaker2024a, brubaker2024b, wink2024, mosbruckergarza2024

Appears on: /domains/cases/benefits-data-trust-winddown

The closure left active government partnerships without a designated successor, including a Pennsylvania Department of Aging workload of nearly 48,000 applications from 27,018 households in the final year and a Philadelphia BenePhilly call-center contract the organization was reported to be exceeding through mid-2024; the navigation function fragmented to higher-friction channels, with the work redistributed across partner agencies and a subcontractor and referral waits reported as several months, which a Pew analyst described as a 'cascading effect.'

Sources: brubaker2024d, burnley2024, mosbruckergarza2024

Appears on: /domains/cases/benefits-data-trust-winddown

London's Strategic Insights Tool for Rough Sleeping probabilistically links records from three separately governed systems - CHAIN street-outreach contacts, In-Form charity casework, and H-CLIC borough statutory applications - into a single rough-sleeping journey per person that is read, in aggregate form only, across all 33 London local authorities; the tool makes no individual-level determinations, and after the build vendor's data-processor contract ended on 2 February 2024 the Greater London Authority contracted Homeless Link, which also operates the CHAIN source system, for its ongoing hosting, management, and maintenance.

Sources: techuk2024, loti2023, londonofficeoftechnologyandi2023

Appears on: /domains/cases/london-rough-sleeping-sit

The Strategic Insights Tool's matcher accepts an association only above an 85% probability threshold chosen to minimise false positives, and the project's own Phase 2 Data Protection Impact Assessment reports 91% recall - conceding that roughly 9 in 100 true cross-system matches are missed so that 'numbers subsequently appear lower in places where they should be higher' and that recall varies as new data of varying quality is ingested; no false-positive rate is published, the accuracy figures are self-reported by the delivery team, and no independent evaluation of the tool's decision impact exists.

Sources: loti2023, lotiannahumplebyandfacultyja2025

Appears on: /domains/cases/london-rough-sleeping-sit

San Jose's vehicle-mounted computer-vision pilot, described by city officials and national housing advocates as the first US experiment training AI to recognize tents and lived-in vehicles, reported sharply class-asymmetric accuracy in the city's own staff-ground-truthed evaluation — 97% for potholes and 88% for trash, but only 70% for RVs (unable to distinguish a lived-in RV from an empty one) and 12.5% for lived-in vehicles, with a March 2024 official interview bracketing the habitation figures at 70–75% for RVs and 10–15% for lived-in cars against a 70% goal; no detection ever generated an operational dispatch, and after investigative exposure and structured engagement the city removed every habitation-detection use case, its March 2025 status report declining to recommend implementing AI object detection in city operations at this time.

Sources: feathers2024, cityofsanjoseinformationtech2025, usdepartmentoftransportation2025

Appears on: /domains/cases/san-jose-encampment-detection

The pilot's published data-usage protocol declares that the footage cannot be actively monitored for law-enforcement purposes while preserving a police request path to it — verbatim, 'Law enforcement may request access to previously stored footage. Law enforcement is not actively monitoring any data collected' — and requires de-identification or deletion within one month; the CIO stated data was not shared with police during the pilot, yet public-records reporting documented that one vendor's system ran optical character recognition of license plate numbers despite the city's no-identification claim, so the declared authority rule and the feasible data flows diverged, a gap surfaced by journalists rather than by any standing audit, and the no-law-enforcement-use clause is city protocol language rather than statute.

Sources: feathers2024, cityofsanjoseinformationtech2024, varian2024

Appears on: /domains/cases/san-jose-encampment-detection

The Los Angeles Coordinated Entry System replaced the VI-SPDAT survey for single adults with the Los Angeles Housing Assessment Tool, a 19-item self-report score whose weights were derived by a regression on 71,747 historical assessments linked to county records; where the CESTTRR research estimated the VI-SPDAT scored near chance (AUC 0.54) with racial false-negative gaps up to 8.5 percentage points, the equity-adjusted successor was deliberately traded down in overall accuracy (AUC 0.60, from an accuracy-only 0.64) to close those gaps to under one percentage point, and every such figure is a pre-deployment estimate on 2015 to 2018 held-out data rather than an observed post-launch outcome.

Sources: rice2023, losangeleshomelessservicesau2025a

Appears on: /domains/cases/lahsa-triage-revision

During the dual-tool transition the two instruments' PSH-consideration thresholds were 8-plus on the VI-SPDAT and 17-plus on the LA HAT, and by LAHSA's account initial quantitative data and provider feedback showed participants were more likely to obtain an eligible score under the VI-SPDAT, so direct-service providers opted to administer it, a trend LAHSA states 'perpetuated the racial bias of the VI-SPDAT in the System'; on April 22, 2026 the CES Policy Council lowered the LA HAT threshold to 12-plus, ruled the most recent LA HAT score supersedes a coexisting VI-SPDAT score, and forced deactivation of new VI-SPDAT completions (for LA HAT-access programs on May 1, 2026 and system-wide on June 30, 2026), though LAHSA has not released the underlying eligibility-rate numbers.

Sources: losangeleshomelessservicesau2025a, losangeleshomelessservicesau2026b, losangeleshomelessservicesau2026a

Appears on: /domains/cases/lahsa-triage-revision

In a registered randomized controlled trial of 1,263 imminent-risk applicants (514 treatment, 749 control) run by the University of Notre Dame's evaluation lab, households offered flexible emergency financial assistance averaging about 2,000 dollars, typically one to two months of back rent, through Santa Clara County's homelessness-prevention system were reported 81 percent less likely to become homeless within six months and 73 percent within twelve months; the peer-reviewed article's abstract states the assistance reduced homelessness by 3.8 percentage points from a 4.1 percent base rate, and the researchers conservatively estimated 2.47 dollars in community benefits per net dollar spent.

Sources: phillipsandsullivan2025, universityofnotredamenews2023, phillipsandsullivan2021

Appears on: /domains/cases/santa-clara-prevention

Because becoming homeless is statistically rare even among at-risk applicants - about 96 percent of the trial's control group never became homeless without assistance - the program's own co-author cautions that prevention resources can flow to households that would have stayed housed anyway, making screening precision on a low base rate the binding constraint; as of February 2026 the model is being replicated across about ten heterogeneous US jurisdictions under a 77-million-dollar initiative, with the same evaluation lab as the common evidence partner assessing each site.

Sources: kendall2026a, phillipsandsullivan2025, destinationhome2026b, universityofnotredamenews2026

Appears on: /domains/cases/santa-clara-prevention

At the Calgary Drop-In Centre, a University of Calgary engineering group and the NGO shelter operator built deliberately interpretable screening for chronic and episodic shelter use - explicit stay-count thresholds (for example 81 or more stays in a 90-day window) and database-queryable rules derived from the shelter's own administrative records, reported to flag candidate clients at a median of about 98 days versus 285 days under the Government of Canada definition and 365 under the Alberta definition - and, rather than surface a risk score, deployed a co-designed data-navigation interface that shows frontline staff raw client histories; no fetched source confirms the thresholds running as an automated production screener, and the deployed, studied artifact is the raw-history interface.

Sources: messier2021, arulesearchframeworkfortheea2022, masrani2025

Appears on: /domains/cases/calgary-drop-in-shelter-ml

Across a 2022 to 2024 embedded deployment study of the interface (16 staff across 7 role categories; 29.5 hours of qualitative data; five committee observations; three deployed versions), the participant-research team documented a stakes-dependent 'data-outsourcing continuum': staff were reluctant to outsource high-stakes barring decisions, treating the data as a starting point for collaborative discussion, while reporting more willingness to accept automated data-driven recommendations for lower-stakes housing triage; the finding is the staff's own articulated practice rather than a measured override or agreement rate, all deployment evidence is authored by the embedded research team, and no independent evaluation, usage logs, or decision volumes are published.

Sources: masrani2025, thehumanbehindthedatareflect2023

Appears on: /domains/cases/calgary-drop-in-shelter-ml

An AI quality-assurance tool deployed on a national 988 backup line scores crisis counselors' own call practice rather than callers, expanding measured review from the under-3% of calls that had been reviewed by hand toward nearly all of them; a peer-reviewed reliability study of 476 labeled calls reported agreement with human ratings at 98 percent of human interrater agreement for detecting any risk assessment, with average F1 of about 0.86 at call level and 0.66 at statement level, and its authors include four holders of equity in the vendor.

Sources: imel2024, aguilar2023, nihreporternationalinstitute2025

Appears on: /domains/cases/lyssn-protocall-988

The registered randomized crossover trial of the tool's counselor feedback (81 call-takers) completed on October 31, 2025, but as of mid-2026 no results were posted to the trial registry or found in the peer-reviewed literature and participant-level data were marked unavailable for proprietary reasons, so reported counselor-skill-improvement effects remain vendor claims pending independent publication.

Sources: clinicaltrialsgovusnationall2026, lyssn2026

Appears on: /domains/cases/lyssn-protocall-988

Gaggle's student-communication safety monitoring, used by roughly 1,500 US districts covering about 6 million students as of a March 2025 AP and Seattle Times investigation, scans school-issued accounts around the clock and routes flags through a multi-hop chain (a machine flag, an off-site vendor reviewer, district safety staff, and, for imminent-danger after-hours alerts, occasional police welfare checks); in Vancouver Public Schools nearly 2,200 students (about 10% of enrollment) triggered alerts in one year, in Lawrence USD 497 more than 1,200 incidents were logged in ten months with about two-thirds deemed nonissues by officials (a figure the plaintiffs drew from district records), and the archive of flagged documents was accidentally released to reporters as nearly 3,500 unredacted files through unprotected links, while a 2023 RAND review found only scant evidence of either benefit or risk and the vendor publishes no accuracy figures.

Sources: bryanandlurye2025, associatedpress2025, lawrencejournalworld2025

Appears on: /domains/cases/gaggle-school-monitoring

After nine Lawrence, Kansas students sued their district in early August 2025 over its use of AI communication monitoring, court filings revealed the district had ceased using Gaggle mid-litigation and substituted a different monitoring vendor with no board vote or public disclosure — surfacing only as a line in a check register — and the plaintiffs' amended complaint argued the swap does not moot the case because the core practice of suspicionless scanning, flagging, and seizure of student speech continues; on April 10, 2026 a federal judge found the district violated the Kansas Open Records Act in withholding the substitution and phase-out records, and on June 4, 2026 ordered it to pay the students' attorney fees, characterizing the conduct as drawn out, hollow and perplexing, with a jury trial on the surviving constitutional claims set for January 2027.

Sources: heimsoth2025, heimsoth2026a, heimsoth2026b

Appears on: /domains/cases/gaggle-school-monitoring

In March 2026 the UK Parliamentary and Health Service Ombudsman partly upheld a complaint that an NHS mental health trust installed camera-based, contact-free bedroom monitoring on a psychiatric ward without seeking a patient's consent, gave her no information about it, and did not switch it off when she asked; the case documentation and investigative reporting describe an internal clinical evaluation that the vendor is reported to have authored the business case for and shaped, a rebrand of the vendor during a statutory inquiry, and an open data-protection investigation, while the tool's own outcome-reduction figures are vendor claims contested by a campaign-linked meta-analysis and its adoption share across NHS mental health trusts is reported only as a contested range.

Sources: parliamentaryandhealthservic2026, williamson2026a, williamson2026b, nationalsurvivorusernetwork2025, stopoxevision2026

Appears on: /domains/cases/oxevision-nhs-wards

The ombudsman's report on the case (decision 27 March 2026) found the trust did not seek or revisit the patient's consent for the bedroom monitoring, did not turn the camera off when she asked, gave her no information about it, and kept no record of how staff used it, and that even the trust's revised 2025 procedure still permits overriding a capacitous patient's refusal on clinically-safe grounds with multidisciplinary-team approval; on the separate question of over-reliance the ombudsman found on balance, cross-referencing observation charts, a nurse-adviser review and door key-card data, that in-person observations had continued and did not uphold that part of the complaint.

Sources: parliamentaryandhealthservic2026

Appears on: /domains/cases/oxevision-nhs-wards

In 2025 ODMAP's pre-set county thresholds - a rolling 24-hour count against a threshold each agency sets or accepts, recommended by the system as two standard deviations above the county's own previous 90-day mean, a deterministic rule rather than a machine-learning model - fired 74,805 advisory spike-alert notifications from 498,003 suspected, unconfirmed overdose events that only about 1,362 of its 5,605 approved agencies actually submitted; these figures are self-published by the program in its own annual report and manuals, and ODMAP states its data are suspected, incomplete, not a system of record, and should not be generalized beyond participating agencies.

Sources: washingtonbaltimorehidta2025b, washingtonbaltimorehidta2026c, washingtonbaltimorehidta2025a

Appears on: /domains/cases/odmap-overdose-spike-alerts

ODMAP's shared overdose store is housed inside a federal drug-enforcement program, and its operating policies both state that ODMAP is neither an intelligence sharing database nor a pointer index records system and grant the host permission to use the data as the HIDTA sees fit, including combining it with other databases it manages for law enforcement and public health products; a 2024 peer-reviewed stakeholder study documented divergent public-health versus public-safety data-privacy standards, and a 2025 peer-reviewed analysis argues the integration risks racialized surveillance and criminalization of people who experience overdose, a contested scholarly critique of the link structure rather than a documented misuse incident.

Sources: washingtonbaltimorehidta2022, syvertsen2025, allen2024

Appears on: /domains/cases/odmap-overdose-spike-alerts

The Targeted Real-Time Early Warning System (TREWS), a machine-learning sepsis early-warning model, was evaluated prospectively across five hospitals of an academic health system covering 590,736 monitored patients — the largest prospective study of an ML sepsis system on record. Its central finding was conditional on the human loop: sepsis patients whose alert was evaluated and confirmed by a provider within three hours had a 3.3 percentage-point absolute and 18.7 percent relative adjusted reduction in in-hospital mortality, with less organ failure and shorter stays, while the alert on its own did not; a companion study found provider uptake varied with experience, unit culture, and alert context.

Sources: adams2022a, henry2022a

Appears on: /domains/cases/johns-hopkins-trews, /pan-lab

The TREWS mortality-benefit evaluation was prospective and peer-reviewed but observational and developer-led: it was built at the deploying institution and commercialized through a company founded by its principal investigator, and confirmation-associated benefit is an observational association rather than a randomized effect of the algorithm — providers who engaged with alerts may differ from those who did not in ways the adjustment does not capture. The strongest numbers in the record therefore come from the party with the strongest interest in them, and no independent replication of the mortality effect had been published.

Sources: adams2022a

Appears on: /domains/cases/johns-hopkins-trews, /pan-lab

The Advance Alert Monitor is an in-hospital deterioration model running around the clock across 21 hospitals of an integrated health system, scoring inpatients hourly and firing roughly twelve hours before predicted deterioration; a 2020 New England Journal of Medicine evaluation associated its alert-driven rapid-response workflow with lower mortality. Its defining feature is where the alert goes: not to the bedside, but to a dedicated regional tier of critical-care virtual quality nurse consultants who screen every alert around the clock, work up the chart, and only then escalate to the on-site rapid-response team — so the measured benefit is priced against the whole two-tier staffing topology, not the model alone.

Sources: escobar2020a, thekaiserpermanentenorthernc2022

Appears on: /domains/cases/kaiser-aam-deterioration, /pan-lab

Sepsis Watch is a deep-learning sepsis-detection system scoring every emergency-department patient every five minutes over 86 variables, deployed at an academic hospital under a registered clinical trial, with alerts fronted by rapid-response-team nurses who track treatment-bundle completion on three- and six-hour timers. Its structural fault line is an authority split: the operator who receives the alert (the nurse) is not the operator empowered to act on it (the physician who holds treatment authority), so the correction runs through a peer-persuasion edge. An independent ethnography found the system worked because nurses performed hidden repair work — mediating the professional hierarchy and doing the emotional labor of communicating a risk score upward — labor that was structurally necessary, largely invisible to the deployment's formal description, and undervalued.

Sources: sendak2020a, elish2020

Appears on: /domains/cases/duke-sepsis-watch, /pan-lab

A widely implemented proprietary sepsis-prediction model shipped inside a common electronic-health-record platform and switched on across hundreds of hospitals was externally validated in 2021 across 38,455 hospitalizations at an academic health system: it achieved an area under the curve of 0.63, identified only 33 percent of sepsis cases, and had a positive predictive value of about 12 percent, generating roughly 109 alerts for every true sepsis case — a real-world performance the vendor had not fully examined before selling the model, and which an investigation attributed in part to undisclosed features such as antibiotic-order data that inflated internal validation.

Sources: wong2021c, statnews2021

Appears on: /domains/cases/epic-sepsis-michigan, /pan-lab

After external criticism, the vendor overhauled the sepsis model — retraining it, changing the sepsis-onset definition, and reducing its reliance on antibiotic-order features. A 2026 multicenter prospective validation of the updated model across 227,091 encounters reported an area under the curve of 0.82 to 0.92 with positive predictive value of 0.13 to 0.26 and substantial between-site variability, and its authors urged local validation and alert-silencing strategies rather than trusting the model out of the box — a correction that arrived only after independent scrutiny of a model that had already been deployed at scale behind a corporate firewall shielding it from outside inspection.

Sources: statnews2022, wong2026a

Appears on: /domains/cases/epic-sepsis-michigan, /pan-lab

The largest documented ambient-scribe deployment ran a 10-week pilot at an integrated medical group and then scaled to 7,260 physicians and 2,576,627 patient encounters over fourteen months, with roughly 16,000 hours of documentation time saved and sustained physician support measured along the way. The system records the visit and drafts the clinical note; the clinician edits and signs, and the model-to-record write is gated both by that clinician review and by a standing internal quality-assurance program over the AI output — a real subsystem with a real cost, because the drafted note becomes a permanent record that later clinicians and later tools read as fact.

Sources: tierney2024a, tierney2025a

Appears on: /domains/cases/kaiser-tpmg-scribe, /pan-lab

The scale numbers from a single ambient-scribe deployment are the deployer's own first-party measurements and should be read as that system's dashboard rather than a guarantee of the product class: a multisite study of 8,581 clinicians across five health systems found more modest effects — on the order of 13 to 16 fewer minutes per day with no meaningful after-hours relief — and a validated per-note evaluation found hallucinations in about 31 percent of ambient-generated notes under structured review, versus about 20 percent of physician-written gold-standard notes, making ambient notes more thorough but less accurate. The clinician review and quality-assurance program are the controls that stand between that error rate and a contaminated permanent record.

Sources: rotenstein2026a, palm2025a

Appears on: /domains/cases/kaiser-tpmg-scribe, /pan-lab

The strongest causal evidence in the ambient-scribe family comes from a 24-week stepped-wedge, individually randomized trial of an ambient scribe across 66 practitioners and 71,487 notes (38 percent AI-generated), which found work exhaustion significantly reduced, professional fulfillment unchanged (a recorded null), roughly 22 minutes per day of documentation time returned, and diagnostic coding accuracy improved. Unlike the larger first-party deployment reports, this is a randomized estimate of the well-being and time effects — though it measured practitioner well-being and time, not per-note error rates.

Sources: afshar2025a

Appears on: /domains/cases/uw-health-abridge-scribe, /pan-lab

The same team that ran the ambient-scribe trial released an open operations playbook for safety and effectiveness monitoring of ambient AI in production — the rare case where the organization-side monitoring function exists as a citable, designed subsystem rather than an assumed practice. That monitoring is the org's stated answer to a documented system-level risk of the technology: a coding arms race, in which better AI documentation raises coding intensity, payers recalibrate in response, and clinician attestation liability grows — so the improved coding accuracy the trial measured sits next door to an upcoding pressure the monitoring is meant to watch.

Sources: afshar2025b, dai2025a

Appears on: /domains/cases/uw-health-abridge-scribe, /pan-lab

A peer-reviewed evaluation of an ambient documentation platform at a large multi-specialty system found note time per appointment reduced (6.2 to 5.3 minutes) and NASA-TLX cognitive load reduced — but the burnout change (42.1 to 35.1 percent) was not statistically significant, the domain's honest null bound of cognitive-load relief without a demonstrated burnout effect.

Sources: stults2025a

Appears on: /domains/cases/sutter-ambient-scribe, /pan-lab

In the same evaluation, benefit varied sharply by clinician group: 85.8 percent of primary-care physicians reported improved satisfaction against 36.4 percent of medical specialists — the same tool, in the same system, under the same workflow, helping one operator class and largely failing another, so any uniform service term overstates the effect for the group it helps least.

Sources: stults2025a

Appears on: /domains/cases/sutter-ambient-scribe, /pan-lab

Company-run, pre-registered, peer-reviewed randomized rollouts of a commercial code-completion assistant across 4,867 developers at three enterprises found a pooled 26.08 percent increase in completed tasks, with gains concentrated among less-experienced developers. An independent randomized study of 16 experienced open-source maintainers on 246 tasks in familiar repositories bounded the expert tail from the other direction: those developers were about 19 percent slower with the AI while believing themselves about 20 percent faster — a measured perception-reality gap that means a uniform productivity number overstates the effect for senior engineers.

Sources: cui2025a, becker2025a

Appears on: /domains/cases/msft-accenture-copilot, /pan-lab

Individual coding-assistant gains do not automatically compose to organization-level delivery outcomes: a cross-industry research program measured a roughly 1.5 percent decrease in delivery throughput and a 7.2 percent decrease in delivery stability for every 25 percent increase in AI adoption, evidence that the churn the assistant adds must be absorbed by code-review and testing gates or the individual speed-up degrades the organization's delivery performance.

Sources: googleclouddora2024

Appears on: /domains/cases/msft-accenture-copilot, /pan-lab

An in-house machine-learning code-completion system built, deployed, and measured by a company's own platform organization for more than 10,000 internal developers reported, against a control group, a 25 to 34 percent suggestion-acceptance rate, a 6 percent reduction in coding iteration time versus control, and 3 percent of new code characters coming from the model at the time of measurement. The measuring party, the building party, and the deploying party were the same organization, and the numbers were published as an engineering-blog self-report rather than a peer-reviewed or independent evaluation.

Sources: tabachnyk2022

Appears on: /domains/cases/google-internal-completion, /pan-lab

The same company's cross-industry research program reported that AI-assisted software development amplifies an organization's existing strengths and weaknesses rather than substituting for them, with policy clarity and platform investment identified as the levers that determine whether AI adoption improves or degrades delivery — evidence that the individual coding gains do not compose to organization-level outcomes on their own, and that the deploying organization's existing gates and platform quality are what decide the result.

Sources: googleclouddora2025

Appears on: /domains/cases/google-internal-completion, /pan-lab

A regulated bank ran a structured six-week internal experiment with about 100 of its 5,000 engineers before scaling a commercial coding assistant to roughly 1,000 engineers, publishing its own measurement of the rollout. The bank's engineers reported productivity and code-quality improvements — and recorded the security impact as explicitly inconclusive, a real gating decision taken and documented under uncertainty rather than resolved by assertion, with the honestly recorded unknown carried forward into the scaled deployment.

Sources: chatterjee2024a, theregister2024

Appears on: /domains/cases/anz-bank-copilot, /pan-lab

What the bank's inconclusive security finding leaves open is not hypothetical: an independent security assessment of code generated by a widely used assistant found that about 40 percent of generated programs contained vulnerabilities across scenarios spanning the CWE top-25 weaknesses, and separate research documents developers accepting insecure suggestions with overconfidence — so the security unknown a deployment carries forward unresolved sits against a class-level literature in which insecure generation is common.

Sources: pearce2022a

Appears on: /domains/cases/anz-bank-copilot, /pan-lab

A mid-size enterprise ran a systematic four-phase evaluation-to-rollout of a commercial coding assistant across more than 400 developers, publishing acceptance telemetry (a 33 percent suggestion-acceptance rate, with 20 percent of suggested lines accepted), a 72 percent satisfaction figure, documented per-language variation, and stated limitations. Its evaluation instrument is acceptance-rate telemetry — which the productivity literature identifies as the measure most correlated with perceived productivity rather than outcome, and perception is measured to be miscalibrated for experienced developers, so acceptance telemetry captures adoption feel, not delivered output.

Sources: bakal2025a, ziegler2024a

Appears on: /domains/cases/zoominfo-copilot, /pan-lab

The deployment report stated its limitations but reported no security evaluation at all — an unrecorded unknown, one step less honest than a deployment that runs a security check and records the result as inconclusive, because an absence no one has written down is not a governed object and cannot be carried forward or resolved. The value of the case is the documentation quality of an ordinary, competent adoption — phase gates, telemetry definitions, per-language deltas, and stated limitations by the deployer itself — with the missing security question priced as the one thing even that documentation did not name.

Sources: bakal2025a

Appears on: /domains/cases/zoominfo-copilot, /pan-lab

A global bank replaced rules-based transaction monitoring with a cloud vendor's machine-learning anti-money-laundering product as its primary monitoring system in key markets, reporting two to four times more confirmed suspicious activity with roughly 60 percent fewer alerts. Every one of those numbers is a vendor-and-customer self-report with no independent audit — which is itself the honest structure of the domain, because a peer-reviewed deployment-scale benefit measurement inside a named financial-crime operation does not publicly exist, and the alert-volume reduction the vendor advertises is precisely the lever a regulator scrutinizing an under-monitoring risk would question.

Sources: googlecloud2023

Appears on: /domains/cases/hsbc-aml, /pan-lab

Two structural dynamics govern fraud and financial-crime detection. Under extreme base rates, detection precision is dominated by the false-alarm rate rather than by accuracy, so at realistic prevalence a threshold change moves the burden of alerts rather than the truth of them (the base-rate fallacy). And the labels the model learns from are the investigators' own dispositions: only a small set of flagged transactions is ever verified, and models are retrained on the analysts' calls, so a rise in 'confirmed' activity is partly a measure of what the system taught its reviewers to confirm rather than an independent ground truth (the label-feedback loop).

Sources: axelsson2000a, dalpozzolo2018a

Appears on: /domains/cases/hsbc-aml, /pan-lab

A neobank's fraud algorithms — triggered heavily by pandemic-era government benefit deposits — froze and closed the accounts of legitimate customers at scale, holding their balances for thirty to more than ninety days, and the company admitted some of the closures were mistakes. The false-positive tail here lands on real people as immediate hardship, concentrated among benefit-deposit recipients and low-balance households for whom a frozen account means no access to funds for weeks.

Sources: kessler2021

Appears on: /domains/cases/chime-fraud, /pan-lab

A 2024 federal consent order priced the downstream operational failure rather than the model: thousands of consumers waited weeks to months for their balances after account closure, and the order imposed a 3.25 million dollar civil penalty plus at least 1.3 million dollars in consumer redress for the delayed refunds. The harm ran through three stages inside the organization's control — the scoring model's false positives, the operations backlog that turned a freeze into months without funds, and the refund process whose delay drew the regulator — and the enforcement attached to the last stage, the backlog, not to the model that started it.

Sources: consumerfinancialprotectionb2024

Appears on: /domains/cases/chime-fraud, /pan-lab

A Nordic bank's rules-based legacy fraud system ran at roughly 40 percent detection with a 99.5 percent false-positive rate — a measured pre-machine-learning baseline whose badness is the most credible datum in the record, since a 99.5 percent false-positive rate is not a marketing claim. The vendor-published rollout of a deep-learning engine scoring transactions in real time (under 300 milliseconds) claims false positives cut by about 60 percent and true-positive detection raised by about 50 percent; those figures are an organization-named, trade-press-covered vendor case study, entered here as claimed magnitudes against that legacy baseline because they were not independently audited.

Sources: teradata2017, groenfeldt2017

Appears on: /domains/cases/danske-fraud, /pan-lab

The same institution that improved its in-line fraud scoring later ranked worst among UK banks for reimbursing victims of authorized-push-payment scams in the regulator's bank-by-bank performance data — better detection and worse victim-outcome performance coexisting in one organization. And it was a rule, not a model, that moved the institutional behavior: the regulator's mandatory-reimbursement regime raised sector reimbursement from roughly two-thirds to about 89 percent, demonstrating that detection quality and the justice of the disposition are different levers held by different actors, and that the victim-outcome lever is a regulatory rule rather than a better classifier.

Sources: ukpaymentsystemsregulator2023

Appears on: /domains/cases/danske-fraud, /pan-lab

An internal team built an experimental recruiting engine — roughly 500 models scoring resumes one to five stars per role and location — trained on ten years of the company's own hiring decisions, a period whose hires were predominantly male. The models learned that history: they penalized the word 'women's' and downgraded graduates of women's colleges, reading gender proxies as negative signal. The team patched the identified terms but concluded that term-level fixes could not guarantee neutrality against unknown proxies, because the model had learned the pattern rather than the words, and the company scrapped the project around 2017; per the company, recruiters saw the tool's recommendations but it was never used as a sole ranking.

Sources: dastin2018

Appears on: /domains/cases/amazon-resume-engine, /pan-lab

Training a screener on an organization's past hiring decisions imports the past's selection function: research on hiring as exploration finds that models trained on prior hires raise hire rates but replicate historical selection, and that a screener which values exploration rather than only exploitation breaks that lock-in loop. Two governance lessons follow — the patch lever has a documented ceiling, since removing named proxies does not remove a learned correlation, and abandonment can itself be a governance outcome, taken here before any external harm was documented rather than after an adjudication.

Sources: li2020a

Appears on: /domains/cases/amazon-resume-engine, /pan-lab

An applicant-tracking platform whose AI screening and recommendation features operate inside thousands of employers' hiring pipelines at once is the subject of a live federal collective action testing whether the vendor is directly liable as the employers' agent. On the litigation record, the court sustained the agent theory at the dismissal stage in 2024 and preliminarily certified a nationwide age-discrimination collective in 2025, covering applicants forty and over since September 2020, on a record in which the lead plaintiff reported more than one hundred rejections across employers using the platform. The litigation is ongoing and nothing here is an adjudicated finding of discrimination; these are allegations and procedural rulings, not a verdict.

Sources: mobleyvworkday2024

Appears on: /domains/cases/workday-screening, /pan-lab

A 2026 discovery ruling in the same matter held the vendor's internal bias-testing data privileged because counsel had curated it — meaning the testing record exists and is legally unreachable, a configuration in which audit opacity is not the absence of testing but testing shielded from external verification. The case surfaces two further structural facts: a single vendor's screening model multiplied across many employer boundaries, so one learned defect can propagate as widely as the platform, and accountability diffusion between deployer and vendor, each holding part of the governance the other points to, against a survey backdrop showing assessment vendors' validation and bias-mitigation claims are often unverifiable from outside.

Sources: raghavan2020a, u2023b

Appears on: /domains/cases/workday-screening, /pan-lab

A graduate-hiring pipeline chained a games-based assessment with automated video-interview scoring, and the deployer reports roughly a 90 percent reduction in time-to-hire (from about four months to about four weeks), around 50,000 candidate interview hours saved, about one million pounds in annual savings, and a 16 percent improvement in diversity. Every one of those figures is company- or vendor-reported and none is independently audited, so they are the deployer's own dashboard rather than an external measurement — which is exactly what the family's service regime looks like from inside.

Sources: bestpracticeai

Appears on: /domains/cases/unilever-hiring, /pan-lab

Both vendors' audit machinery is on the public record in an honest but partial form. The games vendor underwent a cooperative academic audit with source-code access, in which its four-fifths-rule de-biasing pipeline was found faithfully implemented — with the independence caveat that vendor staff were co-authors — and the video vendor retired its facial-analysis input under scrutiny after internal research found visual features added only about 0.25 percent predictive power, publicizing a narrow-scope external audit. The family's structural blind spot applies in full: rejected candidates never re-enter the outcome data, so the claimed quality and diversity effects are measured on hires only.

Sources: wilson2021a, maurer2021a

Appears on: /domains/cases/unilever-hiring, /pan-lab

A machine-learning underwriting and pricing platform using education and other alternative data operated for five years under a regulator's no-action letter with a reporting obligation, and the regulator published the access results: 27 percent more applicants approved than a traditional model at 16 percent lower average APRs, with near-prime applicants (FICO 620 to 660) approved at roughly twice the rate, and gains across the tested demographic segments. This is the lending family's only regulator-verified service term. Underwriting is fully automated with no per-application human review, so the organizational levers are all upstream — model choice, the testing regime, the search for alternatives, and the reporting channel to the regulator.

Sources: consumerfinancialprotectionb2019

Appears on: /domains/cases/upstart-underwriting, /pan-lab

The same deployment carries the family's most detailed public fair-lending testing record: four reports from an independent monitorship agreed with civil-rights organizations found no close protected-class proxies quantitatively, but identified approval disparities for Black applicants, flagged a likely viable less-discriminatory alternative model, and ended in a documented methodological impasse over how hard the law requires an organization to search for such an alternative. Independently of the disparity question, adverse-action notices must give specific, accurate principal reasons for a denial regardless of the model's complexity — a governed explanation duty a complex model does not discharge by being accurate.

Sources: relmancolfaxpllc2021, consumerfinancialprotectionb2022b

Appears on: /domains/cases/upstart-underwriting, /pan-lab

A bank's automated credit-decisioning for a widely used consumer card was investigated by a state regulator after viral allegations of gender bias in credit-line assignment. The regulator analyzed roughly 400,000 in-state applicants and found no unlawful discrimination on a prohibited basis — the model was cleared on the numbers. But the same investigation documented failures of explanation, customer service, and perceived transparency: applicants could not learn why they received the terms they did, front-line staff could not explain the decisions, and the resulting opacity destroyed consumer trust even though the underwriting itself was found lawful. This is the domain's cleared-but-faulted case: a statistically clean model paired with a failed duty to explain.

Sources: newyorkstatedepartmentoffina2021

Appears on: /domains/cases/goldman-apple-card, /pan-lab

The lesson the cleared-but-faulted outcome carries is that a lawful, statistically clean model does not discharge the separate duty to explain a decision. Regulators have made explicit that adverse-action notices must give specific, accurate principal reasons regardless of how complex the model is, and that a model being a black box is not a defense — checking the nearest sample-form box does not comply. The explanation and customer-service channel is therefore a distinct, separately-resourced surface that can fail on its own: an organization can pass its fair-lending testing and still fail the people it decides on by being unable to tell them why.

Sources: consumerfinancialprotectionb2022b

Appears on: /domains/cases/goldman-apple-card, /pan-lab

A state attorney general reached a $2.5 million settlement with a student-loan lender over its AI underwriting. The documented conduct is the domain's cleanest failure-then-mandated-governance arc: the model used a cohort-default-rate feature — a school's aggregate default rate priced into an individual applicant's terms — that disparately impacted Black and Hispanic applicants, and an immigration-status rule that automatically denied certain non-citizen applicants, while the organization ran no disparate-impact testing and gave inadequate adverse-action notices. The remedy did not fine-and-close: it mandated the missing program — model governance, disparate-impact testing, documentation, and reporting controls — so the enforcement action wrote the governance the deployment had never built.

Sources: officeofthemassachusettsatto2025

Appears on: /domains/cases/earnest-ai-underwriting, /pan-lab

The mechanism the case turns on is the facially-neutral aggregate feature: a cohort default rate is a property of a school, not of the applicant, and no input names a protected class — yet pricing a group's aggregate history into an individual's terms can carry protected-class impact, which is exactly what disparate-impact testing exists to catch. Here that testing was not done, so the impact went unmeasured until an enforcement action found it. The remedy installed the program the deployment lacked, which is the governable reading: an aggregate feature can look neutral input-by-input and still produce a disparity only outcome testing would reveal, and the absence of that testing is itself the failure.

Sources: officeofthemassachusettsatto2025, consumerfinancialprotectionb2022b

Appears on: /domains/cases/earnest-ai-underwriting, /pan-lab

The strongest field evidence for an agent-assist copilot in customer service comes from a staggered randomized rollout of a generative-AI assistant to roughly 5,000 customer-support agents at a large software firm. Measured against a control group, the copilot raised issues resolved per hour by about 15 percent on average, and it also improved customer sentiment and agent retention. The gain, however, was sharply uneven: novice and low-skill agents improved by roughly 30 to 34 percent, agents with two months of experience performed like agents with six months and no AI, and the most experienced agents gained close to nothing, with some evidence of slight quality degradation. This is the contact-centre domain's cleanest measured benefit, and it is a distribution rather than a single number.

Sources: brynjolfsson2025a, brynjolfsson2023

Appears on: /domains/cases/fortune500-agent-copilot, /pan-lab

The lesson the randomized evidence carries is skill compression: an agent-assist copilot mostly raises the floor. Because almost the entire measured gain accrues to less-experienced agents and the most experienced gain close to nothing, an average productivity number overstates the effect for the agents who least need it and hides that the tool does little for the experienced while possibly costing a small amount of quality there. The governable reading is that the benefit must be measured as a distribution across agent skill, not reported as a scalar — a copilot that helps novices a great deal and experts not at all is a real and specific benefit, and describing it with one average misstates who it helps.

Sources: brynjolfsson2025a

Appears on: /domains/cases/fortune500-agent-copilot, /pan-lab

An airline's customer-facing website chatbot told a customer they could claim a bereavement fare retroactively — a policy that did not exist. The customer relied on the chatbot's statement, bought a ticket, and was then refused the fare by the airline's human staff. A civil-resolution tribunal found the airline liable for negligent misrepresentation and awarded damages, and in doing so rejected the airline's argument that the chatbot was a separate legal entity responsible for its own actions. The tribunal held that the organization is responsible for all the information on its website, whether it comes from a static page or a chatbot, and that a customer has no way to know which source to trust. This is the contact-centre domain's cleanest accountability ruling: the bot is a tool the company answers for, not an entity that answers for itself.

Sources: moffattv2024, sookman2024

Appears on: /domains/cases/air-canada-chatbot, /pan-lab

The duty the ruling establishes is that an organization must take reasonable care that its chatbot's representations are accurate, because the chatbot is a tool it deploys rather than a separate entity that answers for itself. A hallucinated policy or a wrong rule stated by the bot is therefore the organization's own misrepresentation, and a posture that treats the AI as speaking only for itself does not transfer that responsibility away. The governable reading is that a customer-facing chatbot is a channel the organization is accountable for exactly as it is accountable for a page on its own website — so the accuracy control on what the bot states, and the ownership of what it says, are the organization's to build, not the bot's to carry.

Sources: sookman2024, moffattv2024

Appears on: /domains/cases/air-canada-chatbot, /pan-lab

An organization published striking first-month numbers for its customer-facing AI assistant: it handled about two-thirds of customer-service chats (some 2.3 million conversations), was described as doing the equivalent work of about 700 full-time agents, cut average resolution time from about 11 minutes to under 2, was said to match human customer satisfaction, and was projected to improve profit by tens of millions. Every one of those figures was self-reported and not independently audited. Roughly a year later the same organization reversed course on quality grounds — its chief executive said cost had become too predominant an evaluation factor and the result was lower quality — and committed to always keeping a human available to customers who want one. This is the contact-centre domain's cleanest benefit-then-cost arc: the deflection numbers and the walk-back come from the same deployment.

Sources: klarnabankab2024, ivanova2025

Appears on: /domains/cases/klarna-ai-assistant, /pan-lab

The lesson the benefit-then-cost arc carries is that deflection is not resolution. A published deflection number reports how many contacts the AI handled, not whether it handled them well, and a figure that is impressive on cost can hide a quality cost that only shows up later — which is what the organization's own reversal described. The survey backdrop sharpens it: most customers say they would rather not meet AI in service and fear it makes reaching a human harder, and industry analysts expect a large share of organizations to abandon plans to reduce their customer-service workforce with AI. The governable reading is to measure resolution and repeat contact against deflection rather than counting deflection as a win by itself, and to protect the path to a human as the safety valve a deflection-maximizing design tends to erode.

Sources: ivanova2025, gartner2025b

Appears on: /domains/cases/klarna-ai-assistant, /pan-lab

A video platform ran an unintended natural experiment on automated content moderation. When the pandemic sent its human reviewers home, the platform said it would rely more on automated removal and deliberately chose over-enforcement rather than let harmful content stay up. The result, from the platform's own transparency reporting, was that automated removals more than doubled in a single quarter (to about 11.4 million videos), appeals roughly doubled, and the reinstatement rate on appeal jumped from about 25 percent to about 50 percent. The platform also withheld strikes where no human had reviewed the removal, treating the automated decision as provisional. The doubling of the reinstatement rate is the finding: it is direct evidence that the automation was making roughly twice the rate of catchable errors, and that the human review and appeals path was the loop catching them.

Sources: youtubegoogle2020

Appears on: /domains/cases/youtube-covid-enforcement, /pan-lab

The lesson the natural experiment carries is that the human review and appeals path is the error-correction loop for automated enforcement, not an optional add-on. Automated moderation makes errors at scale, and a doubling of the reinstatement rate when human review thinned is a measurement of those errors — they were always being made at that rate, and were visible only because the appeals queue surfaced them. Two things follow. Over-enforcement versus under-enforcement is a chosen trade-off: with review capacity cut, the organization decided which error to make, and that was a governance decision. And proactive removal acts before anyone sees the content, so an over-broad takedown is invisible unless appealed — and some removals are irreversible, as when automated systems destroyed documentation of war crimes with archival access declined, leaving no correction loop at all.

Sources: youtubegoogle2020, humanrightswatch2020

Appears on: /domains/cases/youtube-covid-enforcement, /pan-lab

A platform enforces its content standards with automated classifiers at a scale no human team could match, backed by a layered correction structure: an internal appeals process, and above it an external oversight board that issues binding decisions on the individual cases it takes and non-binding policy recommendations to the platform. In one year the board overturned the platform's original decision in around 90 percent of the cases it decided, and the platform reported implementing, in progress on, or already aligned with the large majority of the board's cumulative recommendations. This is the moderation domain's most built-out, institutionalized correction structure — layered appeals rising to an independent-adjacent external body that publishes its reasons.

Sources: metaplatforms, oversightboard2024

Appears on: /domains/cases/meta-content-enforcement, /pan-lab

The reach of the correction structure is the governable limit. The roughly 90 percent overturn rate is measured on selected cases — the board chooses emblematic disputes to set precedent, so the figure is evidence that escalated decisions were often wrong, not a random error rate, and the overwhelming majority of automated enforcement decisions never reach the board at all. The board is funded through a platform-established trust, which makes it independent-adjacent rather than fully independent, and its policy recommendations are non-binding. The honest reading is that this correction structure is real and genuinely better than most, and its reach is bounded to the tiny fraction of cases that escalate — so the governing question is whether the correction reaches the scale of the enforcement it is meant to check.

Sources: oversightboard2024, oversightboard2025

Appears on: /domains/cases/meta-content-enforcement, /pan-lab

A media outlet published AI-drafted finance explainers under a human-sounding staff byline without disclosing to readers that the articles were machine-written. When the practice came to light, the outlet's own audit found it had to issue corrections on a majority of the AI-written articles — on the order of 41 of 77. A byline implies a human review that the reader trusts, and a correction rate that high is a direct measurement that the review the byline implied was not actually performed before publication. A later and sharper case saw another outlet publish articles under entirely fabricated author personas presented as real people, so the failure ran from undisclosed AI drafting to invented human bylines.

Sources: bonifacic2023, harrisondupre2023

Appears on: /domains/cases/cnet-ai-drafting, /pan-lab

Editorial AI moves the failure from a takedown to a publication, but the governable structure is the same as in moderation: the byline is the accountability object, and it stands for a review that either happened or did not. Two things are owed to the reader — disclosure that AI was involved, and an editorial check that actually took place — and this deployment gave neither, publishing under a staff byline that implied both. When a large share of AI-drafted articles needs correction, the review was not performed, and the byline misrepresented who did the work. The governable reading is that a human byline on machine-drafted content is a claim about review and disclosure, and a high correction rate is the evidence that the claim was false.

Sources: bonifacic2023

Appears on: /domains/cases/cnet-ai-drafting, /pan-lab

A large automaker deployed in-line AI inspection at production scale: camera and acoustic systems that detect defects during assembly and feed real-time flags to the line worker via a smart device, on a line running on the order of a thousand-plus vehicles a day at a takt of under a minute per station. The system has been established as a company standard and is being extended to suppliers. The governing design is that the AI flags and a human on the line responds — the inspection is wired into a resourced response loop, including the ability to stop the line, so the benefit runs through the human response the flag triggers rather than through the model alone. The documented facts here are the system's function, the worker-interaction model, the scale, and the standardization; the deployment's benefit is reported through corporate and trade channels, and defect-rate deltas from a primary source are not public.

Sources: bmwgrouppressclub2025, metrologyandqualitynews2026, leanenterpriseinstitute

Appears on: /domains/cases/bmw-aiqx-inspection, /pan-lab

The lesson the deployment carries is that an in-line inspection AI is only as good as the human-response loop it triggers, and that loop is the governable object. When the AI flags a defect, a resourced response — a worker with the time to check the flag and the authority to stop the line — is what turns a detection into a caught defect; without it, the flag is just a decision no one acts on. This is why the failure modes in this domain are matters of the loop's calibration rather than the model's raw accuracy: too many false alarms and operators stop responding, too much trust and they stop checking. The honest boundary is that the benefit is reported through corporate and trade channels, and no named manufacturer has publicly attributed a shipped-defect escape to its AI inspection, so the response loop is drawn as the resourced strength and its calibration as the thing to govern, not as a claim about defects that did or did not ship.

Sources: bmwgrouppressclub2025, leanenterpriseinstitute

Appears on: /domains/cases/bmw-aiqx-inspection, /pan-lab

A peer-reviewed heavy-industry predictive-maintenance case study achieved a large, measured reduction in false alarms — on the order of 90 percent — through a closed operator-feedback loop: the maintenance crews investigated the alerts, labeled which were real, and the model retrained on those labels, so the false-alarm rate fell sharply over successive rounds. This is the industrial-QA domain's best-measured quantitative benefit, and it comes from an anonymized study site rather than a named-manufacturer press release, which is the pattern across this domain — the peer-reviewed magnitudes are at anonymized or smaller sites, while the named deployments report their benefit through corporate and trade channels.

Sources: hermansa2021a

Appears on: /domains/cases/heavy-industry-pdm, /pan-lab

The lesson the case carries is that the same closed feedback loop that produced the benefit is the thing that can break it, because the loop depends on the crews continuing to engage with the alerts — investigating them, labeling them, responding — and that engagement fails in two opposite directions. Alert fatigue: if too many false alarms arrive before the loop has tuned them down, crews stop trusting the alerts and stop responding, so the feedback the model needs to improve never arrives and the loop stalls. Automation bias: if crews defer to the alerts and stop applying their own judgment, the labels the model retrains on become an echo of its own calls rather than an independent check. Either way the loop degrades, so the measured benefit is contingent on the loop staying calibrated — enough trust that crews respond, enough independence that their labels still carry real judgment.

Sources: romeo2025a, wittbold2026

Appears on: /domains/cases/heavy-industry-pdm, /pan-lab

Machine-learning automated visual inspection of filled injectable drug products flags particulate and cosmetic defects that manual inspection or fixed-rule cameras would otherwise judge. In this safety-critical, regulated manufacturing setting the error trade-off is asymmetric and deliberate: a false accept — a missed defect in an injectable that reaches a patient — is a patient-safety failure, while a false reject — scrapping a good vial — is a cost, so the system is tuned to over-reject rather than risk a miss. Because the inspection sits inside a validated pharmaceutical quality process, the AI cannot simply be switched on; it must be qualified within that process, and a regulator is actively developing the framework for how AI in drug manufacturing should be validated and monitored.

Sources: veillon2023a, usfda2023

Appears on: /domains/cases/pharma-avi-inspection, /pan-lab

Two governable surfaces follow from putting AI inside a regulated inspection. First, qualification: an AI in a validated quality process is not simply deployed but must be qualified and monitored for drift, and because the regulator's AI-specific framework is still developing, the qualification of the model's behavior over time is an emerging, not-yet-settled check rather than a solved one. Second, the human backstop: the manual inspector is what catches the false accepts the over-reject tuning is meant to avoid, so if inspectors come to defer to the AI and stop scrutinizing, that backstop erodes exactly where it matters most — the missed defect the asymmetric tuning was designed to prevent. The governable reading is that the over-reject tuning lowers the visible risk without removing it, and the qualification and the human backstop are what keep the residual risk covered.

Sources: veillon2023a, usfda2023

Appears on: /domains/cases/pharma-avi-inspection, /pan-lab

A state's statewide dropout early-warning system used ensemble machine learning to label every grade 6 to 9 student's risk of not graduating on time and delivered the label to school staff through dashboards for about a decade. An independent, decade-scale audit found the system was wrong roughly 74 percent of the time when it predicted a student would not graduate, produced higher false-alarm rates for Black and Hispanic students, and that the deployer's own internal equity research had gone unpublished — while a survey of districts found administrators reporting no training on how to interpret a 'high risk' label. The state stopped publishing the dashboards in 2023 and said it was evaluating the system's future. The deployment is the education domain's clearest case of a risk label whose error and group disparity entered how students were seen rather than the help they received.

Sources: feathers2023, knowles2015a, wisconsindepartmentofpublici2023

Appears on: /domains/cases/wisconsin-dews, /pan-lab

The lesson the case carries is that a risk label is only as good as the intervention it triggers and the training of the human who reads it. A label that is wrong most of the time, delivered to staff with no guidance on interpreting it, imports the model's error and its group disparity into how students are perceived rather than into a resourced response — the flag becomes a lens on the student rather than a trigger for help. Set against this, a large district's transparent, low-tech on-track indicator, built on interpretable research and paired with real intervention, accompanied a rise in graduation to a record level. The contrast locates the benefit in the intervention the indicator makes legible enough for staff to act on well, not in the sophistication of the prediction — an interpretable indicator that drives help can outperform an opaque model that only labels.

Sources: allensworth2007, feathers2023

Appears on: /domains/cases/wisconsin-dews, /pan-lab

A public university required students to pan their webcam around their home before an online exam, using remote-proctoring software that flags suspected cheating from the video. A federal court held that the pre-exam room scan was an unreasonable search under the Fourth Amendment — a first-of-its-kind ruling that a routine proctoring practice violated a student's constitutional rights in their own home. Separately, peer-reviewed measurement of automated proctoring found the software produced more face-detection failures, more red flags, and higher priority scores for darker-skinned and Black students, with no corresponding difference in actual cheating. The deployment is the education domain's clearest case of surveillance-based integrity AI whose costs — a rights violation and a demographic burden of suspicion — are each independently established.

Sources: ogletreev2022a, yoderhimes2022a

Appears on: /domains/cases/cleveland-state-proctoring, /pan-lab

The lesson the case carries is that surveillance-based integrity AI is not a free default: it carries a rights cost that can be independently adjudicated and a demographic burden that can be measured, and both are owed a reckoning before the surveillance is imposed, not after a court or an audit finds the harm. A room scan of a student's home was held to be an unreasonable search, so the surveillance has a rights dimension a court can rule on regardless of the integrity goal. And because the software flags darker-skinned and Black students more often with no more actual cheating, and a flag is an accusation the student must answer, a disparate flag rate is a disparate burden of suspicion. The governable surfaces are the proportionality of the surveillance to the integrity problem it is trying to solve, and the measured flag rate by group.

Sources: yoderhimes2022a, ogletreev2022a

Appears on: /domains/cases/cleveland-state-proctoring, /pan-lab

A parcel carrier's route-optimization system is a documented operations-research success: it re-optimizes delivery routes across the fleet and was reported to save on the order of 100 million miles and about 10 million gallons of fuel a year, a genuine and peer-reviewed efficiency gain. The same system that computes the efficient route also dictates it to the driver and monitors adherence through vehicle telematics, so the efficiency is enforced through workplace surveillance — the optimization and the monitoring are one system, and the driver's discretion over how to run the route is what it replaces. The benefit is real and measured in miles and fuel; the cost is the driver autonomy the enforcement removes and the surveillance the enforcement requires.

Sources: holland2017, levy2023

Appears on: /domains/cases/ups-orion-routing, /pan-lab

The lesson the case carries is that an optimization which manages the worker executing it couples the efficiency gain to a cost the efficiency metric does not see: the worker's autonomy, and the surveillance required to enforce the plan. The system measures miles and fuel, not whether the pace it sets is feasible for a person or whether the monitoring it requires is proportionate — so the governable surfaces are whether the optimization internalizes the human executing it, meaning a route that is feasible and humane rather than merely optimal on paper, and whether the surveillance that enforces it is governed rather than treated as a free byproduct of routing. An optimization is a success on its own terms and can still externalize a cost onto the worker that never appears in the miles-and-fuel number it reports.

Sources: levy2023, holland2017

Appears on: /domains/cases/ups-orion-routing, /pan-lab

A warehouse operation's algorithmic management pairs a genuine, peer-reviewed human-robot picking benefit — robots and workers collaborating to raise throughput, documented in the operations-research literature — with a documented injury-productivity trade-off. When the algorithm sets the pace of the physical work, a federal safety regulator cited the operation for exposing workers to ergonomic hazards, and a legislative inquiry tied the speed the system demands to warehouses it described as uniquely dangerous. The productivity gain and the worker-injury risk are therefore coupled: the same pace that raises units per hour is the pace regulators and the inquiry connected to injury. The benefit is real and the injury cost is separately documented, one in the OR literature and one in safety-inspection findings and a legislative report.

Sources: allgor2023, ussenatecommitteeonhealth2024, usdepartmentoflabor2023a

Appears on: /domains/cases/amazon-fulfillment-management, /pan-lab

The lesson the case carries is that when an algorithm sets the pace of physical work, the productivity metric it optimizes — units per hour — cannot see the cost the pace imposes on the body executing it. The injury shows up in safety-inspection data and a legislative inquiry, not on the throughput dashboard, so a productivity number can rise while the cost accumulates unrecorded on the metric that reports success. The governable question is whether the pace-setting internalizes the worker's safety, treating a sustainable rate as part of what 'optimal' means, or externalizes it as an injury the metric never records. Ethnographic research describes this algorithmic management as a 'game' whose rules the worker cannot change, which is what makes the pace a management decision the organization owns rather than a fact of the work.

Sources: cheon2025, ussenatecommitteeonhealth2024

Appears on: /domains/cases/amazon-fulfillment-management, /pan-lab

A federal asylum agency uses dialect-recognition AI to estimate an applicant's country or region of origin from a short speech sample, as one input into the credibility assessment of their claimed origin. The tool's reliability is limited: government-reported recognition is around 80 percent for Arabic — roughly a 20 percent error rate — and computational linguists judge separating some closely related language varieties close to hopeless. The agency's own caseworkers describe the tool as only a rough compass, too imprecise to resolve the hard cases, and its outputs as clues rather than determinations. Used honestly as one clue among several it is defensible; the documented risk is that an imprecise output acquires more authority than its accuracy supports, in a determination where the state's tool is set against the applicant's own account of who they are.

Sources: lulamae2022a, scheel2024a

Appears on: /domains/cases/bamf-dias-dialect, /pan-lab

Two governable surfaces follow from putting a low-reliability signal into a high-stakes credibility determination. First, whether the tool's documented imprecision actually bounds the weight it carries: a rough compass treated as one is honest, but the same output can harden into a credibility finding it cannot support once a phrase like the software indicates a particular origin enters the record and confronts the applicant. Second, whether the applicant can see and contest the signal: in asylum determinations the person with the most at stake and the most knowledge of their own origin is often unable to see or challenge the AI's estimate, so the correction that would catch an error is severed on exactly the side that holds the truth. The governable reading is that reliability must bound authority, and the affected person must be able to contest a signal used against them.

Sources: scheel2024a, lulamae2022a

Appears on: /domains/cases/bamf-dias-dialect, /pan-lab

A government's immigration-enforcement triage algorithm identifies and recommends people for enforcement actions — returns, bail conditions, casework — drawing on sensitive data including detention, health, vulnerability, and location-monitoring records. Uncovered through roughly a year of freedom-of-information litigation, its training materials show an asymmetric override design: officials must record a justification for rejecting a recommendation but not for accepting one. That design builds a rubber-stamping incentive into the workflow — accepting the algorithm is frictionless, overriding it requires work — so the human in the loop is nominal rather than a real check. It is the corpus's clearest documented instance of automation bias engineered into an agency workflow, in one of the highest-stakes enforcement settings a state operates.

Sources: privacyinternational2024b, privacyinternational2024a

Appears on: /domains/cases/home-office-ipic, /pan-lab

The lesson the case carries is that nominal human oversight is not real oversight. An asymmetric override — where accepting the algorithm's recommendation is frictionless and rejecting it requires a recorded justification — engineers automation bias into the process by making deference the path of least resistance, so the claim that a human makes the final decision can be true and empty at once. Two governable surfaces follow. Whether the review is genuinely symmetric: the official as free and as prompted to reject as to accept, so an error is as likely to be caught as waved through. And whether the affected person is told the AI is used and can contest it: applicants are frequently not told, which severs the correction on the side that could challenge the recommendation, so the one check that survives the asymmetric override — the person it is about — is cut out too.

Sources: privacyinternational2024c, privacyinternational2024a

Appears on: /domains/cases/home-office-ipic, /pan-lab

A large automaker developed an in-house deep-learning system to detect hairline cracks in pressed sheet-metal parts, trained on several terabytes of images drawn from seven presses at its home plant plus several sister plants, in development since mid-2016 and tested for series deployment. The documented change is a generational replacement: the system takes over an inspection duty previously performed by manual visual checks plus fixed-rule camera systems, rather than augmenting a human inspector's judgment on each part. The record — a reprint of the manufacturer's own press material with its CIO quoted — documents the development lineage, the data scale, and what the system replaced; it publishes no quantitative defect-rate figures, so the deployment's benefit magnitude is a corporate claim, not an audited measurement.

Sources: justauto2018

Appears on: /domains/cases/audi-press-shop-inspection, /pan-lab

The governance shape of this deployment is inheritance rather than assistance: by replacing the manual visual check and the fixed-rule camera generation, the learned system inherits the whole inspection duty for the defect class it covers, so there is no per-part human judgment running alongside it to catch what it misses. Its training data is pooled across presses and plants, which means one model's blind spots are correlated across every line it inspects. The failure regime is mechanism-level — drift as dies wear and parts change, complacency over an inspection nobody re-performs — because no named manufacturer, including this one, has publicly attributed a shipped-defect escape to its AI inspection.

Sources: justauto2018

Appears on: /domains/cases/audi-press-shop-inspection, /pan-lab

A rail service provider operates sensor-based predictive maintenance on a high-speed fleet under a priced availability contract: roughly 300 sensors per train read at five-minute intervals (on the order of a million readings per train-year), overlaid with human-written failure reports, maintained against a promise that refunds the full fare if a journey is delayed more than fifteen minutes. The documented results are only one noticeably delayed journey in 2,300 (by five minutes) and discovered failure signatures such as an engine-temperature pattern preceding failure by three days. The fleet outcome is documented in a vendor-side trade case study; the analytics' own precision is not published, so the model-level figures remain unstated while the operational outcome is on the record.

Sources: rcrwirelessnews2016

Appears on: /domains/cases/siemens-renfe-velaro-pdm, /pan-lab

The governance shape of this deployment is uptime-as-contract: the party that operates and tunes the analytics is the service provider who pays for misses under the refund promise, so the incentive to prevent a delay is priced into the same organization that holds the model levers - an alignment the domain's other deployments lack. Two documented dependencies temper it: the failure signatures were discovered by joining sensor streams to human-written failure reports, so the discovery loop runs on documentation crews write for their own purposes; and a continuous sensor stream is the analytics' only view of the machine, so a failing sensor and a failing train arrive looking the same until someone goes and looks.

Sources: rcrwirelessnews2016

Appears on: /domains/cases/siemens-renfe-velaro-pdm, /pan-lab

A large urban school district operationalized a transparent ninth-grade indicator - course credits earned plus no more than one core-course failure - from consortium research showing it predicts high-school graduation with about 85 percent accuracy, and wired it to school-level attention rather than to an opaque score. District graduation rates subsequently rose to record highs. The indicator is a rule anyone can read: a teacher can explain to a student exactly why they are off-track and exactly what would change it, so the contest-and-correction loop that opaque early-warning deployments sever is open by construction.

Sources: allensworth2007

Appears on: /domains/cases/cps-freshman-ontrack, /pan-lab

The documented limits are as instructive as the result. The indicator's accuracy and the district's graduation rise are associational at district scale - no randomized trial assigns schools to use it - and the benefit mechanism runs through the intervention, not the flag: an indicator wired to attention still depends on the attention being resourced, and the research base's central finding is that what predicted graduation was a condition schools could act on (freshman-year course performance), not a fixed trait of the student. The rule's power is that it points at something changeable, and the district's practice is what changed it.

Sources: allensworth2007

Appears on: /domains/cases/cps-freshman-ontrack, /pan-lab

A storied sports outlet published product-review articles under entirely fabricated author personas - invented names, AI-generated headshots, fictional biographies - produced by a third-party content contractor, with the AI involvement disclosed to no reader. An investigation surfaced the personas by reading the public site; the articles were then deleted rather than corrected, the outlet attributed the content to the contractor, and the parent company's chief executive was subsequently fired. The corroborated record documents the fabrication, the deletion, the contractor attribution, and the executive consequence.

Sources: harrisondupre2023, npr2023

Appears on: /domains/cases/sports-illustrated-advon, /pan-lab

The governance failure ran across an organizational seam: the drafting, the bylines, and the personas were produced by a contractor, and the outlet's editorial function demonstrably did not operate across that boundary - the fabrication was discovered by outside investigation, not by any internal check, and the accountability afterward ran through contract and employment rather than through any editorial process. A byline makes two claims to the reader - that a person produced this, and that the outlet's review stands behind it; this deployment fabricated the first and vacated the second, and the deletion afterward removed the evidence rather than correcting the record.

Sources: harrisondupre2023, npr2023

Appears on: /domains/cases/sports-illustrated-advon, /pan-lab

A parcel firm's customer-facing support chatbot, after a system update, was prompted by a customer into swearing and into composing a poem calling its own operator the worst delivery firm in the world. The firm attributed the behavior to the update and disabled the AI element immediately. The documented governance facts are exactly two: the update preceded the behavior, and the off switch worked - the firm learned of the incident from the customer's viral post rather than from any release gate, but the disablement was immediate once it knew.

Sources: itvnews2024

Appears on: /domains/cases/dpd-uk-chatbot, /pan-lab

The failure shape is a change-management regression, not a wrong policy or a deflection metric: guardrails that had held in production stopped holding after a change, publicly, within hours, in a channel that talks to anyone. What the deployment lacked was a release gate between the update and the public - the constraint layer's behavior after the change was tested by a customer with a prompt, not by the firm with a suite - and the discovery path ran through screenshots of one conversation going viral.

Sources: itvnews2024

Appears on: /domains/cases/dpd-uk-chatbot, /pan-lab

Since May 2018, New York City's Administration for Children's Services has scored every open child-protection investigation at day 10 with an in-house machine-learning model (documented as the ASAP Tool / Severe Harm Predictive Model), rank-ordering cases by predicted likelihood of substantiated physical or sexual abuse within 24 months to fill a quality-assurance review worklist of about 3,000 of roughly 50,000 investigations a year; the LL35 register states scores are not shared with staff in the QA unit or the investigative unit, families and their attorneys are not told when a case is flagged, ACS told state auditors there would be 'no basis for a complaint' about a predictive model on an individual case, and the agency's own internal audit acknowledged the training data likely included implicit and systemic biases, that geographic variables may act as partial proxies for race, and that flag predictions are more likely to be incorrect than correct.

Sources: nycofficeoftechnologyandinno2026, lecher2025, newyorkstatecomptroller2023

Appears on: /domains/cases/nyc-acs-qa-risk-algorithm

The May 2026 'Access Denied' report by the NYC Department of Investigation — the Charter-mandated inspector general for ACS — documents that five provisions of NY Social Services Law, as applied by state OCFS, routinely deny, limit, or delay DOI's access to ACS child-welfare records, with unfounded-report and CARES records prohibited entirely; DOI was barred from the full case history in 17 of the 18 child fatalities with prior ACS involvement reported to it in 2025 (13 of 16 in 2024; 19 of 25 in 2023). The report does not mention the algorithm — the linkage is a topological inference — but in 2025 ACS expanded the model to score CARES alternative-response cases, the record class state law prohibits DOI from accessing; the NYC Council's GUARD Act (passed unanimously November 25, 2025) legislated an Office of Algorithmic Data Accountability whose implementation status remains unverified as of mid-2026.

Sources: newyorkcitydepartmentofinves2026, nycofficeoftechnologyandinno2026, statescoop2025

Appears on: /domains/cases/nyc-acs-qa-risk-algorithm

The ACS severe-harm model trains on and scores exclusively from the agency's own administrative records (including hotline call counts and durations, prior involvement, and geography), and the intervention its flags trigger — extra interviews, collateral contacts, service referrals, consults, and QA documentation follow-ups — writes new activity into those same records, enriching the prior-involvement features of any future report on the family; ACS has never studied what effect the flag-triggered extra review has on downstream case outcomes, including whether a child is ultimately removed from the home (while telling state auditors it produces quarterly internal reports tracking the model's use in the QA program), kept no logs of model performance evaluations or updates as of the 2019-2022 state-audit fieldwork, and its claim that the model outperformed experienced caseworkers with fewer false positives and more race/ethnicity equity is an agency self-claim with no published methodology or independent verification.

Sources: lecher2025, newyorkstatecomptroller2023

Appears on: /domains/cases/nyc-acs-qa-risk-algorithm

Colorado's legislature built a distinctive oversight topology — a standing independent statutory ombudsman (created 2010; an independent judicial-department agency since 2016 under C.R.S. 19-3.3) was handed a funded, nine-criteria audit mandate (HB 24-1046, signed May 28, 2024; $109,392 appropriation) over the statewide Family Safety and Family Risk Assessment tool suite used by 64 county-administered agencies — and the resulting ICF audit (176 pages, dated February 27, 2026, released March 2, 2026, with 50 recommendations across nine directives) found the tools well-aligned with policy on paper but inconsistently implemented: only 32% of surveyed caseworkers complete assessments in real time, vignette agreement fell to 59–63% on neglect and poverty scenarios versus 91–96% on abuse scenarios, and the audit recommends redefining safety and risk with observable behavior-based criteria and revising or replacing the actuarial risk tool.

Sources: icfincorporated2026, steffen2026, coloradogeneralassembly2024a, coloradorevisedstatutes2024

Appears on: /domains/cases/colorado-safety-risk-tools-audit

Colorado's statutory audit loop fired end to end on the way up — the Child Protection Ombudsman's complaint-stream observations became testimony to the 2023 Child Welfare System Interim Study Committee (including its brief stating the Family Safety Assessment had never been validated since its 1999 inception), the testimony became HB 24-1046 requiring the ombudsman rather than the child welfare agency to procure a third-party audit, and ICF's competitively procured audit returned to named legislative committees by the March 1, 2026 statutory deadline with four public information sessions following — but as of July 2026 the loop has produced statute, budget, a public audit artifact, and public information only: no follow-up legislation has been enacted, no formal CDHS response has been identified, and the tools remain in statewide operation unchanged, a decade after the state auditor's October 2014 performance audit flagged overlapping assessment and documentation weaknesses without producing redesign.

Sources: coloradogeneralassembly2024, childprotectionombudsmanofco2026, steffen2026, officeofcoloradoschildprotec2023, coloradoofficeofthestateaudi2014, icfincorporated2026

Appears on: /domains/cases/colorado-safety-risk-tools-audit

The ICF audit found Colorado's actuarial Family Risk Assessment 'heavily weights prior reports, which inflate risk scores for incidents that occurred long ago, and disproportionately impacts minority families and perpetuates systemic bias' — with historical risk factors that 'disproportionately elevate scores for families of color' and a DV history item that codes any past incident, even decades old, as recent, penalizing survivors — while the record substrate degrades the oversight signal meant to catch exactly this: 68% of surveyed caseworkers cannot complete assessments in real time (documentation is back-filled into Trails after decisions), and race and ethnicity are so inconsistently documented that the audit says the disproportionality analysis requested by the legislature was limited; these are structural and qualitative audit findings, as no administrative-data disparity analysis has been published.

Sources: icfincorporated2026

Appears on: /domains/cases/colorado-safety-risk-tools-audit

Family-Match, a proprietary two-sided 'relational fit' adoption matching algorithm built for Adoption-Share by former eharmony researchers, produced 1 known adoption in Virginia's two-year test (an official's statement to the AP; VDSS said the tool 'had not proven effective'; the pilot's end is undated in the record) and 2 adoptions in Georgia's year-long pilot ended October 2022; caseworkers in Florida, Georgia, and Virginia said it often led them to unwilling families, and Virginia social workers were perplexed that the algorithm seemed to match all the children with the same group of parents. In Florida the vendor's own quarterly report claimed 603 placements yielding 431 adoptions over five years — figures partner agencies could not verify: FamiliesFirst Network's records showed 76 Family-Match placements with no documented adoption plus 3 failed trial placements since 2019, and Children's Network of Southwest Florida counted 22 matches and 8 adoptions in five years while making hundreds of matches and hundreds of adoptions without the tool over the same period.

Sources: hoandburke2023a, fortuneassociatedpressrepubl2023

Appears on: /domains/cases/family-match-adoption-share

The outcome evidence for Family-Match lived in a vendor-owned, credit-claiming data store: Virginia officials said that once families' data was entered 'Adoption Share owned the data,' assistant director Traci Jones said 'We did not have access to the algorithm even after it was requested,' and agencies 'couldn't explain Family-Match's self-reported data.' The vendor's April 2023 'confidential' user guide instructed caseworkers not to delete cases matched outside the tool but to document them in the system 'so that Adoption-Share could refine its algorithm and follow up with the families'; Miami's Citrus Family Care Network said vendor staff asked social workers to have parents register in the tool even when it played no role in the adoption, and Georgia's spokesperson said Family-Match could claim credit for pairings already in its system. In Georgia the vendor-owned store holds whether foster youth have been sexually abused, the gender of their abuser, criminal records, and whether they identify as LGBTQIA — data typically restricted to secured child protective services case files — and two Florida agencies fed the system data with no contract and would not say how children's data was secured.

Sources: hoandburke2023a, fortuneassociatedpressrepubl2023

Appears on: /domains/cases/family-match-adoption-share

Family-Match's four-state lifecycle turned on procurement authority actions made while states were structurally dependent on the vendor's own ledger for outcome evidence: Georgia ended its pilot on null results in October 2022, then — after Ramirez met with the governor's office and lobbied a statehouse committee — signed a new agreement in July 2023 for free; Virginia dropped the matching pilot as 'not proven effective' and by 2022 awarded Adoption-Share a larger contract for the Faster Families Highway recruitment portal ($188,000 budgeted / $212,546 spent SFY2023, $246,100 planned SFY2024, all 120 local departments enrolled by December 31, 2022, with policies and procedures — including family removal and response-time expectations — drafted only from February 2023, after statewide enrollment); Florida converted a philanthropy-funded rollout to a $350,000 DCF contract in October 2023 and added a Florida DOH contract for a medically-complex-children algorithm; and Tennessee, the only state whose reviewers formally questioned pre-deployment why the tool needed certain sensitive data points and how they influenced the match score, scrapped the rollout before it began. No state has ever published an evaluation of the matching pilot.

Sources: hoandburke2023a, virginiadepartmentofsocialse2023

Appears on: /domains/cases/family-match-adoption-share

Five US states (Michigan 2001, Minnesota 2001/2006, Maryland 2009, Texas 2013, Missouri 2021) operate 'Birth Match' programs in which an identity record-join between new birth registrations and a registry of parents with prior terminations of parental rights, serious-harm findings, or specified convictions fires a mandatory child-protective response with no risk score, no threshold, and no screening discretion: Michigan's current policy (PSM 712-2, reissued 2026-04-01) permits screen-out only for an inaccurate identity match or an already-open case and otherwise requires the referral to be screened in and assigned for investigation, and Maryland's 2025 response policy limits screen-out to a non-qualifying conviction or an adopted child while requiring a 24-hour intake coded 'Risk of Harm: birth match' — a mandatory response with no screening discretion, formally structured in Maryland as a voluntary non-CPS assessment families may refuse, with tracing and escalation duties attached. The only tunable parameters anywhere are registry scope and lookback: 2 years in Texas, 10 in Maryland and Missouri, unlimited in Michigan (records to 1978) and Minnesota.

Sources: cohen2022, michigandepartmentofhealthan2026, marylanddepartmentofhumanser2025, minnesotarevisorofstatutes2021

Appears on: /domains/cases/us-birth-match

The Birth Match trigger is self-arming: the registry that fires it is populated by the system's own outputs, so a birth-match response that ends in a new termination of parental rights writes the parent back onto the match list — permanently in Michigan and Minnesota, which have no lookback limit or retirement mechanism — and every subsequent birth re-fires with strictly increasing history; Michigan additionally adds manual list entries for severe cases without any termination. Maryland law accelerates the loop by letting the agency skip reasonable reunification efforts on account of a prior involuntary termination, and the 2025-26 Maryland legislative fight (HB 944 of 2025, died without a committee vote; HB 48 of 2026, heard 2026-01-29 and dead in committee without a vote when the session adjourned sine die on April 13, 2026) targeted that waiver, not birth-match repeal — with Civil Rights Corps testifying that the combination is 'functionally state sterilization' and coerces parents into 'voluntary' relinquishment, an advocacy characterization in the legislative record.

Sources: cohen2022, richardsonandsteckelcivilrig2026, marylandgeneralassembly2026, michigandepartmentofhealthan2025

Appears on: /domains/cases/us-birth-match

The Birth Match automation runs without ownership or evaluation: match volumes fell by more than half in Michigan (1,186 in FY2019 to 515 in FY2021) and Maryland (243 in CY2019 to 124 in CY2020) without the operating agencies noticing or being able to explain the change; fewer than 10 percent of matched families in any state with data received services (Maryland: 5 of 124 matched families in the source's mixed FFY2020/CY2020 pairing); Maryland's unanimous 2018 expansion statute ordered an independent evaluation of the match's sensitivity, specificity, and predictive value that a public-records request found was never implemented; and the only oversight that demonstrably changed the system — child fatality review teams auditing the matcher's misses — widened the trigger both documented times. Nearly all of these figures flow through one proponent researcher's ad hoc public-records requests (AEI 2022), which the author herself flags as limited and of dubious accuracy; no state publishes birth-match data in any regular report.

Sources: cohen2022, cohen2022a, marylandgeneralassembly2018, baltimorecitychildfatalityre2017

Appears on: /domains/cases/us-birth-match

DC's Child and Family Services Agency published a 16-page pre-deployment AI Values Alignment Report (one of only three district-wide under Mayor's Order 2024-028) for CORA, a staff-facing policy chatbot launched June 16, 2025 inside its STAAND case-management system, whose centerpiece guardrail is a read-side exclusion — 'the chatbot does not draw from confidential case files' and has no access to STAAND case data; by December 2025 the agency's own tip sheets documented a CORA phone app that ingests case documents, photos of handwritten notes, and voice dictation and saves AI-drafted contact notes into the STAAND case record after worker approval — the write direction the no-case-files rule never governed, superseding the May 2025 report's dated statement that generated text is 'guidance, not text to be used by the employee as part of case documentation.'

Sources: dcchildandfamilyservicesagen2025b, dcchildandfamilyservicesagen2025, dcofficeofthechieftechnology2026

Appears on: /domains/cases/dc-cfsa-cora-chatbot

CFSA's May 2025 AI Values Alignment Report concedes that per-output human validation of CORA is impossible — asked whether humans can review and approve AI outputs before they are enacted, it answers 'No,' with validation occurring before content enters the knowledge source (SME validation of every document, annual committee re-review of the corpus) and afterward via quality assurance (monthly internal accuracy reports and a real-time answer-flagging channel, none published); by December 2025 the tool's own landing card carried the in-product warning 'The Ask Feature is under construction and answers may not be fully accurate. Consult your supervisor as needed,' delegating per-query vigilance to workers and supervisors six months after launch.

Sources: dcchildandfamilyservicesagen2025b, dcchildandfamilyservicesagen2025a

Appears on: /domains/cases/dc-cfsa-cora-chatbot

CORA operates as a single-agency closed loop with no external examination of the running system: its knowledge authors, tool owners, operators, and oversight committees are all CFSA units; no OIG or auditor review, no published accuracy or usage data, and no DC Council finding exists as of mid-2026; the deployment sits in CFSA's first era without a court monitor in three decades (LaShawn A. oversight ended 2021, final closure 2022); the 25,000 queries per month figure is a budget ceiling, not observed usage; and every quantitative outcome figure (45 minutes saved per intake report, one to four hours per case, three-week feature delivery at a claimed 20x lower cost) is an unaudited vendor claim from a customer story that never names CORA, so CORA-specific claims rest on agency sources alone.

Sources: dcchildandfamilyservicesagen2025b, microsoft2025, dcchildandfamilyservicesagen2021

Appears on: /domains/cases/dc-cfsa-cora-chatbot

In Richmond, Virginia, Predict-Align-Prevent's open-source place-based model (built with Urban Spatial; report byline Ken Steif, Matthew D. Harris, and Sydney Goldstein, 2019) reported its highest risk tier capturing about 70% of held-out child-maltreatment events against about 35% for a kernel-density baseline, while covering about 10% of city land area holding roughly 48,500 residents including about 8,200 children — builder-generated figures from a commissioned report — and the builders' own published fairness audit stated verbatim that the meta-model 'generalizes well across neighborhoods of varying poverty rates, but does not generalize well across neighborhoods of varying race,' despite race and income being excluded from the feature set.

Sources: steif2019, urbanspatial2019

Appears on: /domains/cases/predict-align-prevent-geospatial

New Hampshire's 2018-2023 federal Community Collaborations cooperative agreement is the only documented operational planner use of Predict-Align-Prevent's maps: state narratives confirm the mapping (funded by Casey Family Programs, using address-level inputs accessed through stewarded databases) was completed for all three service areas before sites received implementation resources, and the March 2024 ACF/OPRE grantee profile documents that Community Implementation Teams used PAP-identified areas of need to target family outreach and issued funded requests for proposals with stipends — documented resource-routing behavior change, with no independent outcome evaluation of maltreatment effects in any deployment.

Sources: administrationforchildrenand2024, stateofnewhampshire2020

Appears on: /domains/cases/predict-align-prevent-geospatial

The 2016 Fort Worth study (Daley et al., Child Abuse & Neglect), trained on 2013 state child-welfare substantiations and Fort Worth police data, reported the top 10% of grid cells capturing 52% of 2014 maltreatment cases against 43% for a conventional hotspot model — figures independently corroborated verbatim by the Marchment and Gill (2021) Crime Science systematic review, which also found it to be the single child-maltreatment application of risk terrain modeling in the reviewed literature; Fort Worth remained a published study plus local advocacy, with no evidence of operational resource allocation by the maps.

Sources: daley2016, marchmentandgill2021

Appears on: /domains/cases/predict-align-prevent-geospatial

In December 2025 the Massachusetts Department of Transitional Assistance piloted an Accenture-built tool that transcribes SNAP eligibility calls in real time and generates a structured, caseworker-editable summary that is saved into BEACON, the state's benefits eligibility system of record; full transcripts are not retained, only the summaries, according to technical documentation reviewed by The Shoestring, and the summarization prompt was withheld as proprietary. About 400 calls had been processed by the April 2026 reporting, against roughly 45,000 calls connected to staff per month, in a record system that fed 31,390 SNAP application dispositions in December 2025 alone. No independent evaluation, inspector-general audit, or completed privacy impact assessment of the tool is on record.

Sources: theshoestring2026, massachusettsdepartmentoftra2025

Appears on: /domains/cases/massachusetts-dta-call-summaries

Oversight of the DTA call summarizer ran through the labor channel: SEIU Local 509, representing DTA call-center workers among roughly 9,000 state employees, reached an agreement with DTA — pre- or early-deployment; the record does not establish it preceded the December 2025 rollout — that made worker use voluntary and protected jobs, without addressing record provenance or client-side safeguards. The formal privacy apparatus sat empty: none of the nine AI use cases Massachusetts disclosed from its internal inventory of at least 40, the DTA summarizer included, reported a completed privacy impact assessment; the tool's interaction-data plan was listed as still to be defined while it was in production; and details on the other 31 use cases were withheld until the Supervisor of Records ordered them submitted for in camera review.

Sources: theshoestring2026, massachusettsexecutiveoffice2026

Appears on: /domains/cases/massachusetts-dta-call-summaries

The record store the AI summaries enter was itself contested while the pilot ran: in California v. USDA, No. 3:25-cv-06310 (N.D. Cal.), a 21-state-plus-DC coalition including Massachusetts obtained preliminary injunctions on October 15, 2025 and February 27, 2026 blocking USDA from cutting SNAP funding over states' refusal to hand over personal SNAP applicant and recipient data, the court holding the proposed data protocol would likely permit sharing beyond the entities allowed under 7 U.S.C. 2020(e)(8). Both orders are preliminary and the litigation is live; whether AI-generated call summaries held in BEACON fall within the demanded data is unresolved, and the Massachusetts AG's office declined to comment on that question.

Sources: massachusettsattorneygeneral2026, jurist2026, theshoestring2026

Appears on: /domains/cases/massachusetts-dta-call-summaries

Illinois DCFS rolled out a vendor natural-language-processing layer statewide over its legacy SACWIS case-management system beginning in 2023, provisioning roughly 7,500 users with a DCFS-estimated 20 percent administrative-time saving; the agency's federal Annual Progress and Services Report states that all notes in SACWIS can be mined, rendered as per-case Risks and Strengths overviews with click-through to source notes. By the vendor CEO's November 2025 congressional testimony, more than 6,000 Illinois staff use the tool and it saves each social worker about five hours per week — vendor figures delivered in a hearing whose majority summary records no AI-risk scrutiny — and no independent accuracy evaluation, published usage metric, or audit of the deployment has been located.

Sources: elisco2025, illinoisdepartmentofchildren2024, governmenttechnology2023, housewaysandmeanscommittee2025

Appears on: /domains/cases/illinois-dcfs-augintel

From the 2023 announcement onward the same platform carried a second, management-facing channel reading the same note store: early-warning signs, caseworker safety, practice-model fidelity, and statewide trends per the launch release; compliance-related data collected in the background from the notes workers already document, service-outcome mining tied to funding accountability, and vendor-analytics deep dives informing where programs should be established or eliminated, per the vendor CEO's 2025 testimony; and a named Chapin Hall partnership using machine learning over case notes to measure practitioners' use of Motivational Interviewing. The associated documentation-behavior effect on the workers whose notes are mined is a structural inference, not a documented finding.

Sources: augintel2023, elisco2025, chapinhallattheuniversityofc2025

Appears on: /domains/cases/illinois-dcfs-augintel

In December 2017 the same agency, Illinois DCFS, terminated its Eckerd Rapid Safety Feedback predictive-analytics pilot — a 366,000-dollar sole-source arrangement that scored more than 4,100 children at 90-percent-plus probability of death or serious injury while assigning low risk to two children who died — after a joint OEIG/DCFS-OIG report found the arrangement had been misclassified as a grant rather than a no-bid contract. That shutdown is the only oversight event at this agency documented to have changed system behavior, and it preceded the current deployment's deliberately score-free extraction design.

Sources: theimprint2017, governmenttechnologya

Appears on: /domains/cases/illinois-dcfs-augintel

California's tax agency ran the first state generative AI call-center assistant through a complete governed procurement arc: a competitive sandbox in which two vendors were each paid one dollar to test for six months in a secure environment, a 10-month pilot in a simulated environment, and production rollout to roughly 375 agents completed August 22, 2025 under a 12-month, 445,000 dollar contract. The pilot projected a minimum 1.5 percent per-call time saving, and by May 2026 the agency had declined to renew the contract, with the official who launched the project saying the working system 'didn't quite save as much time as we had hoped.'

Sources: stateofcaliforniagenaiportal2024, officeofthegovernorofcalifor2025, governmenttechnologyindustry2025, melhado2026

Appears on: /domains/cases/california-cdtfa-genai-call-center

The benefit figures for the CDTFA call-center assistant are pilot projections from a simulated environment, self-reported by the agency through trade press and echoed in vendor marketing: the call-center chief said the simulated environment 'did show some potential to save at least one and a half percent time on some of those calls,' extrapolated to roughly 100,000 minutes per year and capacity for about 10,000 additional calls annually across roughly 800,000 yearly inquiries. No published production-environment measurement, methodology, or independent assessment report has been located.

Sources: governmenttechnologyindustry2025, symsoftsolutionsviabusinessw2025

Appears on: /domains/cases/california-cdtfa-genai-call-center

By May 2026 CDTFA had declined to renew the 12-month, 445,000 dollar contract for its custom call-center assistant and shifted to a similar off-the-shelf tool from Amazon Web Services under an existing larger contract, with Government Operations Secretary Nick Maduros saying the solutions 'worked in practice' but 'didn't quite save as much time as we had hoped.' The decision was a procurement authority action inside a broader portfolio review of eight executive-order-driven generative AI pilots totaling more than 5.8 million dollars, in which three other projects also ended while three continued.

Sources: melhado2026

Appears on: /domains/cases/california-cdtfa-genai-call-center

New Jersey built and hosts its own generative drafting assistant for state employees, with a state-owned interface, hosting and logs and a hosted commercial frontier-model service as the one external dependency in the serving path, an ownership arrangement that leaves the underlying model replaceable without reprocuring an application or moving employees onto a different product. The state reports that roughly 20,000 employees had used the tool across more than 300,000 sessions and more than 1,000,000 prompts by February 2026 at about one dollar per user per month, that access onboarding routes through a responsible-AI course whose curriculum is used by 25 or more states, and that unemployment-insurance staff rewriting claimant emails in plain language saw claimants respond 35 percent faster. Every outcome figure is state self-reported; the 35 percent figure has no published methodology and predates the statewide launch, and no inspector-general evaluation, external audit or peer-reviewed causal study of the tool was located.

Sources: njofficeofinnovation2026a, routefifty2025, statescoop2024, njofficeofinnovation2024, sofi2026

Appears on: /domains/cases/nj-ai-assistant

New Jersey's joint policy circular 25-OIT-001 requires human review of all AI-generated content for accuracy, bias, completeness, accessibility and style; permits sensitive personal information only inside state-approved tools, naming the NJ AI Assistant, with Agency CIO approval; and requires State Chief Technology Officer clearance plus registration for resident-facing or decisional generative systems, a gate the staff-facing assistant has not been reported to pass. The same circular's text says all state employees 'should' take the responsible-AI course, a should-language obligation stated in the governing document itself, while the focal unemployment-insurance agency is reported to have reached full training and tool coverage of its staff. A statewide public-workforce survey preceded the November 2024 AI Task Force recommendations to the Governor, and user surveys and interviews drove the March 2026 rebuild, which added a visible reasoning section intended to help employees catch errors and hallucinations.

Sources: njofficeofinformationtechnol2025, stateofnewjerseyaitaskforce2024, njofficeofinnovation2026, routefifty2025

Appears on: /domains/cases/nj-ai-assistant

New Jersey's unemployment-insurance and TDI/FLI teams built plain-language glossaries, reusable prompt libraries and quality-evaluation rubrics for AI-assisted translation into Spanish and Haitian Creole, with human review by professional translators, subject-matter experts and seven community organizations; the state reports a higher share of Spanish-language unemployment-insurance applications and better follow-through, without quantifying either. US Digital Response gave the department a 2025 SEED Award and republished the materials for reuse by other states, the associated responsible-AI curriculum is used by 25 or more states and local partners, and the March 2026 rebuild moved training guidance inside the tool itself.

Sources: newjerseydepartmentoflaboran2025, usdigitalresponse2025, routefifty2025, njofficeofinnovation2026, innovateus2024

Appears on: /domains/cases/nj-ai-assistant

The Social Security Administration deployed a conversational question-and-answer chatbot on its national 800-number in April 2025, answering 74 frequently asked questions before a caller reaches an employee. Automation on that line went from roughly 300,000 handled calls a month in fiscal 2024 to roughly 2.9 million a month in fiscal 2025, peaking at 5.1 million automated calls in March 2025, while the agency served 68 million callers, a 65 percent increase over the prior year, with a workforce that fell about 10 percent net from roughly 57,000 to roughly 51,400 and about 1,000 field office employees reassigned onto 800-number duty. About 25 million fiscal 2025 calls ended with no service, abandoned in queue or met with a busy signal, and the busy rate spiked to 29.4 percent in March 2025 during the Social Security Fairness Act surge, which affected 3.2 million beneficiaries.

Sources: ssaofficeoftheinspectorgener2025, kffhealthnewsdariustahir2025, aarp2025

Appears on: /domains/cases/ssa-800-number-ai-assistant

The agency's inspector general found the published telephone metrics arithmetically accurate as computed and documented what they cover: the headline average speed of answer, 12.7 minutes in October 2024, a 29.7 minute peak in January 2025 and 7.0 minutes in September 2025, counts a caller who accepts a callback as a zero wait, and the roughly 25 million calls a year ending in a hang-up or a busy signal are excluded. The 23.8 million callers who accepted callbacks in fiscal 2025 waited an average of 61.5 to 151.8 minutes depending on the month, and the 9.3 million who declined and held waited an average of 18.8 to 100.1 minutes. A separate 87 percent first-contact-resolution figure comes from a post-call survey that is not offered to callers served only by automation, and no audit has published a misrouting or wrongful-disconnection rate for the chatbot itself.

Sources: ssaofficeoftheinspectorgener2025, nextgovfcw2025b

Appears on: /domains/cases/ssa-800-number-ai-assistant

Deployment decisions on this line have followed changes of leadership rather than published evaluation results. The chatbot was developed and tested under one administration and shelved as not ready, with the former chief information officer saying the team wanted to ensure the automation produced consistent and accurate answers and that this would take more time; the next administration deployed it in April 2025 and targeted extension to roughly 1,200 field offices by August 2025. A separate phone anti-fraud AI check was added in April 2025 and its three-day claim holds removed in mid-May 2025 after it flagged 2 of more than 110,000 claims while slowing retirement claim processing by 25 percent, a different tool whose figures do not merge with the chatbot's. The predecessor Next Generation Telephony Project, built by Verizon Business Network Services under a February 2020 contract, was abandoned on August 22, 2024 after about ten months and more than 160 million dollars paid, with an audit finding that contract lacked performance-based quality standards; the vendor of the current cloud platform is not named in the public audit. Five commissioners or acting commissioners served during fiscal 2025, and the metrics published on the agency's performance website were added and removed according to what each leadership believed were the most important metrics for the public.

Sources: kffhealthnewsdariustahir2025, nextgovfcw2025a, officeofsenatorelizabethwarr2025, ssaofficeoftheinspectorgener2025b, ssaofficeoftheinspectorgener2025

Appears on: /domains/cases/ssa-800-number-ai-assistant

California's Employment Development Department runs a two-tier chat assistant delivered under the Integrated Contact Center work stream of EDDNext, the state's roughly $1.258 billion modernization of its unemployment, disability and paid family leave systems. The unauthenticated public-site tier became available around the clock in the state's top eight working-age languages in May 2025 and served 554,792 unique customers across 2,103,782 messages between January 1 and June 30, 2025. A live agent chat channel for unemployment customers, which the department dates to July 2025, lets a customer escalate to a person on weekdays between 9 a.m. and 2 p.m. after identity verification, with account details passed to the agent, real-time machine translation in six non-English languages, a redacted transcript saved to the account and a post-chat survey. On May 8, 2026 a second, authenticated tier launched inside the customer portal, answering a signed-in unemployment customer's own claim status, payment and eligibility questions for claims filed in the past three years; the department reported more than 25,000 uses and nearly 18,000 fully self-service interactions in its first two weeks. The platform is documented as intent-based conversational AI; the public sources do not establish generative language modeling. All usage figures are agency self-reported, and the two tiers' counts belong to different systems.

Sources: californiaemploymentdevelopm2023, californiaemploymentdevelopm2025a, californiaemploymentdevelopm2025d, californiaemploymentdevelopm2025c, californiaemploymentdevelopm2026

Appears on: /domains/cases/california-edd-virtual-assistant

No body oversees the EDD chat assistant as such: its accountability is inherited two levels up, from legislative and executive scrutiny of the EDDNext programme it is a deliverable of. That scrutiny is substantial and has demonstrably changed programme behaviour — the 2026-27 Governor's Budget reverts $70.6 million of unused modernization funding early as a budget solution, and the core claims-system replacement was resequenced to do disability and paid family leave first with unemployment integration designated 'mandatory optional' to reduce risk for the state — and it is also documented as partial, since the adopted 2025-26 Budget Act retained extended spending authority against the Legislative Analyst's Office recommendation to drop it. What the oversight record engages with is budget, schedule and procurement: the 2026 analyst-office questions on the record concern the core project's vendor and the rationale for removing unemployment from it, and the February 2026 handout flags that new front-end functionality is linked to the legacy claims mainframe through 'informal and untested data bridges and custom-built interfaces' that have not been stress tested — a finding stated generically about new functionality, without naming the chat assistant. No inspector-general review, state-auditor evaluation or academic study of this assistant's answer accuracy or translation fidelity has been located in the public record.

Sources: californialegislativeanalyst2025, californialegislativeanalyst2025c, californialegislativeanalyst2026, californialegislativeanalyst2026a, californiastatesenate2026

Appears on: /domains/cases/california-edd-virtual-assistant

The success measures published for the EDD chat assistant are deflection-shaped, and the department both produces them and reports them upward. Since July 2025 the agency states that 23 percent more customers resolve questions through self-service and 47 percent fewer need to speak with an agent after using it, alongside more than 2.1 million self-service actions since November 2024, more than 830,000 customers using callback since May 2024, more than 29,900 customers served by live-chat agents, and an in-chat issue-resolution rate above 80 percent. Those figures measure channel exit rather than verified problem resolution, and they are agency self-reported; the human channel they are measured against runs weekdays from 9 a.m. to 2 p.m. and reached nearly 6,000 unemployment customers a month as of October 2025. Separately, the integrator published on its own marketing blog that the bot stack reduced call volume by 35,000 calls a day, cut live-agent volume by 3,800 calls a day, saved constituents 684 hours daily and generated $8.4 million in annual savings; those are vendor claims with no independent verification.

Sources: californiaemploymentdevelopm2025b, californiaemploymentdevelopm2025c, intervisionsystems2025

Appears on: /domains/cases/california-edd-virtual-assistant

Massachusetts built a retrieval-grounded generative-AI Virtual Assistant in house in 83 days and launched it on motor-vehicle registry pages in April 2025 ahead of the May 7, 2025 federal identification deadline, describing it as a state-owned platform that replaced a vendor-managed rule-based chatbot; by 2026 the assistant's own support page listed motor-vehicle, toll, unemployment-assistance, tax, child-support, transitional-assistance, family-and-medical-leave and state-login content. The state reports more than 1,500 conversations a day and roughly 200,000 conversations since launch, a chat open rate rising from 1.29 to 2.92 percent, positive feedback rising from 10 to 56 percent, negative feedback down 60 percent, page abandonment down 40 percent, around-the-clock availability in English, Spanish and Portuguese, a knowledge base refreshed nightly from published state content, and a custom evaluation tool that flags answer issues within hours. Every one of those figures is an agency self-report; the technology agency withheld the cost and usage reports and the technical logs that would confirm its claims, and the underlying foundation model and hosting stack have not been publicly identified.

Sources: massachusettsdigitalservice2025, executiveofficeoftechnologys2026, executiveofficeoftechnologys2025, theshoestring2026, massachusettsdigitalservice2026

Appears on: /domains/cases/massgov-virtual-assistant

Massachusetts policy AI.001, effective January 31, 2025, requires human fact-checking of generative output, conspicuous labeling of AI content, Chief Technology Officer approval for generative procurement and a generative-AI inventory, while routing privacy review through consultation with legal and security teams; the phrase privacy impact appears in it zero times. The Enterprise Privacy Office told the legislature in February 2025 that it has been using a Privacy Impact Assessment and was overseeing a pilot program with its risk and security teams to assess privacy risks during the contracting and development stages. Records obtained by an independent investigation showed at least forty AI use cases in the agency's internal survey with thirty-one withheld, and of the nine entries released not one recorded a completed privacy impact assessment, the fields left blank without explanation including for tools processing Social Security numbers and Medicaid data, with the agency spokesperson declining to answer questions about them; a count across all forty is therefore an inference the agency has not contradicted rather than a verified number, and the assessment is an internal office practice rather than a statutory mandate. After months of negotiation the Supervisor of Records ordered the agency to produce the withheld records for in camera review, allowing ten business days to produce and up to fifteen business days to review; no final determination was found as of July 20, 2026.

Sources: theshoestring2026, commonwealthofmassachusettse2025, executiveofficeoftechnologys2025, massachusettsexecutiveoffice2026

Appears on: /domains/cases/massgov-virtual-assistant

The Massachusetts assistant answers from a knowledge base rebuilt nightly out of published state content, and the state reports that those published pages were updated and simplified to suit the assistant while chat analytics drive prompt tuning, content updates and enhancements, so the authoritative public statement of program rules is adapted to the channel that reads it. On the human-channel side, an independent institute brief reporting state figures records motor-vehicle registry calls falling by about 1,000 a day and emails by about 200 a week after launch, counts about twenty AI use cases publicly reported with three facing the public against forty in the internal survey, and recommends a formal public AI inventory and structured user feedback loops; the assistant offers three languages where the predecessor rule-based chatbot offered twenty. The state technology secretary committed publicly that there will be a human reviewing output before public distribution and framed AI as relieving stretched agencies without adding headcount, while a public-employee union representing about 9,000 state employees argued that staffing rather than AI tooling is what would let the transitional-assistance agency serve more clients.

Sources: massachusettsdigitalservice2025, pioneerinstitute2026, commonwealthbeacon2026, theshoestring2026, executiveofficeoftechnologys2025

Appears on: /domains/cases/massgov-virtual-assistant

MyFriendBen is an advisory multi-state benefits screener: an anonymous survey of about six minutes returns a report of programs a household is likely eligible for, with estimated dollar values, and submits nothing, routing people instead to separate government application channels that make every determination independently. All of its eligibility math is computed by PolicyEngine, a separate nonprofit whose open-source codebase encodes federal and state statute; MyFriendBen deliberately built no proprietary rules logic, and the same substrate sits under each of its state deployments, so an encoding error would misestimate benefits in every state simultaneously and a correction would propagate to every state simultaneously. That substrate's scale is indicated by a $300,000 NSF POSE Phase I award announced on August 18, 2025. The shared-substrate structure is documented in both organizations' technical materials; no incident of a cross-state encoding error appears in the public record. The accuracy figure attached to it, 'over 90%', is a builder, infrastructure-organization and operator claim with no published methodology and no independent audit.

Sources: policyengine2025a, githubmyfriendbenorg2026, policyengine2025b, codethedreammyfriendbenncben2025

Appears on: /domains/cases/myfriendben

The gap between benefits identified and benefits received is measured only from outside the system. Independent journalism reported that in all of 2023 in Colorado about 5,500 households were screened, about $30 million in benefits was identified and about $5 million was estimated to have actually been obtained, with the median user reporting an income just over $8,200 a year and a household size of two. The deployment's own impact totals are self-published or funder claims and are mutually inconsistent, ranging across 20,000+ Coloradans, 55,000+ households, 100,000+ households, 115,000+ households nationally and 65,000+ families, with dollar figures of $33 million, $52 million, $58 million and $1.2 billion identified, on denominators that shift between identified, applied for, unlocked and delivered. The one commissioned evaluation, by the Urban Institute, is described by a funder's post as currently under way; its interim figures from 301 users cover discovery, intent and satisfaction rather than calculation accuracy or verified enrollment, and its primary report was not located in open search. The Aspen Institute's Financial Security Program finds that no systematic evaluation exists of whether benefits screeners increase benefits access.

Sources: thecoloradosun2024, deltafund2026, prnewswiremyfriendbencpalrel2026, aspeninstitutefinancialsecur2024

Appears on: /domains/cases/myfriendben

The deployments are operated by independent local anchors rather than by one organization: Code the Dream with NC 211 in North Carolina, Benefit Illinois's Illinois Benefit Hub, the MASSCAP community-action network in Massachusetts and Child Poverty Action Lab in Texas, each configuring its own program catalog on a shared white-label backend that carries per-state feature flags and admits a program only if it is worth at least $300 a year. The front-line users are employed by those partner agencies and others: 2-1-1 Colorado piloted the tool with its own call staff in autumn 2023, and 100 to 130 individuals at one Dallas employer use it weekly, with one partner reporting that it serves about 4,000 individuals a year through it. The builder's stated reason for navigator uptake is that training a case manager on a single program's rules otherwise takes months. Screening data has also flowed outward into policy: a Colorado Child Tax Credit Calculator derived from the same code identified $177 million, and the builder credits the surrounding ecosystem with unlocking an $810 million state child tax credit through HB24-1311, a rule the shared substrate then had to encode. The catalog, threshold, uptake and policy-loop figures are builder, operator and joint-release claims.

Sources: garycommunityventures2025, prnewswiremyfriendbencpalrel2026, githubmyfriendbenorg2026, masscapmassachusettsassociat2026

Appears on: /domains/cases/myfriendben

SafeRent Solutions (formerly CoreLogic Rental Property Solutions) sold landlords a Registry ScorePLUS tenant-screening model returning a single 200-800 'lease performance risk' score plus an accept/decline/conditional recommendation measured against a landlord-chosen cutoff (500 in the complaint's worked example, with an optional 450-499 conditional band), built from credit bureau reports and scores including non-tenancy debt, bankruptcy records, past-due accounts, payment performance, and eviction and landlord-tenant court records, weighted 'according to their statistical significance in predicting lease performance'; factor weights were undisclosed to landlords, applicants and the public, and the company's marketing told buyers a landlord cannot change the screening algorithm. The plaintiffs' pleaded theory concerns a missing input: as of 2021, HUD data cited in the complaint shows Black and Hispanic voucher holders in Massachusetts paying an average of $423/month toward rent and utilities while public housing authorities paid landlords an average of $1,159/month directly (at least 73.26% of the expected payment), on tenancies averaging over 21 years in the same unit, and that subsidy is not a model input. These are plaintiff-pleaded figures accepted by the court as allegations at the motion-to-dismiss stage only.

Sources: louisv2022, louisv2023, findlawcaselaw2023

Appears on: /domains/cases/saferent-score-voucher-screening

The authority over a SafeRent-screened tenancy decision was documented as inverted: the vendor authored the weights and returned the answer without any housing relationship to the applicant, property management picked the cutoff 'in consultation with SafeRent' without knowing how scores are computed, and the leasing desk that signed the lease received only the bottom-line score ('The Leasing Manager does not receive the detailed credit information at the time of running the applicant screening'; 'CoreLogic sends us a number, and if it is above the predetermined approved number, we move forward... We do not know why they were denied') and disclaimed override authority to a named plaintiff in writing ('we do not accept appeals and cannot override the outcome of the Tenant Screening'). On July 26, 2023 Judge Angel Kelley denied the motions to dismiss the FHA 3604(a),(b) and Massachusetts c.151B race and source-of-income claims on the reasoning that SafeRent 'effectively controls' approval decisions because it alone builds and conceals the algorithm, while dismissing the c.93A consumer counts. The single documented pre-settlement reversal ran outside the screening system entirely: a July 22, 2021 denial, a personal appeal rejected August 26, and a second appeal drafted by the tenant-advocacy organization City Life/Vida Urbana delivered September 1-8, 2021 — roughly six weeks of organized advocacy for one reversal, from which no reversal rate can be generalized.

Sources: louisv2022, louisv2023, unitedstatesdepartmentofjust2023a, civilrightslitigationclearin2025

Appears on: /domains/cases/saferent-score-voucher-screening

The Louis v. SafeRent settlement (executed March 28, 2024; finally approved November 20, 2024; $2.275M total, $1.175M fund with no reversion) remedied the case on the output channel rather than in the model: for five years SafeRent may return no SafeRent Score, no other tenant screening score and no accept/decline recommendation on a report for a voucher-holder application, providing a report of underlying information instead (3.5.2); for the 'market' and 'no-credit' products the score is suppressed by default unless the landlord affirmatively certifies the applicant is not a voucher recipient (3.5.3); any tenant screening score may re-enter the voucher channel only if 'found to be valid when used for voucher-holders by the National Fair Housing Alliance' or another organization mutually agreed by class counsel and SafeRent (3.5.5(ii)(1)); third-party credit scores (FICO, VantageScore) may still be passed through with source disclosure (3.5.5(iii)); customer training is required (3.5.4); and the court retains continuing exclusive enforcement jurisdiction for five years from SafeRent's certification (6.9), approximately through 2029-2030. The geographic reach of the practice-change terms is ambiguous: 3.5.2 and 3.5.3 are not expressly limited to Massachusetts, while the classes, the notice population and the plaintiffs' framing are Massachusetts-centered, and some coverage characterizes the changes as company-wide. Suppression therefore shifts rather than eliminates the decision input. SafeRent settled without admitting liability and states it 'continues to believe the SRS Scores comply with all applicable laws' (vendor claim); as of 2026-07-20 no public post-settlement compliance report, enforcement motion or validating-organization event had been located, so the gate exists on court-approved terms and has never been observed operating. The >18,000 figure is Massachusetts applications scored below the housing provider's accept threshold between May 25, 2020 and September 27, 2023 (the records-pull window), an upper-bound flow figure that does not identify voucher status or race, paired with SafeRent's own October 2023 estimate of 3,300-4,200 putative class members.

Sources: louisv2024, epiqclassactionservices2024, associatedpressviafortune2024, greaterbostonlegalservices2024

Appears on: /domains/cases/saferent-score-voucher-screening

The United States and ten plaintiff states allege in United States v. RealPage, Inc. (M.D.N.C., filed 23 August 2024, amended 7 January 2025) that competing landlords contractually fed a single vendor nonpublic, competitively sensitive information — executed new-lease rents, renewal offers and rates, lease terms and occupancy signals — to train and run a common pricing algorithm that recommended rents back to all of them: the complaint alleges at least 80 percent of the commercial revenue-management software market for multifamily housing and data agreements reaching over 16 million units nationwide including units of landlords who were not customers, describes an 'Auto Accept' setting that implemented daily recommendations with no human review, a 'Governor' feature alleged to constrain price decreases more than increases, and vendor pricing advisors who monitored client acceptance and pushed property managers toward compliance, and reports a national acceptance rate of 40 to 50 percent for new leases across January 2017 to June 2023 against internal analysis finding nearly 60 percent of final floor-plan prices within 2.5 percent of the recommendation and more than 85 percent within 5 percent — allegations only, with no defendant having admitted wrongdoing and no liability adjudicated.

Sources: usdepartmentofjusticeantitru2024, usdepartmentofjusticeofficeo2024, paul2025

Appears on: /domains/cases/realpage-rent-algorithm

The consent decree the Department of Justice proposed on 24 November 2025 edits the deployment's structure rather than its accuracy: competitor data used in models must be at least 12 months old and not drawn from active leases, the existing demand and supply models may not be trained with a geographic variable narrower than a state, the 'Governor' feature must treat increases and decreases symmetrically, automatic-acceptance ranges must be user-set and off by default, vendor-hosted meetings of competing landlords are barred, and a court-appointed monitor holds sweeping oversight for three years under a seven-year term with inspection rights, a written antitrust compliance program and a cooperation obligation — with no fine and no admission of liability; the Proposed Final Judgment and Competitive Impact Statement were published on 5 December 2025 (90 FR 56286), the stipulation was entered 26 March 2026, and the Department responded to eight public comments on 8 May 2026, with final public-interest entry by the court still pending as of July 2026 and no compliance findings published by the monitor.

Sources: usdepartmentofjusticeofficeo2025b, federalregister2026, hoganlovells2025, wilsonsonsini2025

Appears on: /domains/cases/realpage-rent-algorithm

The White House Council of Economic Advisers estimated in December 2024 that rental pricing algorithms cost United States renters more than $3.8 billion in 2023, roughly $70 per month per unit in algorithm-priced buildings, with the software pricing at least 10 percent of all US rental units and nearly one in four multifamily rental units — model-based counterfactual estimates the council explicitly framed as a lower bound because non-participating landlords also raised rents in response, and never measured overcharge; separately, the Middle District of Tennessee preliminarily approved 26 private settlements involving 27 landlord defendants totalling $141.8 million on 21 November 2025 for a class of renters of covered properties between 18 October 2018 and 21 November 2025, over objections from five state attorneys general, with a second batch of roughly $218 million announced in 2026.

Sources: whitehousecouncilofeconomica2024, americanbarassociationantitr2025, multifamilydive2026

Appears on: /domains/cases/realpage-rent-algorithm

New York City's Homebase homelessness-prevention program has since June 2012 routed applicant households on a 15-item Risk Assessment Questionnaire scored 0-25 (each answer worth 1-3 points), distilled by backward elimination from a Cox proportional-hazards model of shelter entry fitted to 11,105 families who applied October 2004 - June 2008 (12.8% entered shelter within three years; decile risk 1% to 37%); a total at or above 7 points routed a household to 'full' services (financial assistance, case management, legal and mediation referrals) rather than a 'brief' one-or-two-visit contact. It is a regression-derived additive point screener, not a machine-learning system: the city's statutory algorithmic-tool register files it under computation type 'Scoring', purpose 'Resource allocation', autonomy 'Monitored', frequency 'Daily', with 'Vendor(s): None' in the CY2025 entry. Against caseworker judgment, which had deemed 66.5% of applicants eligible, the instrument would have increased correct targeting of families entering shelter by 26% and cut misses by almost two-thirds at equivalent false-alarm rates. Services are delivered by seven contracted nonprofit providers across 26 neighborhood offices, and The Department of Homeless Services states the network serves more than 25,000 at-risk households a year. The causal effect evidence for the program comes from outside the deploying agency's own research office: a randomized controlled trial by Abt Associates, commissioned by the Department of Homeless Services (2010-2013, 295 families with children analyzed across eleven sites), found the share spending at least one night in shelter falling from 14.5% to 8.0%, the share applying for shelter falling from 18.2% to 9.3%, and average shelter nights falling by 22.6; an independent community-district difference-in-differences study estimated roughly 5-11% fewer family shelter entries (11.2 log points, 95% CI 3.6-18.8), a $14.2M annual budget avoiding an estimated $20-44M of shelter expenditure. The separately claimed 'prevention rate' of around 90-97% is a city performance metric with no counterfactual and is not an effect size. The threshold of 7 describes the documented pre-2023 configuration; no public source states the threshold in force after the 2023 item revision.

Sources: shinnm2013, nycofficeoftechnologyandinno2026, rolstonh2013, goodmans2016, instituteforchildrenpovertya2024, newyorkcitydepartmentofhomel2025

Appears on: /domains/cases/nyc-shelter-entry-prediction

The deploying agency's own Office of Research & Policy Innovation published a peer-reviewed re-examination (Housing Policy Debate, 2022) of 48,450 deduplicated families with children applying 2013-2016 (58,674 family-years, over a period in which the caseload rose from about 600 cases a month in 2013 to more than 1,500 a month in some 2015 months), and documented two distortions against itself. First, score clustering at the eligibility cutoff: 4,269 cases scored 6, 10,634 scored exactly 7, and 7,756 scored 8, which its researchers wrote 'suggests that Homebase staff may be focused on getting families to that threshold so they qualify for full services' — while the agency separately asserts that the override valve reduces incentives for workers to misreport data to ensure eligibility. Second, override outcomes: 6.1% of the 58,674 applications departed from the score with mandatory supervisor approval (4.8% up to full services, 1.4% down to brief), and only 3.7% of the families moved up later applied to shelter against 25.8% of the families moved down, the paper concluding that worker judgment is less accurate than the RAQ on average; a ten-case note review attributed many downward decisions to needs beyond the program's capacity, such as families needing an apartment immediately with no funds, rather than to a judgment about risk. The same paper reported 73.9% of applications at or above the threshold, shelter application within two years at 13.7% above the cutoff against 5.9% below it (chi-square 699.98, p<.001), an area under the curve of 0.7387 for the deployed instrument against 0.7389 for the revised one, and a simulated alternative threshold of 5 raising precision from 13.7% to 15.2% at similar enrollment volume. These are agency-authored figures, published in a peer-reviewed venue; the paper's own caveats are that the observed score distribution is confounded by the clustering it documents and by low-scoring self-selection out of intake, and that outcomes compared by service tier are contaminated by the overrides being measured.

Sources: mullenej2022, newyorkcityofficeoftechnolog2024, farrelldc2023

Appears on: /domains/cases/nyc-shelter-entry-prediction

Local Law 35 of 2022 (passed by the NYC Council 2021-12-15, lapsed into law unsigned 2022-01-14) added Admin. Code sec. 3-119.5, requiring every city agency to report by 31 December every algorithmic tool it used one or more times during the prior calendar year — expressly including tools that 'generate risk scores' or 'determine what resources are allocated to particular groups or individuals' — with six mandatory disclosure elements, compiled by the Office of Technology and Innovation into a public report delivered to the mayor and Council speaker each 31 March. DSS filed the Homebase RAQ in all four cycles to date (CY2022-CY2025), each entry disclosing in the agency's own words the June 2012 start date, the 2004-2008 training data analyzed with academic researchers, the input factors, the points-and-threshold eligibility mechanism and the worker-override-with-supervisor-permission rule. The loop demonstrably fired: after initial publication of the CY2023 report the register carries the note 'Update 3/27/2024 - The Department of Social Services updated their reporting to include changes made to the Homebase Risk Assessment Questionnaire after initial publication of the report', disclosing the 2023 item revision and adding its peer-reviewed citation; the Council's Committee on Technology held an oversight hearing on the regime on 2024-10-28, at which OTI described coordinating 45 agencies plus 24 further offices. Register entries are unaudited agency self-reports. Separately, three government audits have examined this program and none examined the risk model: the city comptroller's MG12-125A (2013-06-27) found no written monitoring policies, no records of initial ineligibility determinations and all provider risk assessments announced in advance, with the unannounced-visit recommendations rejected; the city comptroller's January 2020 audit of HRA's oversight of the then-$53M-a-year program found 80 of 240 required provider case-file reviews performed, 2,661 of 24,938 FY2018 households (11%) returning one to four times within twelve months, $2,271,797 in provider advances unrecouped some sixteen months after closeout, and 5 of 28 visited client homes not habitable (4 never fixed), issuing 19 recommendations; and the state comptroller's audit 2023-N-8 (issued 2026-01-07, scope July 2021 - July 2025) examined the downstream CityFHEPS rental-subsidy channel for DSS Homebase clients (57,888 new cases and 123,762 individuals housed since 2018 through March 2025; spending $176M in FY2019 rising to $834M in FY2024) and found units with hazardous violations approved, 30 of 75 sampled case records without evidence of income verification, and rents averaging $525 a month above comparables in eleven of thirty sampled statewide cases. None of the three may be cited as an audit of the algorithm.

Sources: newyorkcitycouncil2022, newyorkcityofficeoftechnolog2024, newyorkcitycouncilcommitteeo2024, officeofthenewyorkcitycomptr2013, officeofthenewyorkcitycomptr2020, officeofthenewyorkstatecompt2026

Appears on: /domains/cases/nyc-shelter-entry-prediction

CrimSAFE, a criminal-record tenant-screening product sold by CoreLogic Rental Property Solutions, computes no risk score: it is a deterministic record-matching and filtering engine that matches applicant identity data against a database of court and arrest records aggregated from more than 800 US jurisdictions, classified into three primary categories (Crimes Against Property, Crimes Against Persons, Crimes Against Society) with sub-classifications, and applies filter criteria the HOUSING PROVIDER configures: offense type, disposition and severity across felony and non-felony convictions and charges, and a lookback period configurable from 0 to 99 years for convictions and 0 to 7 years for charges (federal consumer-reporting law permits reporting non-conviction records for seven years). The output is a report carrying a lease decision driven by the provider's criteria plus a credit score, a Record(s) Found flag, message text the provider authors, full record detail for the users the provider authorizes, and an optional provider-customizable adverse-action letter template. Every new CrimSAFE user is by default authorized to receive full record data, with no cap on how many users get full access; a provider must affirmatively change configuration settings to restrict full reports to senior managers. In April 2016 Carmen Arroyo's application to move within ArtSpace in Windham, Connecticut, so her son Mikhail could live with her after a 2015 injury, was denied on 26 April 2016 after the screen returned Record(s) Found; the only matched record was a pending Pennsylvania shoplifting charge, later withdrawn in April 2017. Mikhail's report carried the vendor's default message: 'Please verify the applicability of these records to your applicant and proceed with your community's screening policies.'

Sources: connecticutfairhousingcenter2026, connecticutfairhousingcenter2023, nationalhousinglawproject2018

Appears on: /domains/cases/corelogic-crimsafe

The location of decision authority in this deployment was the contested question, and the Second Circuit resolved it component by component on 20 February 2026 (Cabranes, Wesley, Menashi; opinion by Menashi; Nos. 23-1118(L), 23-1166(XAP)). WinnResidential had suppressed full reports from its own on-site staff so that leasing decisions involving criminal records would be made 'by someone in a more elevated position,' out of concern about leasing-commission incentives, so the on-site agent saw only a Record(s) Found flag and told Arroyo the application was denied without individualized review; answering the state commission's complaint, WinnResidential said it did not know 'the facts behind the criminal background findings' because it had 'trust' in CoreLogic's reports. After a ten-day bench trial Judge Vanessa L. Bryant ruled on 20 July 2023 that CrimSAFE does not disqualify applicants because the housing provider decides what records matter and whether to deny; the earlier August 2020 summary-judgment characterizations that the companies 'acted hand-in-glove' and that CoreLogic 'was an integral participant' belong to that posture and did not survive trial. The Second Circuit rejected the threshold reasoning that screening companies sit categorically outside the Fair Housing Act — a point the United States had urged as amicus on 24 November 2023 — but affirmed on proximate cause, holding the denial came after a chain of the provider's discretionary decisions (configuration, record relevance, staff access, adverse-action letters, final approval) and quoting the district court's own sentence with approval: 'No housing provider who uses CrimSAFE could reasonably believe that CoreLogic makes housing decisions for them.' It rejected the 'cat's paw' theory because the screening policies applied were the provider's own, rejected liability for failing to restrict lawfully reportable non-conviction records as extending liability beyond the first step, and dismissed the Connecticut Fair Housing Center's own claim for lack of Article III standing under a 2024 organizational-injury doctrine unrelated to tenant screening, vacating rather than deciding its merits. Disparate impact was never proven: the race and national-origin claim failed at the prima facie causation step, so the underlying statistics were never adjudicated.

Sources: connecticutfairhousingcenter2026, connecticutfairhousingcenter2023, connecticutfairhousingcenter2020, unitedstatesdepartmentofjust2023b

Appears on: /domains/cases/corelogic-crimsafe

The correction loop in this deployment ran through records the subject could not see at the point of harm. CoreLogic's Authentication Procedure Guide listed only a notarized power of attorney as third-party authorization and escalated 'any scenarios not covered' to a supervisor; staff demanded a power of attorney that Mikhail Arroyo, a conservatee, was legally incapable of executing — a demand the trial court called an 'impossible condition' — across a blocked window running from the 24 June 2016 request with a conservatorship certificate to mid-November 2016, when a 1 November call escalated to CoreLogic's legal department and the company agreed about two weeks later that a conservatorship certificate with a visible probate seal would suffice; the resubmitted copy again lacked a visible seal and the disclosure was never completed. That escalation is the only documented change in vendor behavior in the record. The family learned which record had caused the denial in December 2016, roughly eight months after the denial, from the housing provider rather than the vendor, and the adverse-action letter that should have triggered the correction loop was sent but never received; correction ultimately happened at the original source when Arroyo petitioned the Pennsylvania court and the charge was withdrawn in April 2017. Arroyo's was the first and only conservator file-disclosure request CoreLogic had ever received (478 F. Supp. 3d at 282), a finding both courts used to defeat the disability disparate-impact claim. The Connecticut Commission on Human Rights and Opportunities held an evidentiary hearing on 13 June 2017 and WinnResidential approved the move-in ten days later — about fourteen months after the denial, and the only oversight action documented to have changed an outcome. The sole liability finding, a willful consumer-reporting violation carrying $1,000 statutory and $3,000 punitive damages awarded in July 2023, was reversed on 20 February 2026; final vendor liability was zero, and no injunction, consent decree or policy change was ever ordered.

Sources: connecticutfairhousingcenter2026, connecticutfairhousingcenter2020, connecticutfairhousingcenter2023, courtlistenerandthefreelawpr2023

Appears on: /domains/cases/corelogic-crimsafe

New York City launched NYC Teenspace on November 15, 2023 as a three-year, $26M contract buying population access to Talkspace's existing consumer teletherapy platform for all city residents aged 13-17, with unlimited asynchronous messaging (therapist replies five days per week) plus one 30-minute live session per month from NY-licensed clinicians; inside the therapy chat a proprietary NLP model scans teen-authored messages for suicide and self-harm language and fires an urgent real-time alert to the teen's own treating therapist, which never acts autonomously — escalation (child-protective-services referral, intensive therapy, hospitalization) is entirely the clinician's; in the first six months the scan flagged roughly 50 teens as moderate-to-high suicide risk (under 1% of about 6,800 users) while therapists navigated 36 high-risk events including suicide attempts, a child-abuse report and a drug overdose; enrollment ran from about 6,800 (May 2024) to 19,000+ (DOHMH letter, Dec 1 2024) to a vendor-reported 45,000+ (Feb 2026), with about 80% of early registrants Black, Hispanic, AAPI, bi-racial or Native American, about 70% female, and more than half resident in the city's priority TRIE neighborhoods.

Sources: nycofficeofthemayor2023, chalkbeatnewyork2024, newyorkcitydepartmentofhealt2024, newyorkcitydepartmentofhealt2026

Appears on: /domains/cases/nyc-teenspace-talkspace

A September 2024 advocacy tracker audit of the NYC Teenspace pages counted 15 ad trackers and 34 cookies sharing teen visitors' personally identifiable information with recipients including Facebook/Meta, Amazon, Google and Microsoft, alongside sensitive intake data (name, date of birth, address, school, gender, mental-health screening answers) collected before parental consent was secured; DOHMH's December 18, 2024 letter (General Counsel Landau, Chief Privacy Officer Elcock) recorded that Talkspace removed all social-media and advertising trackers as of December 11, 2024, that the sign-up flow was minimized to age and zip code only, and that the contract's Data Security Rider would be amended to ban marketing use and trackers outright; advocates then documented continuing tracker-based disclosure on pages beyond the landing page (including the page hosting the revised privacy policy) on January 9 and February 12, 2025 with no DOHMH response to their follow-up letter after more than a month, and February 2025 technical testing found the NYC landing page clean as of January 24 while Talkspace's Seattle and Baltimore teen pages still transmitted visitor IP addresses to TikTok, Meta, Snapchat, Google, X, Reddit, LinkedIn, Spotify and Quora until the reporter's inquiry — several recipients being companies NYC had sued in February 2024 over teen mental-health harm. Talkspace's Chief Privacy Officer stated no personal medical information was transmitted; advocates dispute the completeness of the remediation, so the leakage is identifier-level and contested-scope.

Sources: parentcoalitionforstudentpri2024, newyorkcitydepartmentofhealt2024, parentcoalitionforstudentpri2025, gizmodo2025

Appears on: /domains/cases/nyc-teenspace-talkspace

The only published performance figure for the Teenspace suicide-alert algorithm is Talkspace's own claim of 83% accuracy versus a human expert, resting on a 2020 Psychotherapy Research study of ADULT platform data rather than any measurement of the teen population it reads, and the independent evaluation DOHMH said in a September 12, 2024 email that it was planning had still not been published as of mid-2026, with no OIG, comptroller or FTC action specific to Teenspace located; outcome figures (65% 'reported improvement' at six months; 66% 'measurable clinical improvement' among 45,000+ enrollees) are vendor or city self-reports with undisclosed instruments and no independent verification; meanwhile the record store the program writes into is described by Talkspace's CEO as 8 billion words, 140 million messages and 6.2 million assessments, is subpoenable (a Talkspace user's complete therapy history was obtained by her former employer and used in court in 2026 — an adult employer-benefit member, not a Teenspace teen), is stated by the vendor to be used for training behavioral-health LLMs, and transfers intact under the pending $835M Universal Health Services acquisition announced March 9, 2026 and expected to close in Q3 2026, while the NYC contract is still live.

Sources: talkspace2023, kdive2024, proofnews2026, prnewswireanduniversalhealth2026

Appears on: /domains/cases/nyc-teenspace-talkspace

Character.AI built its crisis-safety stack in four dated increments, each within days to weeks of a specific external event: a self-harm phrase screen referring users to the 988 Lifeline (vendor blog dated 22 October 2024, the day the Garcia wrongful-death complaint was filed in M.D. Fla., publicized the 23rd), a separate more restrictive under-18 model (December 2024, after two Texas family suits and a 15-company Texas Attorney General SCOPE Act investigation), a weekly parental usage summary carrying time spent and top characters while deliberately excluding chat content (25 March 2025), and — one week after an investigation found dozens of harmful personas live and seven weeks after FTC 6(b) orders reached seven companies — the removal of open-ended chat for under-18 users, announced 29 October 2025 and effective 25 November, with an interim two-hour daily cap ramping down and a two-stage age-assurance stack; on 7 January 2026 the company, both founders, and Google agreed to settle the Garcia case and four others in New York, Colorado, and Texas, terms undisclosed and approval still required, so causation was never adjudicated. Each retrofit is a company authority action temporally associated with an external event; the operator's own announcement cited several pressures at once, and no sole cause is asserted.

Sources: characterai2024, characterai2025a, cnnbusiness2026

Appears on: /domains/cases/character-ai-crisis-response

The only direct measurement of Character.AI's self-harm detector is a single adversarial test published 29 October 2024: across sixteen conversations with mental-distress-focused bots the 988-hotline pop-up appeared three times, triggered by two exact phrasings ('I am going to commit suicide' and 'I will kill myself right now') while many equally explicit statements including 'I want to end my life' did not trigger it, dismissable and non-blocking so the chat continued, with firing frequency increasing after the publication contacted the company — a sample of sixteen, and therefore anecdotal rather than a measured rate. No independent evaluation of the sensitivity or specificity of the crisis classifier, the two-stage age-assurance classifier, or the separate under-18 model exists in either direction; every efficacy claim about the retrofits is a vendor claim. The first compelled quantitative record of this detector class is prospective: California SB 243, signed 13 October 2025 and operative 1 January 2026, requires a crisis protocol issuing referral notifications and, from 1 July 2027, an annual report to the state Office of Suicide Prevention of the number of notifications issued, posted publicly — a count of referrals, not a measure of what they achieved.

Sources: futurism2024, californialegislativeinforma2025

Appears on: /domains/cases/character-ai-crisis-response

Character.AI's crisis pathway terminates in an automated referral to an external hotline with no staff review, no escalation call, and no capture of what followed; the human roles the record does document sit elsewhere — trust-and-safety moderators taking down user-authored characters reactively, parents receiving a weekly usage summary with no transcript access and only where the teen adds them, and, after November 2025, third-party identity reviewers adjudicating contested age determinations. The external review tier is by contrast unusually crowded and repeatedly behavior-forcing: two federal district courts, the Texas Attorney General twice, FTC 6(b) compulsory orders to seven companies on 11 September 2025, California SB 243, and a Senate subcommittee hearing, alongside investigative journalism that on 22 October 2025 — a year into the litigation — found dozens of harmful bots live including a 'Bestie Epstein' persona that had logged almost 3,000 chats, a gang simulator, school-shooter personas, and a 'doctor' giving antidepressant-tapering instructions. None of these reviewers ever operated inside the decision loop; what they moved was the structure itself.

Sources: thebureauofinvestigativejour2025, ftc2025b, characterai2025

Appears on: /domains/cases/character-ai-crisis-response

Four days after a labor board certified its paid helpline staff's union election, a national eating-disorder nonprofit told those staff they were terminated and that a chatbot would replace a helpline that had fielded nearly 70,000 contacts in 2022 with six paid staff, about two supervisors, and up to roughly 200 trained volunteers; the chatbot was suspended on May 30, 2023, two days before it was to become the sole channel, after testers published screenshots of it recommending a 500 to 1,000 calorie daily deficit, a 1 to 2 pound weekly loss, a 2,000 calorie cap, regular weigh-ins, and where to buy skinfold calipers, and the helpline closed on June 1 as scheduled while the chatbot was already offline.

Sources: wells2023, kffhealthnews2023, nprshots2023, picchi2023, npr2023a

Appears on: /domains/cases/neda-tessa-chatbot

The deployed chatbot had two layers with different evidence status: a closed, pre-scripted program the vendor and the research team describe as unable to depart from its authored content, and a generative question-and-answer feature the operating vendor added in what its chief executive called a systems upgrade covered by the client's contract, a reading the client's chief executive denies by saying the organization was never advised of the changes and would not have approved them; the vendor's public account of the harmful outputs was mixed, saying it was still trying to determine how a closed system allowed such content, so the attribution of the May 2023 advice to the generative layer is the reported and attributed explanation rather than an established mechanism, and the contract scope stays contested with no adjudicated breach.

Sources: nprshots2023, kffhealthnews2023, thewraprepublishedonyahoo2023

Appears on: /domains/cases/neda-tessa-chatbot

The earliest documented external warning about harmful chatbot responses came in October 2022 from the executive director of a peer eating-disorder organization, and the specific language she flagged was quickly removed after she reported it, with no documented systemic review, output monitoring, or reassessment of the deployment following; the vendor's chief executive said the flagged language was part of the pre-scripted content rather than the generative layer, which the research team denies, so its provenance is contested, and system-level action arrived roughly seven months later when public screenshots circulated.

Sources: nprshots2023

Appears on: /domains/cases/neda-tessa-chatbot

The Veterans Health Administration built an opioid overdose and suicide risk model in-house, with no commercial vendor, on its own electronic health record and Corporate Data Warehouse: fitted on 1,135,601 patients with an opioid prescription in fiscal 2010 against 23,790 overdose-related or suicide-related events among them in fiscal 2011 (a 2.1% base rate), reporting an area under the curve above 0.80 in training and test sets, and refreshed nightly as a continuous one-year risk estimate binned into percentile tiers on a population-management dashboard that shows each patient's risk factors and the guideline-recommended mitigation actions rather than a bare number. Its predictors are administrative - demographics, pharmacy records including opioid type and dose and co-prescribed sedatives, mental health and substance use disorder diagnoses, prior overdose-related and suicide-related events, detoxification episodes, and emergency department and other utilization history - so no structured risk questionnaire and no clinician-scored instrument feeds the score. VHA Notice 2018-08, issued 8 March 2018 and effective 18 April 2018, required all 140 VHA medical centers to convene interdisciplinary teams and case-review every patient in the very-high-risk tier, initially the top 1% of scores at a threshold of 0.166. The mandate compels an assessment of risk, prescription appropriateness and mitigation options and never an action: there is no mandated taper and no automatic prescription cutoff tied to the score, and every clinical decision remains with the treatment team. Across 44,042 patients in the top 1% to 5% band from April 2018 to March 2020 the mandate raised the odds of receiving a case review 5.1-fold (95% CI 3.64-7.23) and added 0.498 risk-mitigation strategies per patient (95% CI 0.39-0.61). Facility completion had a median of 71% (interquartile range 48-95%), about one facility in five met the 97% target, and each of the 89 surveyed facilities used a median of 23 distinct implementation strategies. Team composition and workflow were left to each facility.

Sources: oliva2017, strombotne2023, minegishi2022, rogal2020

Appears on: /domains/cases/va-storm-opioid-risk

The Veterans Health Administration randomized two features of its own opioid case-review policy across its 140 medical centers, and the two randomizations are separate experiments whose findings do not combine. The first was timing: when a facility's mandated review tier widened from the top 1% of risk scores to the top 5%, executed as a stepped wedge with waves on 12 February 2019 and 13 August 2019, which the primary trial paper places at study months 11 and 17. The second was language: whether a facility's copy of the policy notice carried an accountability paragraph naming a 97% case-review completion target, with quarterly reporting to the national Office of Mental Health and Suicide Prevention and technical assistance and action plans for facilities below it. Seventy facilities received that paragraph and seventy did not. Across 16,272 very-high-risk patients (8,734 in the accountability arm, 7,538 outside it), about 57% received a case review overall against a pre-mandate baseline of 6.6%, and there was no difference between arms in opioid-related serious adverse events (hazard ratio 1.03, 95% CI 0.97-1.08) or mortality (hazard ratio 1.00, 95% CI 0.91-1.09) - but patients at accountability-arm facilities were less likely to receive a case review at all (hazard ratio 0.91, 95% CI 0.87-0.95). The implementation tracking study reports the same result at facility level: median completion 71% (interquartile range 48-95%), 18 of 89 surveyed facilities (20%) meeting the 97% target, and facilities given the plain mandate meeting it more often than facilities given the accountability language, 30% against 11% (p=0.04). The practice associated with higher completion was regular self-monitoring and adaptation inside the facility (adjusted incidence rate ratio 1.40) rather than reporting upward from it; dashboard use was reported by 97% of surveyed facilities and local opinion leaders by 80%, while patient-engagement strategies were used by 13%. The oversight-backfire finding belongs to the language experiment alone.

Sources: minegishi2022, strombotne2023, rogal2020

Appears on: /domains/cases/va-storm-opioid-risk

The benefit and the harm signals from the Veterans Health Administration's mandated opioid case-review policy travel together and neither may be reported alone. The widely quoted mortality result - four-month all-cause mortality odds of 0.78 (95% CI 0.65-0.94) - is an EXPLORATORY endpoint of a trial whose pre-specified primary composite of nine serious-adverse-event categories did not move (odds ratio 0.995, 95% CI 0.875-1.132). Among patients NEWLY DIAGNOSED with opioid use disorder during the trial (28,251 analyzed, estimated off the stepped-wedge threshold-expansion randomization rather than the accountability-language arm) the mandate was associated with 90-day all-cause mortality odds of 1.74 (95% CI 1.06-2.87) with no significant change in serious adverse events; a post-hoc subgroup with an opioid prescription before but not after diagnosis showed 5.87 (95% CI 1.85-18.58). Nearly every quantitative source on this deployment is a department-affiliated research-operations partnership: this is peer-reviewed agency self-evaluation that published its own null, backfire and harm findings, and it is not third-party replication.

Sources: strombotne2023, auty2023

Appears on: /domains/cases/va-storm-opioid-risk

Internal records subpoenaed by the U.S. Senate Permanent Subcommittee on Investigations show that early-2021 testing of an auto-authorization model inside UnitedHealthcare produced faster handle times together with an increase in adverse determination rate - attributed to finding contraindicated evidence missed in original review - and the internal committee voted to tentatively approve the model at the following meeting; the April 2021 approval of 'Machine Assisted Prior Authorization' was paired with testing that removed six to ten minutes from the average review while the reviewing doctor or nurse still had to verify that the primary evidence is acceptable. Over the same period the insurer's post-acute prior authorization denial rate went from 10.9 percent (2020) to 16.3 percent (2021) to 22.7 percent (2022), and its 2019 skilled-nursing-facility denial rate was nine times lower than its 2022 rate. A January 2022 vendor presentation shows a naviHealth care coordinator completing nH Predict to determine optimal post-acute placement while the patient is still hospitalized, and an April 2022 vendor instruction told call handlers not to guide providers on the questions used to collect the information determinations are made from.

Sources: ussenatepermanentsubcommitte2024, caseyrossandbobherman2023

Appears on: /domains/cases/nh-predict-utilization-review

Across Medicare Advantage in 2022, of 46.2 million prior authorization determinations, 3.4 million (7.4 percent) were denied in whole or in part; 9.9 percent of denials were appealed; and 83.2 percent of appeals resulted in the initial decision being overturned (KFF analysis of the plans' own federal reporting). The figures are program-wide, not plan- or service-line-specific, and the overturn rate is conditioned on the self-selected minority of denials that were appealed, so it overstates what the same review would correct if applied to every denial.

Sources: kff2022

Appears on: /domains/cases/nh-predict-utilization-review

Whether nH Predict was used to make coverage determinations is contested and unadjudicated. The vendor's public statement in STAT's March 2023 series opener was that the tool 'is not used to make coverage determinations' and 'is used as a guide'; the Lokken order (D. Minn., Feb 13, 2025) recites - as pleaded allegations taken as true on a motion to dismiss - an allegation about the share of claim denials reversed on appeal, and records in the same paragraph that 'UHC denies any use of nH Predict'; the Barrows order (W.D. Ky., Aug 14, 2025) recites Humana's use of the tool as pleadings only. Both class actions survived motions to dismiss in narrowed form - contract and implied-covenant counts in both, plus unjust enrichment and common-law fraud in the Humana action and are in active discovery as of August 2026, with no dispositive merits ruling. CMS's contract-year-2024 rule and its February 6, 2024 FAQ constrain the order of operations: an algorithm may assist a coverage determination, the plan remains responsible, an algorithm determining coverage from a larger data set instead of the individual patient's medical history, physician recommendations or clinical notes would not comply with 42 CFR 422.101(c), and a predicted length of stay alone cannot be the basis to terminate post-acute care services - only re-assessing the individual patient's condition can. The naviHealth brand was retired in Q1 2024 into Optum 'Home & Community Care'; the pipeline continues under that name.

Sources: estateofgeneb2025, centersformedicaremedicaidse2024, cfr2024, cbsnews, caseyrossandbobherman2023a, barrows2025, bobhermanandcaseyross2023, georgetownlaw2026a, georgetownlaw2026

Appears on: /domains/cases/nh-predict-utilization-review

The closest ground-truth audit of this decision class is HHS OIG evaluation OEI-09-18-00260 (April 2022): reviewing a stratified random sample of 250 prior authorization denials and 250 payment denials issued by 15 of the largest Medicare Advantage organizations during one week of June 2019, health-care coding experts and physician reviewers found 13 percent of the prior authorization denials and 18 percent of the payment denials met Medicare coverage rules, with identified causes including internal clinical criteria applied beyond Medicare rules, insufficient-documentation findings the reviewers judged unfounded, manual processing errors and system programming failures, and with stays in post-acute facilities among the report's own examples. The evaluation predates naviHealth's management of this benefit and pools fifteen organizations, so it orders - and does not measure - an error term for this deployment.

Sources: u2022

Appears on: /domains/cases/nh-predict-utilization-review

A commercial population-health risk score used to target a scarce high-risk care management program at a large academic hospital (studied 2013-2015) was trained to predict total medical expenditure in the following year while the deployment's stated purpose was to identify the patients with the greatest health needs, with race excluded from its features. The independent evaluation found the model well calibrated for cost across race — roughly equal realized following-year costs at every level of predicted risk, 5,147 versus 4,995 dollars at the median score — while at the 97th-percentile auto-identification threshold Black patients carried 26.3 percent more active chronic conditions than White patients at the same score (4.8 versus 3.8, P < 0.001). Patients above the 97th percentile were automatically identified for enrollment and those above the 55th were referred to their primary care physician; enrollment covered 1.3 percent of observations, and commercial tools of this class are applied to roughly 200 million people in the US each year per industry estimates cited in the study.

Sources: obermeyer2019a, obermeyer2019

Appears on: /domains/cases/cost-proxy-care-stratification

The deployment's designed human check — physician referral above the 55th percentile — was measured: realized program enrollment was 19.2 percent Black against 11.9 percent Black in the whole sample, while simulated race-blind sampling within bins of the same score would have produced 18.3 percent (P = 0.8348 against the observed figure) and sampling on predicted chronic-condition count within bins would have produced 26.9 percent. The study's abstract states that remedying the disparity would increase the percentage of Black patients receiving additional help from 17.7 to 46.5 percent — a figure describing the composition of the helped cohort under a counterfactual ranking, not a rate of selection for any subpopulation.

Sources: obermeyer2019a, obermeyer2019

Appears on: /domains/cases/cost-proxy-care-stratification

Per the Science study, the manufacturer independently replicated the analysis on its national dataset of 3,695,943 commercially insured patients, confirmed the finding, and jointly rebuilt the predictor with the study's authors using the same sample, the same features with race still excluded, and the same training process, changing only the training label to an index combining health prediction with cost prediction. One measure of predictive bias — excess active chronic conditions in Black patients conditional on risk score — fell from 48,772 to 7,758, an 84 percent reduction in that bias measure (counts of excess conditions, not error rates). The relabelled predictor was experimental and holdout at publication, with no production role in enrollment decisions, and no independent evaluation of it by anyone other than its authors has been located. The study's authors later generalized the remediation practice in the Algorithmic Bias Playbook (Chicago Booth Center for Applied AI, 2021), which centers label-choice bias and, like the study, does not name the product.

Sources: obermeyer2019a, obermeyer2021

Appears on: /domains/cases/cost-proxy-care-stratification

The Science study anonymized the product and manufacturer. On its publication day, October 25, 2019, the Superintendent of the New York State Department of Financial Services and the Commissioner of the New York State Department of Health jointly wrote to UnitedHealth Group's chief executive, naming 'Optum's data analytics program, Impact Pro', footnoting the study, and demanding that the company immediately investigate and demonstrate the algorithm is not racially discriminatory or cease using Impact Pro or any similar program, asserting insurers' responsibility under New York law regardless of who built the algorithm; contemporaneous press reports carried the same identification and the company's statement that the cost model 'was highly predictive of cost, which is what it was designed to do.' No publicly documented resolution of the New York inquiry has been located as of August 2026. Separately, 45 CFR 92.210 (adopted in the 2024 HHS section 1557 final rule) prohibits discrimination through the use of patient care decision support tools and imposes ongoing duties to identify tools employing inputs measuring race, color, national origin, sex, age, or disability and to mitigate the resulting risk — a standing duty that postdates the studied deployment and is carried for the duty it imposes, not as an adjudication of this case.

Sources: newyorkstatedepartmentoffina2019, cfr, snowbeck2019, ledford2019

Appears on: /domains/cases/cost-proxy-care-stratification

The Child Abuse Clinical Decision Support system (CA-CDS), built by a consortium at UPMC Children's Hospital of Pittsburgh, combines a five-item nurse-administered screen for every child under 13, a free-text scan of the chief complaint and nursing assessment, physician orders and discharge diagnoses into a rule-and-text trigger — no learned risk score — that raises a dashboard icon and a pop-up recommending a guideline-aligned physical-abuse order set. Implementation measurably moved volume: identification roughly quadrupled at two general emergency departments (P < .001), and triggering roughly doubled at both disseminated health systems (2.4% to 3.5% of children at the University of Wisconsin on Epic; 1.1% to 1.9% at Northwell Health on Allscripts). Its published performance — sensitivity 96.8%, specificity 98.5%, positive predictive value 26.5% — was measured against the hospital child protection team's chart assessment as reference standard: a concordance with an expert record judgment, not detection of abuse in any child. The documented failure mode is a string match: 'burn' in a 3-month-old's chief complaint is a trigger and so is 'burning up with fever'; 'broke' in a young infant triggers on 'the father speaks broken English'; 70% (33 of 47) of audited overtriggers came from the free-text scan, and 81% (195 of 242) of triggers were judged appropriate on chart review.

Sources: berger2019, feldstein2023, rosenthal2019

Appears on: /domains/cases/pediatric-abuse-detection-cds

The CA-CDS deployment's working end product is a mandated report that crosses a one-way organisational boundary. An individual clinician, discharging a personal statutory duty under each state's mandatory-reporting laws pursuant to the federal Child Abuse Prevention and Treatment Act, transmits the report to the state child protection agency: health-care workers are designated reporters in 46 states plus DC and five territories, no institutional policy relieves the individual duty in 17 states plus DC and the Virgin Islands, and the standard is suspicion with no burden of proof. Implementation raised reports to child protective services from 0.6% to 0.9% of children at one health system (P = .03), with 17% of triggered children reported against 0.26% of non-triggered. The receiving agency must screen every referral; in federal fiscal year 2023 states screened in 2,107,473 referrals against an estimated 2,292,000 screened out, on 5,936 intake workers and 21,739 investigation and alternative-response workers completing 66 responses each per year at a mean first-contact time of 102 hours, with 28 states reporting increases and citing staff shortages and turnover. The statutory scheme establishes no duty to return the screening decision or disposition to the reporting clinician's record, and no source documents a record-level write-back onto the hospital chart; the report joins a durable agency record whose prior-involvement and central-registry checks are standard elements of the investigation that follows the next report about the same family.

Sources: childwelfareinformationgatew2023, childwelfareinformationgatew, u2023, feldstein2023

Appears on: /domains/cases/pediatric-abuse-detection-cds

The CA-CDS record measures both its check and the gap over it. The physical-abuse order set produced 100% full guideline compliance when used at the originating hospital (43 uses in a seven-month trial; partial compliance fell from 10% to 3%, P = .04) and 96% (22 of 23 uses) at Northwell Health — while 58% of surveyed practitioners reported not using it at all. Of 71 practitioners analysed across 19 UPMC general emergency departments in February 2020, 69% did not recognise the dashboard icon indicating a trigger, 54% did not know they could view the screen result, 27% could not recall seeing an alert, and 65% were uncertain which tests to order; only 4.5% (3 of 66) disagreed with the recommendations, 75% said the tool raised awareness, 72% discussed alerts face-to-face with the child's nurse, and 54% named lack of social work or ancillary support as a barrier. Full compliance with the guideline evaluation at the moment of reporting ran 80% to 75% at Wisconsin and 33% to 50% at Northwell, and site acceptance diverged sharply on nominally identical software: 81% of Wisconsin respondents wanted continued use against 3% at Northwell.

Sources: suresh2018, feldstein2023, peterson2024

Appears on: /domains/cases/pediatric-abuse-detection-cds

The only deployment-level equity measurement for the CA-CDS is a null: screening rates did not differ by patient or hospital characteristics across 13 UPMC general emergency departments. No source publishes trigger, report or compliance rates disaggregated by subpopulation for this deployment. The domain's best-known disparity measurement — 22.5% of white versus 52.9% of minority children reported for suspected abuse among 388 children under 3 with acute fractures, and skeletal surveys ordered at 8.75 adjusted odds for minority toddlers — comes from a different institution (an urban academic children's hospital in Philadelphia) and era (1994-2000), and is the measured clinician variability a universal, instrument-driven screen is meant to standardise: context for this case, never a measurement of this system.

Sources: berger2019, lane2002

Appears on: /domains/cases/pediatric-abuse-detection-cds

StopNCII.org (operated by the Revenge Porn Helpline within the UK charity SWGfL, developed with Meta, launched December 2021) and Take It Down (operated by the US National Center for Missing & Exploited Children, launched February 2023) run one on-device hash-removal mechanism in two configurations: the person who holds the material generates a hash on their own device, only the hash leaves the device, the original is never uploaded or stored, third-party submission is refused and eligibility is self-attested, and participating platforms match the hash against uploads on public or unencrypted surfaces, reviewing any match under their own policies. StopNCII states its algorithms as PDQ and PhotoDNA for photos and MD5 for videos; independent researchers verified by inspecting the Take It Down web client that it runs PDQ. The operators' own volume reports — advocacy-tier counts of reporting behaviour, never incidence — were 2 million images protected across more than 785,000 cases by November 25, 2025 on the StopNCII side (a reported 97 percent increase over 2024, 17 industry partners) and 130,000+ submissions covering 273,000+ images and videos in 2025 on the Take It Down side, up from 83,000+ submissions in 2024. As fetched 2026-08-27 the two partner rosters overlap and diverge (YouTube on the minor index only, X on the adult index only, Discord on neither), encrypted surfaces are outside both by design, and Google announced on September 17, 2025 that it will use StopNCII hashes in Search — an announcement scoped to Search results, with Google absent from the StopNCII partners page as fetched the same day.

Sources: stopncii, nationalcenterformissingexpl, nationalcenterformissingexpl2025, hawkes2024, swgfl2025, google2025a

Appears on: /domains/cases/stopncii-hash-removal

A peer-reviewed independent evaluation (IEEE Security & Privacy Magazine 2024) reconstructed recognizable pre-images — hair colour and length, face shape, other facial features, some background — from PDQ, PhotoDNA, NeuralHash and aHash hashes using an off-the-shelf conditional image-to-image generative network trained on 1,000 public celebrity-face images on 2015-era consumer hardware, with mean perceptual similarity of 60.10 percent for PDQ and 74.04 percent for PhotoDNA, measured on a public celebrity-face benchmark and never on any reporter's material; the authors concluded the hashes should be treated as sensitive in the same way as the original images. The same team quoted Take It Down's FAQ answer that material 'cannot be reverse engineered or created from the hash values shared with NCMEC', reported that answer to be wrong, wrote to the operator twice between August and December 2023, received no reply, and recorded that the website did not change; the sentence was still on the FAQ page on 2026-08-27. A separate USENIX Security 2023 evaluation demonstrated efficient targeted second-preimage and detection-avoidance attacks against PhotoDNA and PDQ, concluding existing perceptual hash functions are likely insufficiently robust for adversarial settings. No operator in this class publishes an audited error rate for its deployed system.

Sources: hawkes2024, prokos2023, nationalcenterformissingexpl

Appears on: /domains/cases/stopncii-hash-removal

The two configurations diverge on the victim channel by design, and both operators document the propagation surface a withdrawal does not reach. StopNCII gives a reporter a case number, PIN and status page (updates can take 3 to 5 days) and services whole-case withdrawal against credentials the operator states it does not store and cannot reset; Take It Down is anonymous by construction, with no status channel, no notification of matches, and no withdrawal mechanism at all. StopNCII states that hashes persist after the reporter deletes the image and are shared with new partners as they join, and its FAQ states that participating companies 'reserve the right to continue enforcing their policies once they've acquired knowledge of the hash' — so an entry's reach grows after submission and a withdrawal retracts the bank's copy but not partner-held copies. A hash match obliges nobody: matching platforms review the content against their own policies before any action, platform-side matching is deployed self-hosted or by vendor API, and every documented failure of this protective system is a protection gap — a non-participating platform, an encrypted surface, a re-encoded copy that no longer matches, a partner copy a withdrawal cannot reach — never a false accusation.

Sources: stopncii, nationalcenterformissingexpl, thorn

Appears on: /domains/cases/stopncii-hash-removal

A University of Michigan audit study (ACM CSCW 2026) posted 50 synthetic-persona deepfake nude images to X and reported half through X's non-consensual-nudity mechanism and half as DMCA copyright violations: the DMCA reports achieved 100 percent removal within about 25 hours (mean 20.3 hours), while the non-consensual-nudity reports achieved zero removals over 21 days. The study tested X's victim-facing report channels, not hash matching, and X is a StopNCII partner — independent evidence that a platform's participation in a hash program can coexist with a non-functioning victim-facing report channel, which is why the match channel and the platform report channel are modeled separately.

Sources: zhangandcolleagues2026

Appears on: /domains/cases/stopncii-hash-removal

The voluntary hash indexes acquired a mandatory legal overlay during 2024-2026, on separate tracks that operate independently of hash-program membership. In the US, the TAKE IT DOWN Act's Section 3 notice-and-removal duty took effect May 19, 2026: covered platforms must remove reported non-consensual intimate imagery and known identical copies within 48 hours of a valid request, enforced by the FTC, which opened a complaint portal and sent compliance letters to fifteen major companies including Alphabet, Discord and X — companies that do not all participate in either voluntary hash index. In the UK, NCII sharing became a priority offence under the Online Safety Act via 2024 regulations, a creation offence covering purported intimate images including deepfakes came into force February 6, 2026 under the Data (Use and Access) Act 2025, Ofcom's hash-matching code measures were expected in force from summer 2026, and a Crime and Policing Bill amendment announced February 19, 2026 would require 48-hour takedown with penalties up to 10 percent of worldwide turnover. The statutes mandate responding to removal requests; they do not require joining either hash index, and they leave the indexes' unauditability untouched.

Sources: federaltradecommission2026, fladgatellp2024, osborneclarke2026

Appears on: /domains/cases/stopncii-hash-removal

Under Digital Services Act Articles 15(1)(e) and 42(2) — which require indicators of the accuracy and possible rate of error of automated content moderation, broken down by each official Member State language — X published, for the period 1 October 2024 to 31 March 2025, appeal and overturn rates for automated hateful-conduct visibility filtering across twenty named EU languages: appeal rates from 0.0 to 14.0 percent and overturn rates from 0.0 percent (Hungarian) to 72.2 percent (Swedish), with English at 40.4 percent. The independent audit of the sector's harmonised filings — a different set of reports from the April 2025 table above — found that of the eight largest EU platforms, four reported language-wise classification metrics (Facebook and Instagram across all 24 official languages, LinkedIn 22, X 7), three substituted country, and one reported none. Those are self-reported classification metrics; this atlas, not the audit, reads the April 2025 table as the only language-indexed error signal of its kind, because it indexes an outcome of the correction channel — appeals and overturns — rather than a platform's own accuracy figure. The figures are the platform's own legally mandated self-report, filed under Commission scrutiny, and no independent measurement of the deployment's per-language enforcement accuracy exists in the record.

Sources: regulation2022, xinternetunlimitedcompany2025, trujillo2026

Appears on: /domains/cases/dsa-multilingual-hate-speech

As of March 2025, X named 1,352 content moderators with primary-language professional proficiency covering 8 of the 24 official EU languages — 1,197 of them (88.5 percent) in English — plus a secondary list of 185 people, stated to be not distinct from the first, adding three more languages; thirteen official languages appear in neither list. For those, the company states: 'In situations where we need additional language support, we use translation services and/or machine translation tools, to investigate and address challenges in additional languages.' Flagged content is either human-reviewed before action or auto-actioned on the model's historical accuracy, and the models are trained on labels generated by the company's own trained moderators. Civil-society review found the same reviewer-language asymmetry sector-wide (for example Meta at 1 named Maltese reviewer against 3,110 Spanish) and concluded there is 'no clarity on moderation outcomes per country or per language.'

Sources: xinternetunlimitedcompany2025a, internationalnetworkagainstc2025

Appears on: /domains/cases/dsa-multilingual-hate-speech

The published per-language error indicator is an overturn rate conditional on appeal: its denominator, the appeal rate, varies by language from 0.0 to 14.0 percent, and the same metric family exceeds 100 percent for other policies in the same report, marking it as a period ratio rather than a probability or population error rate. The three languages with 0.0 percent appeal rates (Bulgarian, Greek, Lithuanian) carry no overturn rate at all — no error measurement exists for them — and four official languages (Croatian, Irish, Maltese, Slovak) are absent from the table entirely. The platform's own footnote on blank cells ('Cells that are blank mean that there was no enforcement. For cells containing 0.0% value, there were no cases of successful appeals or overturns') is ambiguous, and the reading used here — blank overturn cells aligning with the three zero-appeal columns — is the internally consistent one. The published overturn rates do not sort by reviewer staffing (Swedish, with no named reviewer, at 72.2 percent; Hungarian, also with no named reviewer, at 0.0; English, with 1,197 reviewers, at 40.4), and no monotone relation between staffing and measured error is supported by the record.

Sources: xinternetunlimitedcompany2025, xinternetunlimitedcompany2025a

Appears on: /domains/cases/dsa-multilingual-hate-speech

In the reporting period immediately following the twenty-language table (1 April to 30 June 2025), X published the same visibility-filtering accuracy indicators broken down by country rather than by language, and its first harmonised-template filing covered language-wise metrics for 7 of the 24 official languages per the independent audit; independent analysis of the sector's reports separately found their figures disconnected and their category vocabularies inconsistent across platforms. Every moderation decision files a statement of reasons to the Commission's public DSA Transparency Database, whose landing figure records roughly 42 percent of submitted decisions across all platforms as fully automated. Enforcement against the operator is active but distinct from these tables: the Commission's formal proceedings opened 18 December 2023 — content moderation and dissemination of illegal content among the grounds — remain open with no published findings and were extended on 26 January 2026 (recommender systems, plus a new investigation into the platform's integrated generative AI assistant), and the EUR 120 million fine of 5 December 2025, the first DSA non-compliance decision, concerns transparency obligations (deceptive verified-checkmark design under Article 25(1), the advertising repository under Article 39, researcher data access under Article 40(12)) — not moderation accuracy, hate-speech enforcement, or the language tables.

Sources: xinternetunlimitedcompany2025a, trujillo2026, europeancommission2023, europeancommission2025, eucrim2026, ohnesorge2025, europeancommissiona

Appears on: /domains/cases/dsa-multilingual-hate-speech

M-Shwari, launched November 2012 by the Commercial Bank of Africa (now NCBA) in partnership with Safaricom over the M-PESA rails, scores customers with fully automated credit approval rules run on six months of the mobile network operator's transaction and airtime record: a score assigned at account opening irrespective of when the customer borrows, never disclosed to the customer, expressed only as a credit limit that is zero below the cutoff and grows on timely repayment. Loans in the evaluated era started at KSh 100 (the minimum has been KSh 2,000 since August 2020, with smaller borrowing redirected to the Fuliza overdraft), run 30 days at a 7.5 percent facilitation fee unchanged from 2012 through 2026, roll over automatically with a further 7.5 percent, and are reported to a credit reference bureau after 120 days of non-payment.

Sources: suri2021, cook2015, cytonninvestments2020

Appears on: /domains/cases/alternative-data-digital-credit

A peer-reviewed regression-discontinuity evaluation of M-Shwari found that 34 percent of eligible households took a loan, eligible households were 6.3 percentage points less likely to forego expenses in response to a negative shock, and digital loans did not substitute for other credit; the same evaluation's administrative data show no significant difference in first-loan default between borrowers just above and just below the score cutoff — a coefficient of 0.007 against a control mean of 0.066 — reported by the authors as a check on their study design, local to the cutoff, and the only published measurement in this deployment family of what the score separates at the margin where it decides.

Sources: suri2021

Appears on: /domains/cases/alternative-data-digital-credit

At market level — findings about the Kenyan digital-credit market and its app-based lender class, not about M-Shwari — nationally representative survey evidence (FinAccess 2019, borrower n = 2,194) puts digital-credit default at 13.8 percent against 6.4 percent formal, negative bureau listing at 5.3 percent of digital borrowers against 1.4 percent formal, 38.4 percent of digital defaults reported to a bureau against 22.7 percent of formal ones, and roughly 90 percent of all blacklistings attributed to digital credit, on a digital cohort the source documents as lower-income and more vulnerable than the formal cohort; the Competition Authority of Kenya's market inquiry with Innovations for Poverty Action (a 793-user phone survey plus an audit of provider transaction and account-level data) found a mean effective APR of 280.5 percent and median 96.5 percent, 77 percent of mobile loan users unable to repay at least once, 40 percent able to state their last loan's cost within five percent, and 27 percent aware of other providers' charges.

Sources: johnen2021, competitionauthorityofkenyaa2021

Appears on: /domains/cases/alternative-data-digital-credit

The Central Bank of Kenya's Credit Reference Bureau Regulations of 8 April 2020 barred negative listings at or below one thousand shillings, forced a one-off mass delisting below that floor, and required the credit score in every credit report, while non-bank digital lenders were barred from the credit information system until licensing; the Digital Credit Providers Regulations of 18 March 2022 brought digital lenders under CBK licensing, made two-way credit information sharing mandatory with the KSh 1,000 floor carried forward, required pre- and post-listing notice, capped recovery on non-performing loans, required ability-to-repay assessment and a complaints mechanism with 30-day resolution, prohibited phone-book access and shaming in collection, and made product and pricing-parameter changes subject to prior written approval; the Office of the Data Protection Commissioner's December 2023 guidance documents app-class providers taking 'phone contacts, call logs, pictures, location, messages, all mobile phone applications and everything in the phone' stored 'as a collateral' and sharing customer data with marketing companies without consent, directing collection of transaction detail only. By 14 April 2026 the CBK had licensed 227 digital credit providers from more than 800 applications, with licensed providers having granted 7.5 million loans worth KSh 133.5 billion as of February 2026; the Business Laws (Amendment) Act 2024 recategorised digital credit as non-deposit-taking credit business, and draft successor regulations were exposed for comment in August 2025 — a draft, not a replacement.

Sources: centralbankofkenya2020, centralbankofkenya2022, competitionauthorityofkenyaa2021, officeofthedataprotectioncom2023, centralbankofkenya2026, centralbankofkenya2025

Appears on: /domains/cases/alternative-data-digital-credit

ProPublica and The Capitol Forum (Patrick Rucker, Maya Miller, David Armstrong), computing from internal Cigna records and interviews with former employees, reported on March 25, 2023 that Cigna's PxDx review flags claims where the billed procedure code does not pair with the diagnosis code on a payer-authored list, and that company medical directors then sign the flagged denials in batches without opening patient files: over 300,000 payment requests denied through this method across a two-month period in 2022, at an average of 1.2 seconds of physician attention per case, with individual medical directors signing between roughly 60,000 and 121,000 denials in one to two months. A former Cigna doctor told the reporters, 'We literally click and submit. It takes all of 10 seconds to do 50 at a time.' The same records, per the investigation, show the match list being extended on cost grounds: adding autonomic-nervous-system testing in 2014 carried an internal projection of roughly 2.4 million dollars a year in savings, and the executive credited with developing the process said it had 'undoubtedly saved billions of dollars.' Cigna publicly disputes the article's characterization of the process - a spokesperson called a complaint built on it 'based on an article riddled with factual errors and misinformation' - and has published no substitute figures; the underlying documents are now in discovery. The investigation received the April 2023 Sidney Award. These figures are the investigation's computation and are not adjudicated fact.

Sources: propublica2023, sidneyhillmanfoundation2023, healthcaredive2023

Appears on: /domains/cases/cigna-pxdx

Cigna's own published account of PxDx, which is vendor-tier evidence and is corroborated in outline by the investigation, states that the review is 'procedure to diagnosis' code matching applied to roughly 50 common, relatively low-cost tests and procedures; that 94 percent of the claims subject to it are automatically approved and paid; that denials issued through it are 'less than 1 percent of our total volume of claims'; that the review 'occurs after the patient has received treatment and once their physician bills for the treatment'; that it 'does not involve algorithms, artificial intelligence, or machine learning'; and that in-network patients should not be billed for services denied this way. Two elements of that account are structural facts rather than contested framing: no care is gated by this review, because it runs after treatment has been delivered, so the decision allocates payment rather than access; and the screen is a deterministic list lookup rather than a learned system, which both sides of the dispute agree on. The wrong the record supports concerns the emptiness of the physician-review layer above the match and the authorship of the match list, not a model erring.

Sources: thecignagroupnewsroom2023, propublica2023

Appears on: /domains/cases/cigna-pxdx

The correction channel in this record is documented in two stages with different positions, and its reach and its per-item effect are separate facts. A denial appealed inside the plan goes to a different Cigna doctor; beyond that, an independent review organisation outside the plan can be reached. In the one patient arc the record follows end to end - a roughly 350 dollar vitamin-D blood test denied in autumn 2021 as not medically necessary - the internal appeal upheld the denial and the external independent reviewer reversed it roughly seven months after the denial was issued. What gates the channel is a design expectation rather than a measured rate: ProPublica reports from company records that Cigna internally estimated only about 5 percent of people would appeal a denial, on a set of claims selected for being low-dollar. That figure is an internal expectation about appeal propensity and must not be read as an observed appeal rate. Cigna has published no internal overturn rate for this review. The appeal and overturn statistics quoted in the May 2023 congressional correspondence (roughly one in five denials appealed, about 80 percent of appeals overturned) are Medicare Advantage prior-authorization figures the committee used as an analogy; they measure a different programme and are not measurements of this review.

Sources: propublica2023, rucker2023, u2023c

Appears on: /domains/cases/cigna-pxdx

On October 8, 2025 the California Department of Managed Health Care fined Cigna HealthCare of California, Inc. 500,000 dollars for improperly denying providers' claims as not medically necessary. The Department found that the plan 'reviewed and denied claims without physicians conducting clinical reviews of the claims prior to issuing denials' and that it used a different review process than the policy it had filed with the Department. Cigna agreed to pay the fine and to corrective actions including re-reviewing denials issued under the non-compliant process going back two years and revising and refiling its review policy. Department Director Mary Watanabe said the stability of the health care delivery system is impacted when health plans wrongly deny the payment of claims for health care services. These are regulator findings agreed to by the regulated entity and are stated as fact. The scope caveat is load-bearing: the Department's release names neither PxDx nor the investigation, so this action is described as addressing the plan's claims-review practice, consistent with the documented pattern, and never as a PxDx fine; the respondent is the California-regulated plan entity rather than the national group. No report of the two-year re-review's completion or its results has been located as of August 2026. The statutory standard the parallel state-law claim rests on is California Health and Safety Code section 1367.01(e), under which no individual other than a licensed physician or a licensed health care professional competent to evaluate the specific clinical issues may deny or modify requests for authorization for reasons of medical necessity.

Sources: californiadepartmentofmanage2025, kistingleung2025

Appears on: /domains/cases/cigna-pxdx

The federal class litigation is live, mixed and unadjudicated on the merits. Kisting-Leung v. Cigna Corp. (E.D. Cal., No. 2:23-cv-01477-DAD-CSK) was filed July 24, 2023. On March 31, 2025 Judge Dale A. Drozd granted in part and denied in part the motion to dismiss the third amended complaint: the ERISA section 1132(a)(1)(B) denial-of-benefits claim was dismissed with leave to amend, and plaintiffs elected on April 11, 2025 not to replead it; the ERISA section 1132(a)(3) fiduciary-duty claim proceeds; and the California unfair-competition claim proceeds on the licensed-physician-review theory. Three of the six original plaintiffs, including the named lead plaintiff, were dismissed for lack of standing after Cigna's Rule 12(b)(1) factual attack, supported by a declaration stating there were no PxDx denial letters associated with their claims - which establishes from the defense's own evidence that not every denial by this insurer runs through this review. Assuming arguendo the most deferential standard, the court held that reading the plan term requiring medical-necessity determinations by a medical director 'as allowing an algorithm to make the decision so long as a medical director pushes the button' would conflict with the plain language of the plan and constitute an abuse of discretion. That is a pleading-stage interpretation ruling on allegations assumed true, not a factual finding about what the company did. Cigna answered May 2, 2025; one further plaintiff was voluntarily dismissed by stipulation on August 13, 2026, leaving two. Per the parties' August 24, 2026 stipulation, Cigna has produced roughly 2.1 million pages and depositions are coordinated with Snyder v. The Cigna Group (D. Conn., No. 3:23-cv-1451-OAW, filed November 2, 2023), described there as another class action involving Cigna's PxDx process, so Cigna witnesses sit once for both actions; fact discovery closes in autumn 2026 and class-certification briefing begins October 29, 2026. No class has been certified and the trial setting will move. The House Energy and Commerce Committee's letter of May 16, 2023 demanded PXDX process documents, legality memoranda, the list of plans subject to the review, per-medical-director denial records and 2022 review, denial, appeal and overturn counts by May 30, 2023; no public committee findings, hearing record or released production has been located as of August 2026, and nothing here implies the inquiry concluded or found anything.

Sources: kistingleung2025, kistingleungv2026, georgetownlaw2026b, u2023c, healthcaredive2023

Appears on: /domains/cases/cigna-pxdx

EviCore by Evernorth (eviCore healthcare MSI, LLC), a Tennessee-domiciled utilization-review entity owned by The Cigna Group since 2018, performs delegated prior-authorization review for more than 100 client insurers — including UnitedHealthcare, Aetna, Blue Cross Blue Shield plans, Medicare and Medicaid contractors, and its own parent — covering about 100 million people, roughly one in three insured Americans. The mechanism is common ground between the company and its critics: an artificial-intelligence-backed algorithm scores each submitted request with a probability of approval, requests above the operating threshold are approved with no clinical review, requests below it route to in-house nurses and then to physicians, and only a physician may issue a denial. The algorithm approves or routes; it denies nothing. On October 23, 2024 ProPublica and The Capitol Forum (T. Christian Miller, Patrick Rucker, David Armstrong; co-published by CNN on November 7), working from internal documents, corporate data and interviews including five former employees, reported that this routing threshold is adjustable and that insiders called it the dial: 'If EviCore wants more denials, it can send on for review anything that scores lower than a 95%,' one former employee said, and a former executive said of the review rate, 'We could control that. That's the game we would play.' The same investigation reports strictness sold as a product feature — a marketed return of three dollars of medical spend avoided per dollar of fees, sales staff touting denial increases of up to 15 percent, insurers including Aetna and Cigna requesting 'high touch' configurations that send more cases to review, and 'risk' contracts under which the vendor keeps or splits what it saves below a client's baseline spending target. EviCore and Cigna dispute this characterization, stating that the company 'uses the latest evidence-based medicine' and that its algorithms exist 'ONLY to accelerate approval of appropriate care and reduce the administrative burden on providers.' The dial and everything hanging off it are the investigation's findings, resting on internal documents and largely unnamed former employees, and are not adjudicated fact.

Sources: miller2024, coalitiontostrengthenamerica2024

Appears on: /domains/cases/evicore-prior-auth-dial

The one enforcement action on this vendor's record is procedural, small and admitted. Between September 19, 2023 and January 29, 2024 the Connecticut Insurance Department conducted a market conduct examination of 196 of eviCore's calendar-2021 Connecticut files. Its report and stipulation and consent order of February 5, 2024, Docket MC 24-15, records 77 adverse-determination letters that omitted the mandatory notice of the 120-day external appeal; urgent-care, appeal and retrospective determinations issued past their statutory 48-or-72-hour, 30-day and 60-day clocks; three files whose documentation could not support regulatory review; two denials not reviewed by an appropriate clinical peer; and one appeal decided by the same physician who made the initial denial. The violations are of Conn. Gen. Stat. sections 38a-591b and 38a-591d and Regulation 38a-591-8. eviCore admitted the allegations, paid a $16,000 fine and undertook to file a corrective-action report within 90 days. These are utilization-review compliance findings and are stated here as fact because they were admitted. Their scope is load-bearing and is stated with them: the examination read FILES, it made no finding about the algorithm, the routing threshold or any denial rate, and presenting the fine as algorithmic enforcement would overstate the only enforcement loop on this record that ever closed. No regulator on this record has examined a routing threshold. (ProPublica's shorthand of '77 violations found in a review of 196 files' compresses the order: 77 is the external-appeal-language category alone, and the total findings span seven categories.)

Sources: stateofconnecticutinsuranced2024, miller2024

Appears on: /domains/cases/evicore-prior-auth-dial

Three measured signals sit around the contested threshold, each with its own limit. First, ProPublica's ANALYSIS of data that Arkansas publishes about eviCore's book of business found prior-authorization requests turned down in full or in part almost 20 percent of the time since 2021, against roughly 7 percent for Medicare Advantage plans overall in 2022. The ~20 percent is the reporters' analysis and not a finding Arkansas published, and the ~7 percent comparator covers a different population and a different programme, so the pair is an order-of-magnitude contrast and never a controlled comparison. Second, Vermont Medicaid materials showed quarterly denial rates under this vendor's review ranging from 6.1 percent to almost 15 percent, and a 2019 Vermont presentation recorded advanced-radiology requests falling 16 percent, to 3,629, and cardiology requests falling 38 percent after review began — the demand suppression the industry markets as the 'sentinel effect', whose governance property is that a request never submitted appears in no denial statistic taken by anyone. Third, a former eviCore physician reviewer, maternal-fetal medicine specialist Gail Miller, told the investigation she was required to decide at least 15 cases an hour, one every four minutes, frequently outside her own specialty, and left after nine months. All three are the ProPublica/Capitol Forum investigation's reporting, disputed in characterization by eviCore and Cigna. None of them is a measurement of the algorithm: every denial rate here sits downstream of human review and cannot be decomposed into a model error, and no deployment error rate for this system is published anywhere. Neither is the share of requests approved outright rather than routed to review, which is quantified nowhere public at all. The HHS OIG evaluation of Medicare Advantage prior-authorization denials and the Senate Permanent Subcommittee on Investigations majority staff report of October 17, 2024 document insurer-side prior-authorization automation raising denial rates, but both examine INSURERS and not this vendor; they are domain context, and nothing in them is attributed to eviCore.

Sources: miller2024, u2022, ussenatepermanentsubcommitte2024

Appears on: /domains/cases/evicore-prior-auth-dial

The scoring layer and the human reviewers both work against clinical criteria that eviCore authors itself, and the record contains one dated instance of that arrangement failing. Per the investigation, citing the audit, a 2018 CMS audit found that outdated eviCore cancer guidelines led to inappropriate denials for 30 patients at the Blue Cross insurer HCSC; eviCore retrained staff in response. Two properties of that event matter more than its size. It reached this vendor only through one of its insurer CLIENTS rather than through any regulator of the vendor itself, and it is the single documented demonstration that an outside reader can adjudicate this vendor's determinations against its own criteria — the check that the public record otherwise shows nobody performing. No audit cadence for the criteria library is published, and no independent technical evaluation of the scoring layer exists in the public record. An adjacent settlement suggests the incentive belongs to the market structure rather than to one firm: Carelon, formerly AIM and the utilization-management arm of Elevance, paid $13 million in 2022 over wrongful-denial techniques, without admitting fault.

Sources: miller2024

Appears on: /domains/cases/evicore-prior-auth-dial

The vendor's published position is the other side of this record and is vendor-tier evidence throughout. An Evernorth article updated October 22, 2024 — the eve of the investigation's publication — states 'We aren't in the denial business, we're in the approval business', and reports more than 500 board-certified physicians across more than 60 specialties, more than 1,200 nurses and other clinical specialists, roughly two-thirds of medical decisions rendered in real time, and 90 percent of approvals completed within one business day, with peer-to-peer consultations framed as educational. Every one of those figures is a company claim with no independent verification, and the auto-approve versus routed-to-review split they imply is published nowhere. Two later company statements are operator-tier and are dated rather than characterized. On its Q1 2026 earnings call, held April 30, 2026, Cigna reported removing hundreds of tests, procedures and services from prior authorization altogether and cutting medical prior-authorization volume by about 15 percent, citing the 2025 industrywide voluntary prior-authorization commitments; on the same call, incoming chief executive Brian Evanko announced a strategic review of alternatives for eviCore, contemplating partnership or combination with other industry participants, citing scalability, management attention and industrywide standardization, with no transaction underway per the company. Any reading of this deployment should therefore avoid assuming stable Cigna ownership going forward: the owner of the adjustable parameter may change hands.

Sources: evernorththecignagroup2024, thecignagroupqearningscall2026

Appears on: /domains/cases/evicore-prior-auth-dial

IBM's Watson for Oncology, a treatment-recommendation clinical decision support product trained by Memorial Sloan Kettering Cancer Center physicians, was in use at roughly 50 hospitals across five continents by September 2017, with only two named US adopters and primary markets in India, South Korea, China, Thailand and Mongolia; more than 70 Chinese institutions adopted it amid published reliability concerns, and some adopting hospitals marketed it to patients. It was sold as machine reading of the medical literature. It read no local health record: at each site a person abstracted the case by hand into a structured form of 13 to 17 attributes — 17 at a Danish pilot, 13 in Korean studies — and at Jupiter Medical Center in Florida a nurse spent roughly 90 minutes a week doing so, with the treating oncologist finding the output largely redundant with the plan already in hand. Its recommendation logic was undisclosed to buyers and IBM shipped multiple software versions without transparent change documentation, so evaluations run at different times were measuring different software.

Sources: ross2017, tupasela2020, strickland2019, jncijournalofthenationalcanc2017

Appears on: /domains/cases/ibm-watson-oncology

The system's recommendations were computed from a curated knowledge base built on a small number of SYNTHETIC — hypothetical, hand-written — cancer cases authored by Memorial Sloan Kettering physicians working with IBM engineers, together with literature and guidelines those physicians selected, resting on the expertise of a few specialists for each cancer type rather than on guidelines or evidence; the engineering post-mortem records that synthetic cases were adopted as a workaround after the system could not learn from the medical literature. Internal IBM Watson Health slide decks of June and July 2017, presented by the division's deputy chief health officer to management and later obtained by STAT, recorded 'multiple examples of unsafe and incorrect treatment recommendations' and stated that these raised 'serious questions about the process for building content and the underlying technology.' That phrasing is IBM's own internal-document language as reported by STAT; IBM publicly contested STAT's characterisation, the examples came largely from testing and training exercises, and no patient harm from a recommendation is documented anywhere in this record. Global sales continued for years afterwards, and no correction of the knowledge base following the finding has been located.

Sources: ross2018, strickland2019, ross2017

Appears on: /domains/cases/ibm-watson-oncology

No published study measured whether Watson for Oncology improved patient outcomes; the published evidence base is concordance — how often clinicians chose what the system chose — and it is tiered rather than uniform. The flagship study of 638 breast-cancer cases at Manipal, published in Annals of Oncology in 2018, reported 93 percent agreement, a figure reached after a blinded tumour-board re-review of the non-concordant cases lifted agreement from 73 percent, pooling the 'recommended' and 'for consideration' output tiers, on an author list carrying four IBM Watson Health-affiliated co-authors. The strongest independent evaluation, at Gachon Gil Medical Center in Korea with no IBM involvement and declared conflicts of none, measured agreement at the strict 'recommended' level in 41.5 percent of 65 advanced gastric cancer patients (87.7 percent counting 'for consideration'), attributing the divergence to unaccounted patient history, US-centric regimens outdated or non-standard locally, national insurance not covering recommended agents, and the S-1 regimen — standard practice in Korea — being absent from the knowledge base. A Danish pilot found roughly one-third agreement, its physician citing overweighting of American studies, and the hospital declined to adopt; at UB Songdo Hospital in Mongolia, which lacked oncology specialists, clinicians followed the recommendations approximately 100 percent of the time. A peer-reviewed methodological critique concludes that concordance cannot establish safety or benefit — agreement is compatible with both parties being wrong, and disagreement cannot distinguish system error from clinician error — and documents the MSK-trained system functioning as a de facto ground truth against which other countries' practice was scored.

Sources: somashekhar2018, canadianjournalofgastroenter2019, tupasela2020, ross2017

Appears on: /domains/cases/ibm-watson-oncology

The MD Anderson Oncology Expert Advisor was a separate product — an MD Anderson-owned build on IBM Watson technology, distinct from Watson for Oncology — which never reached clinical use. IBM support ended 1 September 2016; IBM and the university agreed the system was 'not ready for human investigational or clinical use, and its use in the treatment of patients is prohibited,' and it was 'not in clinical use and has not been piloted outside of MD Anderson.' The University of Texas System Administration Audit Office's special review, reported publicly in February 2017, found more than 62 million dollars paid to IBM and PricewaterhouseCoopers as of 31 August 2016 (approximately 39 to 40 million to IBM and 21 to 23 million to PwC across contemporaneous accounts), an 11.59 million dollar deficit spend against donations not yet received, fees consistently set just below the amount that would have required Board approval, invoices paid regardless of delivery, and procurement and IT-governance processes bypassed — while expressly disclaiming any opinion on the scientific basis or functional capabilities of the system. The figure is spend to date, not a fine or an audit-assessed loss, and the review is a procurement audit, not a scientific verdict on either product. No regulator enforcement action, medical-device action, or product-liability litigation over either system's recommendations was located as of August 2026, and no medical-device regulator reviewed Watson for Oncology's recommendations before it was marketed globally. IBM announced the sale of Watson Health's data and analytics assets to Francisco Partners on 21 January 2022; the sale closed on 30 June 2022, launching Merative around six named product families with the oncology treatment-recommendation products not among them. No formal discontinuation of Watson for Oncology was announced.

Sources: universityoftexassystemadmin2016, theregister2017, jncijournalofthenationalcanc2017, ibmnewsroom2022

Appears on: /domains/cases/ibm-watson-oncology

On April 11, 2018 the US Food and Drug Administration granted De Novo request DEN180001, received January 12, 2018 under Breakthrough Device review, permitting IDx LLC of Coralville, Iowa to market IDx-DR — in the agency's own words the first device authorized for marketing that provides a screening decision without the need for a clinician to also interpret the image or results, and the first autonomous diagnostic authorized in any field of medicine. The software is locked and deterministic, paired by the authorization to one nonmydriatic fundus camera model, and returns exactly one of two messages: more than mild diabetic retinopathy detected, refer to an eye care professional; or negative for more than mild diabetic retinopathy, rescreen in 12 months. In the pivotal trial (Abramoff et al., npj Digital Medicine 2018 — funded by the manufacturer, with company-affiliated authors) 900 adults with diabetes were enrolled at 10 primary care sites and 819 were fully analyzable, yielding 87.2% sensitivity (95% CI 81.8-91.2), 90.7% specificity (95% CI 88.3-92.7) and 96.1% imageability against a reading centre's widefield stereo photography and macular imaging, exceeding pre-specified endpoints of 85% and 82.5%; the FDA's own announcement states 87.4% and 89.5% for the same trial, an analysis-population difference. Camera operators were existing clinic staff who attested they had never performed ocular imaging, after a single four-hour standardized training.

Sources: abramoff2018, u2018a, u2026

Appears on: /domains/cases/idx-dr-autonomous-screening

The autonomy is fenced by the labeling rather than by a reviewer. The device is indicated for adults 22 and over with diagnosed diabetes who have not previously been diagnosed with diabetic retinopathy, used with the paired camera; patients are not to be screened if pregnant — retinopathy can progress rapidly in pregnancy — or if they report persistent vision loss, blurred vision or floaters, or have previously been diagnosed with macular edema, severe non-proliferative, proliferative or radiation retinopathy or retinal vein occlusion, or have had retinal laser treatment, intraocular injections or retinal surgery. It detects diabetic retinopathy and no other condition, exposes no severity gradations and issues no confidence figure. That eligibility and exclusion screen is administered by clinic staff before any image is taken, which places the surviving human judgment in this workflow before the camera rather than after the result. The escalation runs the same way: when image quality is insufficient the operator re-images, with pharmacologic dilation as the labeled next step — 23.6% of pivotal-trial participants required it — and current labeling warns that a patient who still yields no result after dilation may have vision-threatening diabetic retinopathy.

Sources: digitaldiagnostics, u2018a, abramoff2018

Appears on: /domains/cases/idx-dr-autonomous-screening

The measured benefit of this deployment is replicated, and every study of it carries a sponsorship or investigator-affiliation label. In the ACCESS randomised trial at two Johns Hopkins pediatric diabetes clinics (Wolf et al., Nature Communications 2024; investigator ties to the vendor disclosed; ages 8 to 21, investigational use below the device's cleared adult indication) 164 youth were randomised, and eye-exam completion within six months was 100% (81 of 81) in the autonomous-AI arm against 22% under scripted specialist referral, with follow-through after an abnormal result 64% against 22%, in a cohort that was 35% Black and 47% Medicaid-insured, and no adverse events. At system scale (Huang, Channa, Wolf et al., npj Digital Medicine 2024; same affiliation caveat; 30-plus primary care sites, roughly 17,600 diabetes patients a year) sites that deployed the system raised diabetic-eye-exam adherence from 46.1% to 54.5% between 2019 and 2021 while comparison sites moved -0.3 points, with Black patients gaining 12.2 points and the Asian-to-Black adherence gap narrowing from 15.6 to 3.5 points. A five-health-system programme covering roughly 151,000 diabetes patients (Journal of CME 2025, an industry-supported education-journal evaluation used for texture rather than headline figures) recorded 20,160 screenings, 72% suitable for diagnosis, 24% of suitable screens positive, ophthalmology referrals up 23% and anti-VEGF treatment up 27%, with one system reporting a 118% rise in screening rate, against barriers of physician hesitancy and workflow fragmentation. What scaled the deployment was payment rather than the authorization: the AMA CPT Editorial Panel created code 92229 for point-of-care automated retinal analysis effective January 1, 2021, and CMS's CY2022 Physician Fee Schedule final rule of November 2, 2021 set the first national Medicare payment for it — approximately 45 to 47 dollars nationally at launch by crosswalk to CPT 92325, effective January 1, 2022, with later-year rates drifting down.

Sources: wolf2024, huang2024, journalofcme2025, digitaldiagnostics2021

Appears on: /domains/cases/idx-dr-autonomous-screening

The one place trial performance and field performance part company is the image-quality gate rather than the classifier. In the only fully vendor-independent evaluation located (Hunfeld et al., Scientific Reports 2026; Karlsburg Diabetes Hospital, Germany; 875 patients, February 2020 to November 2021) 26.1% of patients' images could not be analyzed by the device and 10.5% of patients yielded no image at all, against 96.1% imageability in the sponsor-funded pivotal trial; the measured drivers were capture-side and demographic — mean pupil diameter 2.65mm against 4.28mm, mean age 63.9 years against 46.4, impaired retinal view with cataract documented in 61% of non-analyzable right eyes, and variability between examiners. Among images the device could analyze it held up, at 94.4% sensitivity and 90.5% specificity for severe disease, with exact grade agreement against an ophthalmologist at 54.2%. A five-health-system US programme found 28% of screenings not suitable for diagnosis, corroborating the direction. An earlier external evaluation of the pre-authorization European version of the software (van der Heijden et al., Acta Ophthalmologica 2018; Hoorn Diabetes Care System, Netherlands; 1,415 patients with type 2 diabetes, 898 of sufficient image quality; the founder a co-author) adds a separate caution about measurement itself: sensitivity for referable disease was 68% under one human grading scheme and 91% under another on the same eyes, with the human reference graders agreeing among themselves only 40% to 61% of the time.

Sources: hunfeld2026, abramoff2018, vanderheijden2018, journalofcme2025, digitaldiagnostics

Appears on: /domains/cases/idx-dr-autonomous-screening

The compensating controls that replaced the physician overread are regulatory and contractual, and their record is thin in the way an empty docket is thin. Direct queries of the FDA's public device databases on August 28, 2026 return no adverse-event reports for this device under either product name, no recalls for the product code or for the firm, and exactly one adverse-event report across the entire retinal-diagnostic-software class, for a competitor device; no product-liability or other litigation over the system was located. That is an empty reported-event docket after eight years, not a measurement of clinical safety: a screening false negative that waits out its 12-month rescreen interval would rarely generate a device adverse-event report at all. The De Novo created a durable device class — retinal diagnostic software, 21 CFR 886.1100, product code PIB, with special controls — through which seven follow-on clearances across four firms have since passed, including this device's own version 2.3 in 2021 and 2022, the documented change-control channel for a locked algorithm. IDx renamed itself Digital Diagnostics Inc. on August 19, 2020 alongside an acquisition, and the product was later renamed LumineticsCore. Academic legal literature records that the manufacturer carries medical malpractice liability insurance for the system and assumes liability for injuries arising from it; trade reporting scopes that assumption contractually to the system's diagnostic output rather than downstream care management, and no case has tested it.

Sources: u2026, researchhandbookonhealth2024, digitaldiagnosticsviaprnewsw2020

Appears on: /domains/cases/idx-dr-autonomous-screening

The object corrected here is a clinical equation, not a learned system. The standard formulas for estimating glomerular filtration rate from a serum creatinine result — the 1999 MDRD equation and then the 2009 CKD-EPI creatinine equation — applied a coefficient that raised the estimated kidney function of any patient identified as Black; in the 2009 equation that coefficient was 1.159 (95 percent CI 1.144 to 1.170). The stated biological rationale, higher average muscle mass, treated race as a biological rather than a social category, and the MDRD equation behind it was derived from roughly 1,400 White and fewer than 200 Black patients. Inker and colleagues reported in the New England Journal of Medicine on 23 September 2021 that the race-including equation overestimated measured GFR in Black patients by a median of 3.7 mL/min/1.73m2, and concluded that race in eGFR equations is a social and not a biologic construct; OPTN's own patient materials state in a different register that the race variable automatically increased all Black patients' eGFR values, by as much as 16 percent. A kidney transplant candidate begins accruing waiting time when the estimate reaches 20 mL/min/1.73m2 or lower, so an estimate raised above that line delayed the clock; two studies cited in the nephrology commentary put the lost time at 1.3 and 1.9 years for affected Black candidates.

Sources: inker2021, pavlakis2023, hrsaoptn

Appears on: /domains/cases/optn-egfr-race-correction

The correction arrived in two board actions. After a public comment period running 27 January to 23 March 2022, the OPTN Board of Directors unanimously approved 'Establish OPTN Requirement for Race-Neutral eGFR Calculations' on 27 June 2022, requiring every kidney transplant program to use an eGFR formula without a Black-race variable from 27 July 2022; the National Kidney Foundation, whose joint task force with the American Society of Nephrology had recommended a race-free equation, called the vote an important first step and said there is no place for race-based variables in evaluating organs offered through the allocation system. Because a prohibition runs only forward, the Board then unanimously approved the retroactive remedy on 5 December 2022, effective 5 January 2023: each kidney program had to assess its waiting list, identify Black candidates disadvantaged by race-inclusive eGFR, establish whether a race-neutral calculation would have qualified them sooner, and apply to OPTN to backdate the qualifying date. Eligibility required documentation that the candidate's eGFR was over 20 mL/min under the race-inclusive calculation and 20 mL/min or lower without it. Programs had until 3 January 2024 to complete assessments, submit every qualifying modification, notify all kidney candidates before and after assessment regardless of race, and file an attestation; programs that did not comply would be referred to the Membership and Professional Standards Committee.

Sources: hrsaoptn, nationalkidneyfoundation2022, unos2023, hrsaoptna, pavlakis2023

Appears on: /domains/cases/optn-egfr-race-correction

The undo ran by hand, program by program. Trade guidance written for transplant programs describes the working method: identify candidates through the OPTN custom reporting tool's 'Current waitlisted African American Candidates' query, pull historical laboratory results from the electronic medical record, from external and reference laboratories, from dialysis centers and through record-retrieval and health-information-exchange systems, compare the race-inclusive and race-neutral figures against the 20 mL/min threshold, submit an eGFR Waiting Time Modification Form in UNet with supporting documentation, and send two notifications — before and after assessment — to every kidney candidate regardless of race. The same guidance records the correction's own second wave as a documented operational pressure: some patients were getting a lot of time back, bouncing them to the top of the list to start receiving offers, which it describes as a significant operational impact on many transplant centers, especially in ensuring that a patient had been re-evaluated recently. Accrued waiting time is a term in the ranking donor kidneys are offered down, so a backdated qualifying date changes a candidate's position in the live deceased-donor offer sequence directly rather than serving as a note on a file.

Sources: theallianceorgandonationalli2023, hrsaoptna, hrsaoptn

Appears on: /domains/cases/optn-egfr-race-correction

OPTN published its own counts twice, and each figure carries the date its data was cut. The early monitoring report (published 4 December 2023, data through 5 July 2023) recorded more than 6,100 Black candidates with modified waiting times at a median of 1.7 years, of whom 491 had received a deceased-donor transplant and 15 a living-donor transplant; at that date 12 of 232 active kidney programs had submitted attestations. The one-year report (published 8 May 2024, data through 4 January 2024) recorded 14,701 waiting-time modifications processed at a median of 1.7 years, with roughly half of modified registrations receiving between one and three years, 2,709 modified candidates transplanted from deceased donors and 158 from living donors, and all 230 active kidney programs having submitted attestations confirming that lists were reviewed, candidates notified and required modifications submitted. These are counts of modifications and of registrations at specific snapshot dates rather than counts of distinct people, and they are cumulative figures that grew between the two reports.

Sources: hrsaoptn2023, hrsaoptn2024

Appears on: /domains/cases/optn-egfr-race-correction

An independent national evaluation measured what the programme's own counts could not. Schold and colleagues, publishing in the Journal of the American Society of Nephrology on 6 May 2025, assessed 44,912 Black candidate kidney waitlist registrations nationally and found that 32 percent (14,419) received an eGFR waiting-time modification worth a median of about 610 priority days, and that modified candidates had an adjusted hazard ratio of 2.85 (95 percent CI 2.7 to 3.02) for deceased-donor transplantation against candidates who were not modified. The same study found modification rates varying significantly by candidate characteristics and by transplant center; the investigators read that variability as indicating there was still mixed use of the policies and that the requirement was not articulated clearly at the beginning. The study's primary record was behind a paywall and a consent wall at verification time on 28 August 2026, and these figures are carried as corroborated through the trade report of the same study rather than as directly retrieved from the primary.

Sources: schold2025, healionephrology2025

Appears on: /domains/cases/optn-egfr-race-correction

The measured unevenness produced a further governance action rather than a closed file. The Membership and Professional Standards Committee, having observed programs implementing the requirements in various ways, referred a follow-on project to the OPTN Minority Affairs Committee; after public comment from 21 January to 19 March 2025 the OPTN Board approved 'Monitor Ongoing eGFR Modification Policy Requirements' at its June 2025 meeting, effective 10 September 2025. The update converts the one-time legacy audit into a standing per-candidate obligation: every registered kidney candidate must be assessed for eligibility, and each program must maintain written protocols and document compliance in three areas — confirming a candidate's race, fulfilling the notification requirements, and seeking supporting documentation, naming at minimum which sources will be reviewed. The notification requirements (education, eligibility and outcome) apply to candidates registered on or after 4 January 2024, and the update removes the superseded 3 January 2024 attestation language. Programs must complete the strengthened requirements by 11 September 2026, a date that had not passed as of 28 August 2026.

Sources: hrsaoptn2025, hrsaoptna

Appears on: /domains/cases/optn-egfr-race-correction

The equity framing around this programme belongs to the commentary and policy materials that used it, and the same commentary records the critique. Pavlakis, writing in the Journal of the American Society of Nephrology in 2023, describes the waiting-time modification as a restorative justice project in kidney allocation and also records that the policy has been criticised as being unfair to people suffering under other inequities besides Black or African American race, and as both unfairly too broad and too narrow, leaving open the broader question of whether all patients should accrue predialysis waiting time. The remedy's scope is bounded on the face of the policy: it reaches registered kidney candidates whose documentation meets the eligibility rule, it addresses the eGFR-driven delay and no other source of delay, and it can only advance a qualifying date, never delay one. No litigation and no enforcement action appears anywhere in this record as of 28 August 2026; the record is affirmative governance — a prohibition, a retroactive modification programme, published monitoring, an independent peer-reviewed evaluation, and a tightening in response to measured variance.

Sources: pavlakis2023, hrsaoptna, hrsaoptn2025

Appears on: /domains/cases/optn-egfr-race-correction

Practice Fusion, Inc., a free ad-supported cloud electronic health record used by tens of thousands of provider practices, admitted in a stipulated Statement of Facts that it solicited and received $959,700 from an opioid manufacturer's marketing department for a clinical decision support alert — $144,600 for a retrospective analysis and $815,100 for the alert work, under a statement of work effective 1 March 2016. It admitted that it had modeled the sponsor's return on investment at 5.8 to 7.8 times cost, with a projected 'patient gain' of 2,777 and $8,458,232 to $11,277,643 in additional opioid revenue, and had deliberately kept that model out of the written proposal; that the sponsor's Director of eMarketing proposed edits to the alert's workflow and treatment-option list and Practice Fusion's chief medical officer approved them, neither the closing account director nor the approving officer having experience treating pain or prescribing schedule II narcotics; and that an early patient-safety concept, screening patients for opioid-abuse risk, was discussed and dropped. On 27 January 2020 the U.S. Attorney for the District of Vermont charged the company by a two-count felony information — one count of soliciting and receiving kickbacks under 42 U.S.C. 1320a-7b(b)(1) and one count of conspiracy under 18 U.S.C. 371 — and resolved it by deferred prosecution agreement, in what the Department of Justice and trade press reported as the first criminal action against an electronic health records vendor. The company was never convicted and entered no plea. The global resolution was $145,000,000: a $25,398,300 criminal fine, $959,700 in forfeiture equal to the payment, and a $118,642,000 civil settlement.

Sources: statementoffacts2020, deferredprosecutionagreement2020, healthcaredive2020

Appears on: /domains/cases/practice-fusion-opioid-cds

The Pain CDS was not a statistical model. It was an authored cascade of three chained alerts over chart data: a prompt to record a pain score; a suggestion to complete a Brief Pain Inventory for patients with two or more pain scores at or above 4 out of 10 within three months or a chronic-pain diagnosis; and a prompt to create a pain follow-up plan, firing where pain at or above 4 was recorded twice within four months or an inventory was completed. It terminated in a drop-down of nine treatment options placed on equal footing, including 'Opioid Therapy (short-acting, long-acting/extended release)' alongside biofeedback, non-opioid analgesics, nonpharmacologic care, referral, surgery and 'pain resolved'. The stipulated record is that the CDC opioid-prescribing guideline published 15 March 2016 — start with immediate-release opioids, lowest effective dose, non-opioid therapy preferred — was circulated among the designers at both companies during development and was not incorporated, and that as built the alert offered extended-release opioids to opioid-naive patients and to patients whose pain was not chronic, contrary to that guideline, to the applicable clinical quality measure and to the sponsor's own approved product labeling. One documented sponsor edit is a 29 January 2016 change adding an 'Extended Release Opioid initiated' checkbox to trigger re-assessment. The alert named no drug brand at any point: unbranded clinical messaging was an explicit design feature and the admitted mechanism was steering between treatment categories rather than toward a product.

Sources: statementoffacts2020

Appears on: /domains/cases/practice-fusion-opioid-cds

The Pain CDS ran from 6 July 2016 to the spring of 2019 and, per the stipulated Statement of Facts, 'alerted more than approximately 230,000,000 times' — a count of alert displays, not of prescriptions and not of patients. Through 30 November 2016 alone it had fired during 21 million patient visits involving 7.5 million patients and 97,000 healthcare providers. Practice Fusion's own program analytics recorded that providers who received the alerts prescribed extended-release opioids at a higher rate than those who did not, with a general shift from immediate-release toward extended-release largest in emergency medicine, orthopedics and pain medicine: a comparison the platform made on its own platform data, not an independent causal study. The same analytics answered the clinical question in the other direction. Presented to the sponsor at its headquarters on 14 December 2016, they reported extended-release opioids as the least effective of the listed options at lowering pain — 39.17 percent of patients treated with them had lower pain — and second-least effective among chronic-pain patients. That analysis was delivered to the sponsor as the program reporting it had contracted for and not to the prescribers being alerted, and no channel returning observed outcomes into the alert's content is documented. A sponsor attorney at that meeting expressed reservations and considered pausing the program; it continued.

Sources: statementoffacts2020, americanmedicalassociation2021

Appears on: /domains/cases/practice-fusion-opioid-cds

The 2020 deferred prosecution agreement built a governance control where none had existed and the record then shows that control failing in operation. Its forward-looking terms: a three-year term extendable to a maximum of five, with the government obliged to seek dismissal with prejudice within 30 days of expiry; an independent Oversight Organization required to review and approve any sponsored clinical decision support before implementation; a public online repository of the documents underlying the conduct, hosted at Practice Fusion's expense with the sponsor's identity, employees and drug brands redacted; a compliance program separating clinical from commercial activities; and an obligation to report evidence of kickbacks by other record vendors. The Oversight Organization subsequently resigned. A U.S. Attorney's Office notice letter of 25 August 2021 alleged that the company had failed to retain a replacement, to give the organization adequate access to information and witnesses, and to pay certain of its expenses. A letter agreement of 17 March 2022, filed on the criminal docket, settled those allegations for $200,000 with an express no-admission clause and extended the agreement's term by eleven weeks, to 13 April 2023. The criminal docket shows a termination date of 9 May 2023; the dismissal order itself is PACER-gated and was not retrieved for this record.

Sources: deferredprosecutionagreement2020, settlementandreleaseagreemen2022, unitedstatesv

Appears on: /domains/cases/practice-fusion-opioid-cds

The civil settlement reached conduct far beyond the single criminal count, and every part of it is a government allegation resolved with an express no-admission clause. The United States alleged that Practice Fusion entered fourteen separate sponsored clinical decision support arrangements with various pharmaceutical manufacturers, first entered between 11 November 2013 and 17 August 2017, in which paying sponsors participated in designing the alerts — selecting the guidelines an alert cited, setting its trigger criteria, and in some cases drafting its language — with claims alleged tainted from April 2014 to April 2019. The Department of Justice's press release describes thirteen arrangements other than the criminally charged one; the settlement agreement itself enumerates fourteen in total, one contract having contained two. The United States separately alleged that the company obtained 2014 Edition ONC certification for software that disabled data export and lacked required SNOMED CT and LOINC support, causing false meaningful-use attestations between 2014 and 2017. The civil total was $118,642,000, of which $113,374,952 was federal — half of it, $56,687,476, as restitution — and $5,267,048 was escrowed for state Medicaid settlements; Connecticut announced its own share at $336,087.89. Allscripts Healthcare acquired Practice Fusion on or about 13 February 2018, after the conduct.

Sources: civilsettlementagreementbetw2020, stateofconnecticutofficeofth2020, healthcaredive2020

Appears on: /domains/cases/practice-fusion-opioid-cds

The Practice Fusion resolution papers pseudonymize the sponsor as 'Pharma Co. X' and the court-ordered public repository was required to redact its identity, so the sponsor is identified here only on the strength of the sponsor's own criminal case and later Department of Justice releases — never on the Practice Fusion documents. Purdue Pharma L.P. pleaded guilty on 24 November 2020 in the District of New Jersey to three felonies, among them conspiracy to violate the Anti-Kickback Statute through its payments to an electronic health records company to install prompts intended to cause prescribing of its extended-release opioids, as part of a resolution reported at more than $8 billion. Later Department of Justice releases in the related obstruction case state that Purdue paid Practice Fusion almost one million dollars in exchange for altering its physician-facing user interface to generate more opioid prescriptions. Separately, Steven Mack, Practice Fusion's former Director of National Accounts on that account, pleaded guilty on 8 March 2021 to attempting to obstruct the grand-jury investigation by deleting hundreds of files from his company laptop, and was sentenced on 13 May 2024 to one year of probation, a $20,000 fine and forty hours of community service involving people suffering from drug addiction. In June 2021 the American Medical Association's House of Delegates, citing this case, adopted policy opposing direct-to-prescriber pharmaceutical promotional content in electronic health records and e-prescribing software.

Sources: usdepartmentofjustice2020, vtdigger2024, americanmedicalassociation2021

Appears on: /domains/cases/practice-fusion-opioid-cds

On 13 February 2018 the FDA granted De Novo request DEN170073 for Viz.AI's ContaCT, creating the radiological computer-aided triage and notification device class (21 CFR 892.2080, product code QAS) that later stroke-AI entrants reach by 510(k). The FDA decision summary describes the device as a notification-only, parallel workflow tool: it analyses CT angiograms and alerts a neurovascular specialist 'in parallel to standard of care image interpretation', identification of suspected findings is not for diagnostic use beyond notification, and the benefit-risk section concludes 'there are no major risks for the device because the device operates in parallel to the current usual standard of care'. The three risks the order does list are triage risks: deprioritization of other patients' images, inappropriate use for primary interpretation, and delayed management if prioritization fails. In the pivotal standalone study of 300 CT angiograms from two US sites the algorithm measured 87.8% sensitivity (95% CI 81.2-92.5), 89.6% specificity (83.7-93.9) and an area under the curve of 0.91; in the same record notification arrived a mean 51.4 minutes earlier than the standard radiology pathway (7.3 against 58.7 minutes; medians 5.6 against 51.5; earlier in 42 of 44 cases) — a figure the FDA's own summary flags as potentially biased because it was computed on the 44 true-positive cases that happened to have documented standard-of-care notification times, and which this atlas therefore carries as that study's measurement rather than a field-wide constant.

Sources: usfoodanddrugadministration2018, u2018

Appears on: /domains/cases/viz-lvo-stroke-triage

Independent multi-site evaluation of this detector measured accuracy materially below the pivotal figures, and lowest on the distal occlusions. In a large integrated hub-and-spoke stroke network handling more than 6,000 code strokes a year, 3,851 patients were screened and 220 (5.7%) had a neuroradiologist-confirmed internal-carotid or MCA-M1 occlusion: sensitivity 78.2% (95% CI 72-83), specificity 97% (96-98), positive predictive value 61% (55-67), negative predictive value 99% (98-99), with sensitivity falling to 60.6% once MCA-M2 occlusions were included and false negatives driven by non-main-vessel occlusions, segmentation errors and difficult anatomy; those authors state that the software 'does not allow the clinician to confidently rule out' an occlusion on a negative result. A prospective study of 1,822 consecutive stroke-code CT angiograms (May 2019 to October 2020) at a three-tiered network found 190 occlusions and measured 93.8% sensitivity with 99.7% negative predictive value for internal-carotid-terminus and M1 occlusions, dropping to 74.6% sensitivity and 97.6% negative predictive value once M2 was included, with per-site detection of 100% at the carotid terminus, 93% at M1 and 49% at M2. A positive predictive value of 61% means roughly two proximal alerts in five are false. Both groups publish inside networks with substantial vendor-adjacent authorship, and the prospective study's site is a frequent vendor co-publisher.

Sources: karamchandani2023, matsoukas2022, usfoodanddrugadministration2018

Appears on: /domains/cases/viz-lvo-stroke-triage

In the FY2021 hospital inpatient prospective payment final rule (85 FR 58432, 18 September 2020) CMS approved a new-technology add-on payment for ContaCT of up to $1,040 per case — 65% of an applicant-estimated $1,600 cost — projecting roughly 12,700 cases and about $20.6 million in FY2021, billed through ICD-10-PCS 4A03X5D, with the newness period anchored to 1 October 2018; CMS renewed it for FY2022. The characterization of this as the first add-on payment granted for AI software is the applicant's, peer-reviewed commentary's and the press's framing: the rule text approves the technology without declaring a first, and in the same rule CMS questioned whether AI-based workflow streamlining is a unique mechanism of action, raised what 'new' means when an algorithm updates or a competitor's algorithm performs better, and reserved the category question for future rulemaking. What CMS accepted was the applicant's asymmetry argument — that a false positive causes only an earlier image review while a false negative leaves the patient in the unchanged standard-of-care pathway. The add-on is temporary by construction and its post-FY2022 status was not verified in rule text; this atlas does not assert that it is still being paid. Adoption counts for this deployment are vendor-tier and point-in-time: 'over 800 U.S. hospitals' at the August 2021 renewal release, and 1,400-plus hospitals and health systems across the US and Europe on the vendor's current undated page.

Sources: centersformedicaremedicaidse2021, hassan2021, viz2020

Appears on: /domains/cases/viz-lvo-stroke-triage

The add-on payment left a per-use administrative trace, and that trace is the only system-wide measurement of this deployment anyone holds. A 2026 Harvey L. Neiman Health Policy Institute study in the American Journal of Neuroradiology read 2,116 Medicare inpatient acute-ischemic-stroke episodes at 1,076 facilities between October 2020 and December 2023: add-on-billed use of the tool peaked at 21% of eligible episodes in 2022 and then declined, 14.8% across the whole window. Use concentrated at comprehensive stroke centres and facilities of 1,000 or more beds, with roughly twice the odds in the Stroke Belt, and showed no differences by patient demographics or stroke severity — facility resources, not patient factors, predicted use, which the lead author summarised as access depending 'more on where a patient is treated than on their clinical needs'. Two smaller before-and-after studies measured pathway compression with their designs attached and are not outcome evidence: in a hub-and-spoke network (n=43, 28 before and 15 after) the median interval from angiography at the primary centre to door-in at the comprehensive centre fell from 132.5 to 110 minutes, a 22.5-minute reduction (p=0.047), with reductions in overall and neuro-ICU length of stay; and in a transferred thrombectomy cohort at one academic centre (n=55) the median door-to-team-notification interval fell from 40.0 to 25.0 minutes (p=0.01) while the 25-minute door-to-puncture improvement did not reach significance (p=0.15). No randomised trial of this deployment exists, several of these authors hold vendor-affiliated work, and no outcome or mortality benefit is asserted.

Sources: pelzl2026, harveyl2026, hassan2020

Appears on: /domains/cases/viz-lvo-stroke-triage

After a ten-day ERISA bench trial in October 2017, the United States District Court for the Northern District of California issued 106 pages of Findings of Fact and Conclusions of Law (28 February 2019; public redacted version 5 March 2019) holding that the 2011-2017 editions of United Behavioral Health's Level of Care Guidelines and Coverage Determination Guidelines were significantly and pervasively more restrictive than generally accepted standards of care. The operative 3 February 2026 judgment declares eight specific deviations, among them excessive emphasis on acute crisis stabilization, no effective treatment of co-occurring conditions, failure to err toward a higher level of care where the indicated level is ambiguous, no coverage to maintain function, motivation-based exclusions in the 2014-2017 editions, no child-and-adolescent-specific criteria, an overbroad custodial-care exclusion paired with a narrow active-treatment requirement, and mandatory prerequisites in place of a multidimensional assessment; the editions also omitted the ASAM residential levels 3.1, 3.3 and 3.5. The court found the criteria operated as binding rules rather than guidance: only a physician or doctoral-level psychologist could issue a clinical non-coverage determination, such a reviewer typically spent about thirty minutes talking to the requesting physician and writing up conclusions, every denial letter had to cite the specific guideline relied on, and the company's testimony that reviewers could deviate from the guidelines on clinical judgment was found not credible. No machine-learning system is involved; the guidelines are codified decision criteria applied by human reviewers.

Sources: witv2019, witv2026

Appears on: /domains/cases/wit-ubh-guidelines

The court's central finding is about who wrote the criteria and under what pressure. In the words of the Findings of Fact at paragraph 180: 'The Court finds that the financial incentives discussed above have, in fact, infected the Guideline development process. In particular, instead of insulating its Guideline developers from these financial pressures, UBH has placed representatives of its Finance and Affordability Departments in key roles in the Guidelines development process throughout the class period.' Named Finance and Affordability representatives sat as members of the approving committees — the Behavioral Policy and Analytics Committee from 2011 to 2016 and the successor Utilization Management Committee from 2016 — which reviewed and reissued the guidelines at least annually; proposed changes were modelled for their benefit-expense impact before adoption and proposed liberalizations are recorded awaiting a 'green light' from finance. The court also found that the company prepared detailed benefit-expense forecasts and targets, tracked monthly trends, and took action to address benefit expenses exceeding its projections, and that a 2014 internal presentation named continued use of concurrent review to ensure appropriate utilization as the 'Mitigation Strateg[y]' for the 2008 Parity Act's removal of day and visit limits. Because the guidelines were kept uniform across fully-insured plans, where the company bears the benefit-expense risk, and self-funded plans, where it does not, the court held the conflict tainted its decision-making as to both categories.

Sources: witv2019, witv2026

Appears on: /domains/cases/wit-ubh-guidelines

Four states required these determinations to use external professional-society criteria rather than payer-authored ones, and the court adjudicated violations of all four: Connecticut, where ASAM criteria have been required since 1 October 2013, violated throughout; Illinois, required from 18 August 2011, violated until 1 January 2016; Rhode Island, required from 10 July 2015, violated through the class period; and Texas, under the criteria of its Department of Insurance, violated throughout. The operative judgment also declares that the company's 2013 and 2015 crosswalks told Connecticut regulators that all three ASAM residential levels were included in its admission criteria and that, at the time these statements were made to Connecticut regulators, the company knew them to be false. United Behavioral Health did not appeal this portion of the judgment, and the Ninth Circuit recorded that it therefore remains intact — so the state-mandate rulings survived every appellate cycle unchanged. California later answered legislatively: SB 855 (Stats. 2020 ch. 151, effective 1 January 2021) requires commercial plans to make mental-health and substance-use medical-necessity determinations using the current criteria of the relevant nonprofit clinical specialty association, naming ASAM, LOCUS/CALOCUS and CASII/ECSII, and forbids applying different, additional, conflicting or more restrictive utilization review criteria.

Sources: witv2026, witv2023, californiasb2020

Appears on: /domains/cases/wit-ubh-guidelines

The class-wide wrongful-denial theory did not survive. In its amended opinion of 22 August 2023 (Wit III, 79 F.4th 1068) the Ninth Circuit affirmed Article III standing and affirmed certification of the three classes for the fiduciary-duty claim, but reversed certification of the denial-of-benefits classes under the Rules Enabling Act; held that the district court erred to the extent it determined that the ERISA plans required the guidelines to be coextensive with generally accepted standards of care; held that reprocessing was not appropriate equitable relief under 29 U.S.C. section 1132(a)(3); and remanded the exhaustion question. When the district court's scope-of-remand order tried to preserve more than the mandate allowed, the Ninth Circuit granted mandamus on 4 September 2024 (No. 24-242) and directed entry of judgment for United Behavioral Health on the denial-of-benefits claim, writing that in its thorough analysis of the spirit of the mandate, the district court lost the letter. On remand the district court held on 5 August 2025 that the fiduciary claim survives Wit III insofar as it rests on the duties of loyalty and due care — entering judgment for the company on the duty-to-follow-plan-terms theory — and that the surviving statutory claim requires no administrative exhaustion, alternatively excused as futile on the trial findings. The roughly 67,000-request reprocessing ordered in November 2020 was stayed on 12 February 2021, held unavailable on appeal, and vacated; no coverage request was ever reprocessed under it. This is a bench record: findings of fact by a judge, no jury and no damages, with relief declaratory and injunctive.

Sources: witv2023, witv2025, witv2026, witv2020, manatt2021, congressionalresearchservice2022

Appears on: /domains/cases/wit-ubh-guidelines

The operative remedy is the Amended Remedies Order of 3 February 2026 (dkt. 695), which vacated the 3 November 2020 Remedies Order in its entirety and superseded it. It declares that the company's misconduct in developing and adopting the guidelines was willful and systematic, that the adjudicated editions are irreparably tainted by the company's disloyalty and lack of care, and that the company breached the ERISA duties of loyalty and care under 29 U.S.C. sections 1104(a)(1)(A) and (B) and violated the Connecticut, Illinois, Rhode Island and Texas criteria mandates. It permanently enjoins use of those editions to implement plan terms about generally accepted standards of care, and orders that for five years — through 3 February 2031, with jurisdiction retained — any criteria the company adopts for that purpose shall accurately reflect those standards as established in the court's Findings of Fact and the requirements of any applicable state law. The vacated 2020 order's features are not part of it: there is no special master, no supervised retraining programme, no court-specified external criteria catalogue, and no reprocessing. The ten-year injunction belonged to that vacated order and never took effect; the 2031 endpoint belongs to the new five-year mandate. Attorney-fee litigation was reported ongoing in mid-2026 and appellate review of the 2026 order remained possible as of August 2026, so the injunction posture should be re-verified before republication. Class-scale figures are recorded litigation facts — roughly 67,000 coverage determinations for roughly 50,000 people, because a member can have more than one denial — and the characterization that about half of those people were children or adolescents is plaintiff-side reporting.

Sources: witv2026, witv2020, thekennedyforum2026

Appears on: /domains/cases/wit-ubh-guidelines

Amazon's fulfilment centres time every task an associate performs and feed the result into structured discipline processes that generate written warnings, final warnings and terminations. A letter from an attorney for Amazon to the National Labor Relations Board dated 4 September 2018, obtained and published in April 2019, described a system in which 'Amazon's system tracks the rates of each individual associate's productivity, and automatically generates any warnings or terminations regarding quality or productivity without input from supervisors', and stated that hundreds of workers had been terminated at a single facility between August 2017 and September 2018: about 300 full-time workers at the Baltimore fulfilment centre over that roughly thirteen-month window, representing roughly ten percent of that site's workforce. Amazon disputed the characterisation on the record the day the documents were published — 'It is absolutely not true that employees are terminated through an automatic system' — adding that it would not dismiss an employee without ensuring they received coaching, that managers can intervene in the process, and that terminations can be appealed; a spokesperson described the same period as about 300 employees of productivity-related turnover at that site. Amazon contested the adjective and not the count. Six years later a U.S. Senate committee majority, working from internal documents produced to it and from two letters by Amazon's outside counsel in 2024, found in its own voice that 'When workers cannot keep up, Amazon uses automated systems to initiate disciplinary procedures. These disciplinary procedures progress in severity and eventually result in termination.' That is the finding of a committee majority, disputed by Amazon to the committee, and no court or regulator has adjudicated whether any individual termination issued without a human decision-maker. The tracker behind the 2019 documents was reported as ADAPT, the Associate Development and Performance Tracker; Amazon has confirmed that name in no source verified for this record, and the process names it put on the record with Congress are Structured Productivity Performance Review and Structured Quality Performance Review.

Sources: lecher2019, businesshumanrightsresourcec2019, cbsnews2019, mittechnologyreview2019, ussenatecommitteeonhealth2024

Appears on: /domains/cases/amazon-adapt-productivity-termination

The rule is a percentile of the governed population, and both sides of the record describe it that way. Amazon's counsel told the Senate committee that its speed-related discipline process compares 'each eligible [worker's] performance in a given week to the performance of other employees doing the same work at the same facility', that the slowest five percent may be disciplined, and that as of 2020 the system identified for potential discipline the bottom five percent of performers whose actual rate was 50 percent or less of the expected rate — with the committee recording that it does not know whether that 50 percent threshold remains policy. Eligibility is gated and quantified: speed-related discipline 'applies to only a minority of Amazon [workers] who work at fulfillment centers, specifically Tier 1 (entry level) [workers] who have worked in an eligible process path for at least five hours in a given week and for at least 160 hours over the course of the [worker's] tenure' — roughly sixteen shifts. Speed runs through Structured Productivity Performance Review and quality through Structured Quality Performance Review, driven by counted defects such as scanning an incorrect item, with a third stream running off 'unknown idle time' or time off task; internal charts reviewed by the committee show speed-related write-ups as by far the most common form of discipline, quality second. Amazon's public answer to the California citation describes the same design from the other side: 'The truth is, we don't have fixed quotas. At Amazon, individual performance is evaluated over a long period of time, in relation to how the entire site's team is performing.' Nothing in this arithmetic is machine learning: there is no learned model, no score and no prediction in it.

Sources: ussenatecommitteeonhealth2024, abclosangeles2024

Appears on: /domains/cases/amazon-adapt-productivity-termination

The documented human step sits at the exemption of accrued time rather than at the decision. Internal Amazon documents provided to the committee track unknown idle time 'in some instances, down to the second' — one warning cites 97.68 minutes of unknown idle time on a named date — and that accumulator fills from conveyor breakdowns, manager conversations, pallet problems and restroom trips alike. A manager holds a 'seek to understand conversation' about the accrual and may exempt part of it: in the instance the committee published, the manager exempted 14 of 48 minutes for the worker's travel to and from a restroom in a one-million-square-foot warehouse and issued a first written warning for the remaining 34 minutes as a violation of Amazon's Standards of Conduct. The committee found instances of more than ten days between an alleged time infraction and the delivery of the disciplinary consequence, and identified that delay as making it difficult for workers to defend themselves and increasing the likelihood that they are wrongly disciplined. Above the manager sits an escalation authority: a low-level manager reported that management had discretion over whether to terminate workers after a given number of write-ups and would not terminate when headcount was low, but only if Human Resources permitted the deviation from protocol. Throughput is coupled to labour demand as well as to performance — Amazon told the committee that during the peak holiday period 'non-automated warnings, reprimands, write-ups, and improvement plans are paused', and an internal August 2020 chart shows write-ups dipping through the October-to-New-Year peak of 2019 and rising sharply in the first week of January once peak ended. When speed-related write-ups were paused at the start of the pandemic, an internal Amazon team observed that warehouse managers increased their use of other write-up types: behavioural, attendance and safety. Amazon has published no termination rate, no appeal rate, no override rate, no exemption rate and no false-positive rate, and says only that 'the rate of termination is very low.'

Sources: ussenatecommitteeonhealth2024

Appears on: /domains/cases/amazon-adapt-productivity-termination

A second automated termination path runs off a different metric and is where the record shows a human reconciliation step being removed. The Senate committee found terminations for 'job abandonment' generated when automated time-tracking failed to account for workers on approved medical leave and registered large negative unpaid-time-off balances. A Human Resources employee described having, before automation, to review a daily report of workers with negative leave balances — sometimes finding workers on leave with hundreds of thousands of hours of negative time — and personally removing them so they would not be flagged for discipline. One worker recovering from a foot injury was terminated by email a week before her scheduled return. The two paths share the timekeeping and the termination surface and do not share a trigger. The live vehicle on this path is a pleading and nothing in it is adjudicated: Lyster v. Amazon.com Services LLC, a class action over the operator's workplace absence practices filed 12 November 2025 in federal court in New York. Amazon says claims that it does not follow federal and state law 'are simply not true' and that its accommodations team reviews each request individually.

Sources: ussenatecommitteeonhealth2024, cbsnews2025

Appears on: /domains/cases/amazon-adapt-productivity-termination

Whether the governed worker can read the ledger that judges them is contested on the record, and the gap between the two accounts is the case. Amazon's position, stated in response to the California citation, is that 'Employees can - and are encouraged to - review their performance whenever they wish. They can always talk to a manager if they're having trouble finding the information.' The congressional record describes the same channel differently: disciplinary consequences arriving more than ten days after the conduct they cite, and workers running their own parallel timekeeping in response — 'I keep a timer on my watch to keep track of everything', another worker carrying a notebook — because they cannot see or contest the system's own ledger in time. Six state legislatures have since built a statutory version of the same channel, and what they built is a read right rather than a limit on the rule: a written description of each quota and of the discipline attached to it, provided on hire and in the worker's primary language with notice of changes within two business days, plus a right to request the quota description and the most recent 90 days of the worker's own personal work-speed data, answered in New York within 14 calendar days at no cost together with aggregate data for similar workers at the same site. The worker-kept timer or notebook is therefore a record store that exists because the primary one is not legible to its subject in time to contest an entry.

Sources: ussenatecommitteeonhealth2024, abclosangeles2024, assemblybill2021, newyorkstatedepartmentoflabo2023

Appears on: /domains/cases/amazon-adapt-productivity-termination

Two independent state labour regulators have cited this design, and both cited the notice rather than the threshold. The California Labor Commissioner's Office cited Amazon.com Services LLC $5,901,700 for 59,017 violations of the state Warehouse Quotas law at the Moreno Valley and Redlands fulfilment centres, the violations occurring between 20 October 2023 and 9 March 2024 and penalised at $100 per violation under Labor Code section 2699(f); the inspection opened on 22 September 2022 and the Warehouse Worker Resource Center assisted. Labor Commissioner Lilia Garcia-Brower stated that 'The peer-to-peer system that Amazon was using in these two warehouses is exactly the kind of system that the Warehouse Quotas law was put in place to prevent. Undisclosed quotas expose workers to increased pressure to work faster and can lead to higher injury rates and other violations by forcing workers to skip breaks.' Amazon appealed: 'We disagree with the allegations made in the citations and have appealed.' Minnesota reached the same conclusion under its own statute: after an October 2023 inspection at the Shakopee facility, Minnesota OSHA issued two serious citations in April 2024 totalling $10,500, one of them for the violation that 'warehouse employees who were expected to meet a quota of selecting, stowing and packaging products were not provided a written copy of the quota before they were expected to meet the quota.' Amazon contested the citations. Both are agency determinations rather than judicial findings and neither was resolved in any source located for this record; the per-facility split of the California total that circulates in secondary coverage is stated by no primary source and is not carried here.

Sources: californiadepartmentofindust2024, minnesotadepartmentoflaboran2024, abclosangeles2024

Appears on: /domains/cases/amazon-adapt-productivity-termination

The correction channel the law actually built for this decision surface is a disclosure-and-data right, not a limit on the rule. California Labor Code sections 2100 to 2112, effective 1 January 2022, define a quota as a work standard under which an employee is required to perform at a specified productivity speed or handle a quantified amount of material within a defined time period and under which the employee may suffer an adverse employment action for failing to meet it; require a written description of each quota on hire; bar adverse employment action for failing to meet an undisclosed quota; give employees the right to request the written quota description and a copy of the most recent 90 days of their own personal work-speed data; and create a rebuttable presumption of retaliation for adverse action within 90 days of such a request or complaint. New York's Warehouse Worker Protection Act adds a fourteen-calendar-day no-cost response deadline, notice of quota changes within two business days, provision in the worker's primary language, and a right to aggregate speed data for similar workers at the same site. Six states now carry such statutes — California (2021, effective January 2022), New York (19 June 2023), Minnesota (1 July 2023), Washington (31 May 2024), Oregon (1 January 2025) and Connecticut (signed March 2026, effective 1 July 2026 with notices to current employees due by 1 August 2026). The Senate committee majority's own prescription aims one step further in, and is a proposal rather than law: the No Robots Bosses Act, which would 'prevent employers from exclusively relying on automated systems to make decisions about disciplining or firing workers' and 'require employers using automated decision-making systems to tell workers how the system works and how workers can appeal system decisions', recommended on the committee's finding that 'Amazon subjects workers to discipline based on automated systems that are prone to errors, including firing workers who are on medical leave'; and the Stop Spying Bosses Act, recommended on the finding that Amazon 'closely tracks workers' movements and actions throughout the workday, and uses this information to make disciplinary decisions.'

Sources: assemblybill2021, newyorkstatedepartmentoflabo2023, littlermendelson2026, ussenatecommitteeonhealth2024

Appears on: /domains/cases/amazon-adapt-productivity-termination

The litigation record on this mechanism is thinner than the volume of coverage suggests, and every item in it stops short of the question. The only merits ruling to date on the quota machinery went on pleading specificity: in January 2023 a U.S. magistrate judge in the Northern District of California dismissed a proposed class action alleging that hourly quotas of roughly 150 to 300 items discriminate against older workers, holding the allegations too vague and writing that 'simply because physical strength declines with age does not automatically mean that older workers are more likely to get injured or fail to keep up with the quotas.' That is reasoning about a discrimination theory and about the sufficiency of a pleading; it is not a holding that the quotas do not exist. The live federal vehicle, filed 12 November 2025 in the Southern District of New York, pleads the adjacent automated absence-control path and is wholly unadjudicated.

Sources: wiessner2023, cbsnews2025

Appears on: /domains/cases/amazon-adapt-productivity-termination

The governed side of this deployment has been quantified once, in a weighted opt-in national survey rather than a probability sample, and it is labelled as such wherever it is used. The Center for Urban Economic Development at the University of Illinois Chicago surveyed 1,484 frontline Amazon warehouse workers between April and August 2023, drawing respondents from 42 states and 451 facilities, recruited through targeted advertising with CAPTCHA screening, a fake-facility-code trap and fraud-cluster removal, and weighted to Amazon's 2021 workforce demographics. In it, 77 percent said the technology can tell whether they are actively engaged in their work always or most of the time, against 47 percent of warehouse workers industry-wide; 72 percent said how fast they work is measured in detail by company technology, against 58 percent; and 58 percent said their pace is always or most of the time ranked and compared with the pace of their coworkers, against 46 percent. Asked what the electronic monitoring is used for, 45 percent said it is mainly used to control or discipline workers and 36 percent said it is mainly used to develop workers' skills and abilities, with the control-or-discipline share rising to 52 percent among workers of more than three years. 45 percent said keeping up with the pace of work is hard (47 percent at fulfilment centres against 31 percent at sortation centres), 41 percent feel pressure to work faster always or most of the time, and 53 percent report always or most of the time feeling watched or monitored.

Sources: centerforurbaneconomicdevelo2023

Appears on: /domains/cases/amazon-adapt-productivity-termination

The largest governance instrument attached to these facilities does not reach this decision surface, and that is what separates this case from the pace-and-injury case that shares the same buildings. On 19 December 2024 the U.S. Department of Labor announced a corporate-wide settlement with Amazon requiring Site Ergonomics Leads who review corporate risk assessments and prepare annually updated site-level assessments, together with multiple employee channels — including anonymous ones — for raising ergonomic concerns, across fulfilment centres, sortation centres and delivery stations in federal OSHA jurisdiction, for a $145,000 penalty. The settlement contains no quota provision, no pace-setting provision and no discipline provision. The instruments that do reach this decision surface are the state warehouse-quota-notice statutes and the two contested citations issued under them, whose remedy is written disclosure of the rule and access to the worker's own speed data. Nothing in the record adjudicates whether the threshold itself is correct.

Sources: usdepartmentoflabor2024

Appears on: /domains/cases/amazon-adapt-productivity-termination

Amazon Flex, a first-party last-mile delivery programme launched in September 2015, rates its United States contract driver fleet into four standing tiers — Fantastic, Great, Fair and At Risk — computed from arrival punctuality at delivery stations, completion of routes inside the reserved block window, compliance with customer special requests, and delivery-quality signals. A Bloomberg investigation published on 28 June 2021, read here through the syndication carrying its full text, reports that algorithms scan incoming performance data and decide which drivers get more routes and which are deactivated, that human feedback is rare, and that terminations arrive by automated email; it interviewed fifteen drivers, four of whom said they were wrongly terminated, together with former Amazon managers and a former engineer, and documents deactivations following circumstances the input set cannot represent — locked apartment gates on predawn routes, malfunctioning lockers, hour-long waits at understaffed stations, a nail in a tire, snowbound rural roads and failed selfie identity checks. Former insiders told the investigation that the programme's benefits far outweigh the collateral damage and that the company decided it was cheaper to trust the algorithms than to pay people to investigate mistaken firings. Amazon disputes the characterisation: its spokesperson called the driver accounts anecdotal and unrepresentative and said the company has invested heavily in technology and resources to provide drivers visibility into their standing and eligibility to continue delivering, and investigates all driver appeals. No court or regulator has ever ruled on how standing is computed or on whether any particular deactivation was substantively correct, and no scoring internals, error rate, termination rate or reversal rate has been published by anyone. Scale figures are the operator's own and count downloads rather than active drivers — approximately 4 million globally and 2.9 million in the United States, with more than 660,000 in one quarter of 2021, up about 21 percent year over year; the only hard official count in this record is the 140,128 individual drivers a federal regulator paid tip refunds to for a single 2016-to-2019 window.

Sources: soper2021, thespokesmanreview2021, dent2021, aiincidentdatabase2021, federaltradecommission2021

Appears on: /domains/cases/amazon-flex-driver-rating

The correction channel around the automated decision is slow, templated and priced. As reported in June 2021, a deactivated Flex driver has ten days to appeal by email; first replies arrive the next day, read as machine-generated and are typically generic rather than specific to the incident, signed with a support agent's first or full name; a promised six-day review is often exceeded, and in the two cases the investigation follows end to end the final answer lands on day 26 and on day 11, with no pay in the interim. The only escalation beyond email under the platform terms then in force carried a $200 arbitration filing fee — verbatim, drivers 'pay $200 to take their dispute to arbitration, but few do, seeing it as a waste of time and money' — against roughly $80 of net pay on a typical four-hour block, a fee legal commentary describes as an unrealistic prospect for workers barely making minimum wage. Drivers also report that standing takes months to recover from delays outside their control while standing feeds route and block allocation. The figure is date-stamped: Amazon's widely reported 2021 removal of mandatory arbitration applied to consumer terms rather than to the Flex driver terms, and the present fee level was not independently verified. One consolidating source overstates the position by saying drivers had no opportunity to contest or appeal; the primary record documents a channel that is automated, slow and fee-gated rather than absent, and no appeal-outcome or reversal rate exists from any side.

Sources: soper2021, thespokesmanreview2021, dent2021, bajgiran2022, aiincidentdatabase2021

Appears on: /domains/cases/amazon-flex-driver-rating

Since 1 January 2025 the deactivation channel itself is regulated in one jurisdiction. Seattle's App-Based Worker Deactivation Rights Ordinance (Seattle Municipal Code 8.40, Ordinance 126878; administrative rules SHRR Chapter 260 effective 24 June 2025) covers network companies with 250 or more app-based workers worldwide, Amazon Flex among them, and requires a published deactivation policy reasonably related to safe and efficient operations; fourteen days' notice before deactivation except for egregious misconduct or where law requires immediacy; a written statement of the reasons and the specific incidents together with all records relied on and considered; investigation of the alleged violation to a more-likely-than-not standard before deactivating; penalties applied consistently and in proportion to the violation with the circumstances of the work considered; and an internal challenge procedure invocable within ninety days, with a private right of action after the company's response or fourteen days after the challenge. A worker is covered at twenty-five percent of completed or cancelled-with-cause offers in the prior 180 days performed in the city, or by a single incident there, and the Office of Labor Standards enforces procedural compliance only until 1 June 2027, not whether a deactivation was substantively warranted. That office's published resolved-investigations record for October to December 2025 shows two informal resolutions under this ordinance against Amazon Logistics, Inc. doing business as Amazon Flex, each returning $1,245.70 to one worker and requiring Amazon Flex to restart that worker's deactivation process, alongside equivalent resolutions the same quarter against three other network companies. In Uber Technologies, Inc. v. City of Seattle, Nos. 25-228 and 25-231 (9th Cir. 4 March 2026), a panel of Graber, Clifton and Bennett affirmed the denial of a preliminary injunction sought by Uber and Instacart, holding that the ordinance regulates nonexpressive conduct — the unwarranted deactivation of worker accounts — that any compelled disclosure would in any event be commercial speech surviving Zauderer review, and that 'reasonably related to safe and efficient operations' is not unconstitutionally vague, with Judge Bennett dissenting in part. That is a preliminary-injunction affirmance rather than a final merits judgment, and Amazon was not a party.

Sources: cityofseattle2025, seattleofficeoflaborstandard2025, ubertechnologies2026, komonews2025

Appears on: /domains/cases/amazon-flex-driver-rating

The independent-contractor classification underneath the deactivation regime is contested and has repeatedly gone against the operator in adjacent forums, while no forum has reached the rating rule. A Wisconsin Department of Workforce Development audit of more than 1,000 Flex drivers covering 2016 to 2018 found the vast majority employees for unemployment-insurance purposes with an assessment of about $205,000; that determination was upheld by the state Court of Appeals in 2023 and left standing when the Wisconsin Supreme Court dismissed Amazon's appeal as improvidently granted on 26 March 2024. The New Jersey Department of Labor and Workforce Development sued Amazon on 20 October 2025 in Essex County Superior Court alleging Flex misclassification since at least 2017 with millions of dollars in annual losses to state benefit funds; that suit is pending and entirely unadjudicated. In Rittmann v. Amazon.com, Inc., No. 19-35381 (9th Cir. 19 August 2020), the court held that Flex delivery workers are transportation workers engaged in interstate commerce and exempt from the Federal Arbitration Act under 9 U.S.C. § 1 even without crossing state lines, and that the arbitration provision with its choice-of-statute clause was unenforceable under federal or Washington law, affirming the denial of the motion to compel arbitration of the wage claims. That holding concerns the WAGE-claims clause; the deactivation-dispute channel continued to route through individual arbitration as reported in 2021, and the two facts are distinct.

Sources: pbswisconsinassociatedpress2024, rittmannv2020

Appears on: /domains/cases/amazon-flex-driver-rating

What the scorer measures, and what it has no field for, are both documented. Human Rights Watch's May 2025 cross-platform study — 95 workers interviewed across 13 states including Flex drivers, plus a 127-worker Texas survey — records that Flex times a worker from arrival at the warehouse through completion of the delivery and detects whether the worker is driving, walking or running, while noting that Flex is the one studied platform paying a posted flat hourly block rate of $18 to $25 rather than an opaque per-job wage algorithm; the study's $5.12 hourly after-expenses median is a cross-platform Texas figure and is not a Flex figure. Mandatory selfie identity checks are part of the same surface, and failed checks are among the documented contributors to adverse action. A Harvard Law School labour-law commentary of January 2022 additionally characterises the reporting record as including monitoring of seatbelt use, acceleration and screen touches while driving; that characterisation mixes Flex with Amazon's separate Delivery Service Partner van programme, whose inward-facing camera telematics does not apply to Flex drivers, who use their own vehicles. What none of these inputs can represent is the reason a stop went wrong — a locked building, a closed office, a station queue, weather, road conditions or a vehicle failure — which the investigation records arriving in the measurement as the driver's own shortfall.

Sources: humanrightswatch2025, bajgiran2022, soper2021, thespokesmanreview2021

Appears on: /domains/cases/amazon-flex-driver-rating

Aon Consulting, Inc., the human-capital arm of Aon plc, designs, markets and administers a suite of pre-hire assessments sold into other companies' hiring pipelines. Three of them are at issue: ADEPT-15, a computer-adaptive forced-choice personality test scoring fifteen constructs from item pairs matched for social desirability out of a bank of more than a thousand statements, with no skip option and no agree-with-both option, whose outputs are construct scores, job-fit scores and generated interview scripts for the employer; gridChallenge, a gamified working-memory test of nine items each pairing a timed target-icon memorisation task with a symmetry or shape-judgment distractor task; and vidAssess-AI, an asynchronous video product that transcribes a candidate's spoken answers with a commercial speech-to-text service and scores phrases in that transcript as far-extreme positive or negative indicators of the same ADEPT-15 constructs. On Aon's own marketing, quoted in the ACLU's complaint, the company administers more than 30 million assessments a year in over 40 languages across 90 countries, and third-party market data quoted in the same paragraph places it at the second-largest share of the global pre-hire assessment market. The assessments are marketed to screen out applicants prior to more resource-intensive hurdles such as resume screening and interviews, and Aon's own case-study marketing is cited for one client that screened out 62 percent of applicants on the assessment alone; for the population removed at that step there is no human review, and the assessment is the first and the final decision point. Every scale and screen-out figure here is Aon's own marketing and none is independently audited.

Sources: americancivillibertiesunionf2024, americancivillibertiesunion2024, hrdive2024

Appears on: /domains/cases/aon-assessment-suite

The figures at the centre of this record are Aon's own, published in Aon's technical documentation and quoted at table and page level in the ACLU's complaint to the Federal Trade Commission; the numbers are Aon's and the reading of them is the ACLU's. On ADEPT-15, the reliability table shows ten of fifteen constructs producing two-week test-retest coefficients below the 0.7 floor Aon's own documents tell buyers to demand, the lowest being 0.44 on 'Awareness', which the complaint identifies as the construct with the greatest overlap with autism diagnostic screeners. On gridChallenge, the gridChallenge and GAME test documentation at Table 43 reports Black assessment-takers scoring on average 0.48 standard deviations below white assessment-takers, with two-or-more-ethnicities at 0.38, Hispanic or Latino at 0.37 and Asian at 0.35; Aon's documentation for sibling cognitive tools reports Black test-takers averaging 0.86 standard deviations lower on scales clues and 1.21 lower on switchChallenge, which are 'large' effects under the Cohen's-d heuristic Aon itself applies in the same documents. An average score difference is NOT an adjudicated adverse impact: the complaint concedes at paragraph 62 that actual selection impact depends on the cut-off scores an employer sets and on how the assessment is combined with other tools, which Aon does not control. No error rate, accuracy figure or independent evaluation exists for any of the three products, from any source. The manuals these figures come from were removed from public access after the EEOC charges were filed and could not be independently re-verified; the ACLU states it knows of no other public source and cites archived copies. Aon disputes the complaint and states that its assessment solutions follow industry best practices as well as the Equal Employment Opportunity Commission, legal and professional guidelines and are intended to be fair to everyone.

Sources: americancivillibertiesunionf2024, hrdive2024

Appears on: /domains/cases/aon-assessment-suite

The ACLU's disability theory runs through construct overlap rather than through any measured screen-out. Statements the EEOC charging party recalled encountering on ADEPT-15 — among them 'I tend to avoid large groups of people' and 'I have difficulty determining how someone feels by looking at their face' — are set in the complaint's Table 1 beside near-parallel items on the clinical autism screeners AQ and RAADS-R; the statement wordings are the charging party's recollections and precise wording may differ. Aon's own interpretation guides flag extreme-end scorers as potential problems for employers. Meta-analyses cited in the complaint show autistic people scoring significantly lower on working-memory measures of the kind gridChallenge uses. And the design record is the load-bearing part: across a 166-page ADEPT-15 technical documentation, the only disability-bias measure the complaint identifies is an outside-attorney review of item wording that flagged physical-disability language such as 'seeing', 'speaking' and 'hearing' — with no clinical or neurodiversity sensitivity review and no study on disabled populations anywhere in the programme. What the complaint alleges is a high RISK of screening out autistic and disabled applicants, together with one charging party's individual account. No adjudicated or statistically demonstrated screen-out of disabled applicants exists in this record. In August 2025 Bloomberg Law reported the EEOC charge as filed and unresolved and recorded that an Aon spokesperson declined to comment.

Sources: americancivillibertiesunionf2024, bloomberglaw2025

Appears on: /domains/cases/aon-assessment-suite

Two federal forums were opened on one product suite and neither has publicly moved. In late 2023 the ACLU, with co-counsel Winston Cooks, LLC, filed class-wide charges of discrimination with the Equal Employment Opportunity Commission under the Americans with Disabilities Act and Title VII against Aon AND a client employer, on behalf of a biracial Black and white autistic job applicant with mental-health disabilities; the employer is publicly identified only as a mid-sized company headquartered in the United States. Those charges cover ADEPT-15 and gridChallenge. On 30 May 2024 the ACLU filed a complaint and request for investigation with the Federal Trade Commission against Aon Consulting, Inc., alleging that marketing the assessments as bias-free, fair, having no adverse impact and able to improve diversity is a deceptive act or practice under Section 5 of the FTC Act, and that selling assessments alleged to discriminate while failing to assess disability harms is an unfair practice injuring both workers and employers; that complaint covers all three products including vidAssess-AI. The relief requested asks the Commission to open an investigation, enjoin the claims, require truthful risk information, and require Aon to pause sale or administration of the assessments until discrimination is eliminated or discontinue products where it cannot be. Management-side analysis of the filings records the procedural register: an advocacy complaint to the Commission initiates agency review and triggers no automatic enforcement, and an EEOC charge is confidential while pending and is commonly a precursor to private litigation. The enforcement environment contracted during the pendency: in January 2025 the EEOC removed its artificial-intelligence hiring technical-assistance guidance following the administration change, and an executive order of 23 April 2025 directed federal agencies to deprioritize disparate-impact liability enforcement, the theory underlying the race claims; legal-press analysis in March 2025 listed the Aon matters among pending ones with no indication of closure. As of 28 August 2026 no public FTC action, no public EEOC determination, settlement or follow-on lawsuit, and no product withdrawal has been located. Because charge proceedings are confidential and the Commission never docketed a public matter, that silence establishes nothing in either direction: it means no PUBLIC action, and an unannounced staff inquiry can be neither ruled in nor ruled out.

Sources: americancivillibertiesunion2024, americancivillibertiesunionf2024, fisherphillips2024, bloomberglaw2025a

Appears on: /domains/cases/aon-assessment-suite

The contested link in this record is between two sets of documents the same company wrote. The ACLU's complaint quotes Aon's marketing as describing the assessments as 'fair[] for all: no adverse impact', 'scientifically proven not to have bias', 'culturally-agnostic', and as excluding prejudices, and alleges that selling on those sentences while Aon's own technical documentation reported the group score differences above is a deceptive act under Section 5. Aon responded through trade press in June 2024 that its assessment solutions 'follow industry best practices as well as the Equal Employment Opportunity Commission, legal and professional guidelines' and are 'intended to be fair to everyone'; that is a vendor statement and no adjudication of any of it exists. The record contains one documented act by Aon after the EEOC charges were filed, and it went to evidence rather than to claims: the public link to the assessment technical documentation on Aon's Norwegian support portal was disabled, and the ACLU states it knows of no other public source and cites archived copies. The marketing did not change. When Aon's pre-hire talent assessment page was fetched on 28 August 2026 it advertised 'Fair and unbiased assessments' and asserted that 'Assessment tools that are scientifically proven not to have bias are essential' — twenty-seven months after those claims were formally alleged to be deceptive, with all three products still on sale. That page is recorded as a vendor claim, its content can change, and the fetch date is the citation of record.

Sources: americancivillibertiesunionf2024, aonplc2026, hrdive2024

Appears on: /domains/cases/aon-assessment-suite

Two documented properties of this deployment decide what an assessed person can do about a score. First, the position of the instrument: the assessments are marketed to screen out applicants prior to more resource-intensive hurdles, and for the population screened out there is no human review at all — the assessment is the first and the final decision point, and Aon's own case-study marketing is cited for one client that screened out 62 percent of applicants on the assessment alone. Second, what the candidate is told before responding: Aon's test-taker guide states that 'there is no possibility of bias — for or against specific candidates — either during the test or during the evaluation of answers', and discusses accommodations only for physical disabilities. The ACLU argues that a candidate who reads that does not make the accommodation request that would generate the only disability-relevant signal anywhere in the system, and that the absence of the signal then reads as evidence that nothing was needed; invoking the request costs the candidate a disclosure of disability. The quoted text is Aon's and the alleged suppressing effect is the ACLU's. The employer, not Aon, holds the decision that converts a score into an outcome: which constructs are used, where the cut-off falls, and what the assessment is combined with. No cut-off or vetting practice for any named client appears in this record, and the client employer named as an EEOC co-respondent is publicly unnamed.

Sources: americancivillibertiesunionf2024, americancivillibertiesunion2024

Appears on: /domains/cases/aon-assessment-suite

Checkr, Inc. is a consumer reporting agency founded in 2014 whose automated background-check platform sits one company upstream of the gig platforms that buy from it: it retrieves criminal items from county, state and national sources, matches them to a named applicant, normalises the charge data, applies each platform customer's own eligibility matrix, and furnishes a report, after which the platform executes its own access decision. Every scale, speed and automation figure for it is the vendor's own marketing, fetched from its product pages on 28 August 2026 and independently audited by nobody: 90 percent of gig economy background checks run on Checkr; 140,000-plus businesses already run on Checkr; 260 million-plus real identities mapped, covering 96 percent of US adults; instant criminal checks that screen users in under a second; 89 percent of national criminal record checks complete within one hour; Checkr AI normalises criminal charge data; automated adjudication tools reduce manual workflows by up to 80 percent; and continuous checks are offered as a standing-monitoring product. Those claims are self-serving in both directions — they are capability marketing and they are also the clearest available statement of how unmanned the generation path is. Lyft, Hyer and GigSmart are named as gig clients on the vendor's own pages; the Uber relationship is documented litigation-side, where a defence-bar analysis records Uber moving its New York City driver checks to Checkr in mid-2017.

Sources: checkr2026, huntonandrewskurthllp2021

Appears on: /domains/cases/checkr-gig-background-screening

The federal consumer regulator wrote down what the Fair Credit Reporting Act requires of automated background screeners, and its statements are about the industry rather than about any one company. In an advisory opinion of 4 November 2021 (Fair Credit Reporting; Name-Only Matching Procedures, 86 FR 62468) the Consumer Financial Protection Bureau held that name-only matching does not assure maximum possible accuracy under 15 U.S.C. 1681e(b), stated that when background screening companies and their algorithms carelessly assign a false identity to applicants for jobs and housing they are breaking the law, warned that the risk of mistaken identities from name-only matching is likely to be greater among Hispanic, Black and Asian communities because there is less surname diversity in those populations, and observed that because of the sheer scale of background screening activity even ostensibly low error rates can harm significant numbers of consumers. In two companion advisory opinions of 11 January 2024 (Background Screening, 89 FR 4171; File Disclosure, 89 FR 4167) it stated that screeners must prevent the reporting of expunged, sealed or legally restricted records, ensure dispositions accompany any reported arrest or charge, prevent duplicative reporting and honour per-item obsolescence windows such as the seven-year bar on non-conviction arrests, and that a consumer's file disclosure must include both the originating sources and any intermediary or vendor sources. An independent legal-bar reading of the January 2024 opinions records the same four accuracy-procedure requirements and the same file-disclosure source rule. No error rate, accuracy figure or per-cohort disparity measurement for Checkr's pipeline has ever been published by anyone.

Sources: consumerfinancialprotectionb2021, consumerfinancialprotectionb2024a, ballardspahrllp2024

Appears on: /domains/cases/checkr-gig-background-screening

The correction channel that reaches this deployment is statutory rather than discretionary, and the record documents it being outrun. Under 15 U.S.C. 1681i a consumer reporting agency must reinvestigate a disputed item and correct the file, within 30 days and 45 with an extension; 15 U.S.C. 1681k imposes a currency-or-contemporaneous-notice duty on public-record items reported for employment purposes; and 15 U.S.C. 1681b(b)(1) bars furnishing an employment-purpose report at all unless the user certifies compliance with the notice and pre-adverse-action duties. Because the report is assembled and held at one upstream agency, a dispute that succeeds corrects the record for every platform that reads it. Against that clock, the 2021 class complaint pleads that Checkr furnished Uber the report and that Uber deactivated Job Golightly one day later without any notice, process or communication. The same complaint records the recurrence conditions: approximately 80,000 TLC-licensed rideshare vehicles in New York City, the majority on Uber, and an Uber policy of re-checking current drivers at least every two years through Checkr. The vendor separately markets continuous checks, a standing-monitoring product that re-furnishes on new record events. No dispute volume, reinvestigation resolution rate or reversal rate for this vendor appears in any public source.

Sources: classactioncomplaint2021, consumerfinancialprotectionb2021, checkr2026

Appears on: /domains/cases/checkr-gig-background-screening

On 12 May 2025 the Consumer Financial Protection Bureau withdrew all four of the Fair Credit Reporting Act advisory opinions covering this deployment class — Name-Only Matching (86 FR 62468), Permissible Purposes (87 FR 41243), Background Screening (89 FR 4171) and File Disclosure (89 FR 4167) — inside a mass withdrawal of 67 guidance documents, confirmed on the Bureau's own withdrawn-guidance list. The statutes those opinions interpreted were untouched: 15 U.S.C. 1681e(b), 1681i, 1681k and 1681b(b) all remain in force and privately enforceable, and courts may still reach the same results. What was withdrawn is the regulator's stated reading of them, not the duties themselves, and no source in this record describes the accuracy duty as repealed or ended. The withdrawal is a documented contraction of the interpretive layer over a deployment class that continued operating unchanged.

Sources: consumerfinancialprotectionb2025, consumerfinancialprotectionb2024a

Appears on: /domains/cases/checkr-gig-background-screening

Two class vehicles have reached this pipeline and neither has been adjudicated on the merits. Golightly v. Uber Technologies, Inc. and Checkr, Inc., No. 1:21-cv-03005 (S.D.N.Y., filed 8 April 2021), pleads five counts of which four are against Uber — the New York City Fair Chance Act, disparate racial impact, and the federal and New York consumer reporting statutes — and one is against Checkr: furnishing employment-purpose reports without obtaining the user's compliance certification under 15 U.S.C. 1681b(b)(1). It alleges that Job Golightly, a Black Bronx driver who had driven for Uber since 2014 averaging about $1,500 a week, was deactivated on or around 28 August 2020 one day after Checkr furnished his report, over a single 2013 Virginia speeding offence at 22 miles per hour over the limit — a misdemeanour under Virginia law and a civil infraction under New York law — with no pre-adverse-action notice, no copy of the report, no Article 23-A individualised analysis and no three-business-day window, and that he learned the reason months later. No misreporting by Checkr is pleaded in that case; the record was accurate as pleaded and the failures alleged are process failures. The complaint estimates classes of at least 1,000 and at least 300 since 11 January 2020, and records, citing research from The New School's Center for New York City Affairs, that 87 percent of New York City transportation independent contractors are persons of colour, 81 percent lack a college degree and 90 percent are foreign-born. On 21 December 2022 Judge Lewis J. Liman granted the motion to compel individual arbitration and stayed the claims; the order itself could not be read from the environment that verified this record, its specific treatment of the claims against Checkr is not asserted, and no public resolution has been located as of August 2026. Davis v. Checkr, Inc., No. 0:26-cv-60088 (S.D. Fla., filed 26 January 2026), pleads a different mechanism: a putative nationwide class under 15 U.S.C. 1681e(b) alleging that a Checkr report included criminal case records plainly unassociated with the consumer and that the company knowingly and willfully maintains deficient procedures because reporting more information is more profitable, covering consumers who within two years received a Checkr report that incorrectly included criminal case records belonging to someone else. It is pending and pre-certification. All of the above are allegations.

Sources: classactioncomplaint2021, mobilizationforjustice2021, huntonandrewskurthllp2021, opinionandorder2022, topclassactions2026

Appears on: /domains/cases/checkr-gig-background-screening

Federal enforcement in neighbouring screening markets sets the posture the Fair Credit Reporting Act establishes for this class of automation, and none of it is an action against Checkr. On 12 October 2023 the Federal Trade Commission and the Consumer Financial Protection Bureau settled with TransUnion's rental-screening arm for $15 million — $11 million in redress and a $4 million penalty — over duplicate eviction entries, misreported outcomes, sealed records that were not removed, and undisclosed third-party sources. On 11 September 2023 the Commission took $5.8 million from TruthFinder and Instant Checkmate over background reports marketed as most accurate that were assembled from sources disclaiming accuracy, with a Flag as Inaccurate button that triggered no investigation; the Commission's position there was that report assemblers marketing for employment and tenant screening are consumer reporting agencies bound by the maximum-possible-accuracy and permissible-purpose duties. No public Commission or Bureau enforcement action against Checkr itself was located as of 28 August 2026. The one completed money resolution in this record is platform-side: Aguilera v. Uber Technologies, Inc. d/b/a Uber Eats, No. 509275/2023 (N.Y. Sup. Ct., Kings County), settled for $3.35 million over allegations that Uber Eats used a flawed criminal background check process that unfairly denied individuals access to the platform and failed to provide the legally required disclosures, covering New York City couriers denied between 24 October 2015 and 28 July 2021, split into serious-conviction and less-job-related groups, with automatic payments and a final approval hearing on 8 February 2024. Checkr was not a defendant there and no admission of liability was located.

Sources: ftccfpb2023, federaltradecommission2023, topclassactions2023

Appears on: /domains/cases/checkr-gig-background-screening

Around January 2021 a Milton, Massachusetts resident applied for a CVS supply chain position, sat a HireVue video interview and was not hired. As recited in the court's published opinion from the amended complaint, the interview asked integrity-framed questions — what integrity means to the applicant, and a time the applicant acted with integrity — and HireVue uploaded the recordings to Affectiva, a Boston affect-analysis firm spun out of the MIT Media Lab, whose AI 'analyzes candidates' facial expressions, eye contact, voice intonation, and inflection' to draw conclusions about the applicant's degree of cultural fit; HireVue then provided CVS with employability scores. The amended complaint lists the affect features as smiles, surprise, contempt, disgust and smirks, and describes the score as rating traits including conscientiousness and responsibility and an innate sense of integrity and honor. Every mechanical element of that description is an allegation accepted as true for the purpose of a motion to dismiss; none of it was ever adjudicated, and no error rate, accuracy figure or independent evaluation of this screen exists from any source.

Sources: bakerv2024, hrdivecrist2024, thebostonglobejohnston2023

Appears on: /domains/cases/cvs-hirevue-integrity-screen

Massachusetts General Laws chapter 149, section 19B has banned lie detector tests in employment since 1959, and a 1985 amendment added both a private civil action and a mandatory notice. The statute defines the banned instrument by PURPORTED function: it covers 'any test utilizing a polygraph or any other device, mechanism, instrument or written examination' used 'for the purpose of purporting to assist in or enable the detection of deception, the verification of truthfulness, or the rendering of a diagnostic opinion regarding the honesty of an individual.' Every Massachusetts employment application must carry, in clearly legible print, one sentence: 'It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.' Remedies run to a civil action within three years by any person aggrieved, not less than 500 dollars in damages per violation, treble damages for lost wages or benefits, costs and reasonable attorney fees, with criminal penalties of 300 to 1,000 dollars for a first violation and up to 1,500 dollars and 90 days thereafter. The notice provision then went roughly forty years without being enforced, until filings began in 2023. The 500-dollar figure is a statutory minimum, not a payment made to anyone in this case.

Sources: massachusettsgenerallaws1959, agencychecklists2025, morganlewislawflashengelman2025

Appears on: /domains/cases/cvs-hirevue-integrity-screen

On 16 February 2024 Judge Patti B. Saris denied both of CVS's motions in their entirety in Baker v. CVS Health Corporation, No. 1:23-cv-11483 (D. Mass.) — the failure-to-state-a-claim motion aimed at the notice count and the separate Article III standing motion. No count and no defendant was dismissed. The court held that the statute's private right of action for any person aggrieved reaches notice violations, and that denial of information to which a plaintiff has a legal right can be a concrete injury in fact: the required notice 'would have specifically informed Baker that the HireVue Interview was a lie detector test,' and he participated without that warning. CVS did not challenge the sufficiency of the counts alleging that the screen itself violated the lie-detector prohibition, so the court accepted that characterization as plausibly pleaded rather than deciding it. This is a pleading-stage holding. There is no merits finding that the HireVue interview was a lie detector test, and no finding of any kind about the screen's accuracy.

Sources: bakerv2024, arentfoxschiffdavidson2024

Appears on: /domains/cases/cvs-hirevue-integrity-screen

Liability in this case landed on the employer while the two companies that built and ran the contested inference sat outside it. The defendants were CVS Health Corporation and CVS Pharmacy, Inc. only; HireVue and Affectiva were never parties and no claim was ever adjudicated against either. On the pleaded account the inference executed two organizational hops from the party bearing the statutory duty: the recordings were captured on the interview platform, uploaded onward to the affect-analysis firm, composed into an employability score at the platform, and returned to CVS recruiters. The statutory duties of section 19B bind the employer, not the vendors. The applicant-facing surface was the recorded interview alone: the complaint pleads three deprivations — no notice, no opt-out, and no ability to challenge the assessment results — and the record documents no applicant access to the recording, the affect analysis or the score, and no feedback of hiring outcomes into retraining. Because neither vendor's internal pipeline was ever put in evidence, the court's recitation of the pleaded mechanics is the authoritative public account of how this system worked.

Sources: bakerv, bakerv2024, hrdivecrist2024

Appears on: /domains/cases/cvs-hirevue-integrity-screen

The case ended without deciding anything about the technology. A settlement notice was filed 17 July 2024; the public docket shows the case terminated 22 July 2024; a stipulation of voluntary dismissal with prejudice followed on 20 September 2024. The settlement was individual and confidential and was reached before any class-certification ruling: no monetary terms, no practice changes and no admission of liability were disclosed, and no class member received anything. The named claim was extinguished and the pleaded class went unrepresented. Both docket dates are carried here rather than a single 'settled' date. The mirrored opinion does not recite an original filing court or removal path, so the filing history is stated only as reported by the Boston Globe on 22 May 2023 and on the federal docket from 30 June 2023.

Sources: hrdivecrist2024, bakerv, thebostonglobejohnston2023

Appears on: /domains/cases/cvs-hirevue-integrity-screen

After the February 2024 ruling, litigation under section 19B surged: more than twenty class actions in roughly a year, largely by a single New York firm, often with the same individuals suing multiple employers, including Procter & Gamble. Most allege ONLY that an employment application lacked the statutory notice, with no actual screening device involved — so what propagated was the notice theory rather than any finding about AI screening. Compliance advisories now direct every Massachusetts employer to print the statutory notice in the application itself rather than in a policy filed elsewhere, and illustrate the exposure arithmetically: an employer processing 200 Massachusetts applications a year faces roughly 100,000 dollars of annual notice-violation exposure at the 500-dollar statutory minimum. That figure is an illustration of the statutory structure, not a measured exposure for CVS or any other employer, and no CVS Massachusetts application volume or class size appears in the verified record. The system-level correction now visible — notice lines appearing on Massachusetts employment applications — is driven by per-application statutory damages and copycat litigation pressure rather than by any adjudicated finding about the technology.

Sources: morganlewislawflashengelman2025, agencychecklists2025

Appears on: /domains/cases/cvs-hirevue-integrity-screen

Two things in this record cut against the pleaded account and both are carried rather than resolved. HireVue removed facial analysis from new assessments in March 2020 and announced the change in January 2021 alongside a commissioned algorithmic audit, retaining speech-and-language analysis; it told the Boston Globe that visual and audio analysis 'have since been eliminated,' and its chief data scientist rejected the deception-detection characterization, saying the assessments measure work competencies 'statistically linked' to job success using 'validated industrial organizational psychology.' The application in this case was made around January 2021, and whether Affectiva affect analysis in fact ran on it, or on any class member's interview, was never adjudicated. Separately, the scientific question the 1959 statute was answering was never reached either: Brandeis psychologist Leonard Saxe told the Globe that there is no neurological signal of deception and 'no way for an automated system to distinguish a falsehood from the truth.' That is an expert view recorded in a newspaper, not a finding, and it is a statement about the construct rather than a measurement of this or any deployment.

Sources: maurer2021b, thebostonglobejohnston2023

Appears on: /domains/cases/cvs-hirevue-integrity-screen

HireVue's video-assessment platform is a vendor-layer deployment: one scoring engine behind hundreds of separate employers' hiring pipelines. Candidates record answers to a structured question set, or play game-based assessments; per-assessment models score verbal and paraverbal features of the responses into competency scores that place each applicant in a Bottom, Middle or Top tier, and the client employer chooses where to cut — one deploying employer's published bias audit records it evaluating at Top-plus-Middle against Bottom. The client also chooses whether to use algorithmic scoring at all: as of January 2021 the vendor reported that approximately 20 percent of its customers used the predictive-analytics feature and that the rest used the platform for human review of recorded video. Every scale figure is vendor-reported and none is independently audited: more than 19 million video interviews and more than 700 customers as of January 2021, and more than 33 million interviews, 200 million chat-based candidate engagements and more than 800 customers as of January 2023. The models are built on historical applicant data pooled across employer implementations, and the mandated bias audits are computed on that same pooled store. A rejected candidate generates no outcome data anywhere in that loop, so the store that trains and audits the models cannot observe the population the models screened out.

Sources: fortune2021, maurer2021b, dciconsultinggroup2023, hirevue2021

Appears on: /domains/cases/hirevue-video-assessment

The vendor removed visual and facial analysis from new assessment models in early 2020 — the Society for Human Resource Management reports the discontinuation as March 2020 — and announced the decision publicly on 12 January 2021, together with the results of an algorithmic audit it had commissioned from O'Neil Risk Consulting and Algorithmic Auditing. Its stated reason was that advances in language analysis had left visual features contributing little: internal research put the nonverbal visual contribution at about 0.25 percent of the model's predictive power in most job models and about 4 percent for high-customer-contact roles, figures given by the vendor's chief data scientist and never independently audited. Its chief executive said it was not worth the concern it was causing people. The sequence is complaint in November 2019, removal in about March 2020, public announcement with the audit in January 2021; causation by the complaint is an inference the record does not make, and the vendor's own stated reasons were the low measured contribution and rising public concern. The removal is a rare documented instance of an input dropped as its measured value approached zero while the cost of scrutiny rose, and it was applied platform-wide by a party no single client employer could have compelled.

Sources: maurer2021b, fortune2021, hirevue2021, schellmann2021

Appears on: /domains/cases/hirevue-video-assessment

Two audit regimes reach this deployment and their perimeters are documented. The 2020 commissioned audit, announced 12 January 2021, examined one representative pre-built early-career assessment use case, did not examine the tool's technical design or its training data, proceeded largely by structured stakeholder interviews, and its report is published on the vendor's own site only behind a nondisclosure agreement; the phrase that the assessments work as advertised with regard to fairness and bias is the vendor's characterization of exactly that scope, and the recorded criticisms are that its depth compared unfavourably with a contemporaneous source-code audit of a competitor, that an auditor paid by the audited party carries a conflict risk, and that neither audit addressed whether the products improve hiring at all. The audit did produce recommendations, on investigating accent bias and on the flagging of candidates who give brief answers. From January 2023 the vendor engaged DCI Consulting Group for the annual bias audits New York City Local Law 144 requires, covering competency-based and game-based algorithms across race, gender and intersectional groups. A summary produced 5 July 2023 and published by Pfizer as a deploying employer reports the mechanics: nationwide applicant data from January 2021 to December 2022, pooled across employer implementations, analysed per implementation and then aggregated; on the Communication assessment for intern and new-college-graduate jobs, 20,060 male against 9,121 female applicants, selection rates of 0.66 and 0.65 for Top-plus-Middle against Bottom, a gender impact ratio of 0.98, race and ethnicity ratios from 0.87 to 0.96 and intersectional ratios down to 0.82; on the Adaptability assessment for the same population, 7,161 male against 3,884 female applicants, a gender ratio of 0.95 and intersectional ratios from 0.79 to 1.01. A peer-reviewed study of all 116 publicly available Local Law 144 bias audits published between July 2023 and November 2024 found that DCI conducted 20 percent of them, that 54 percent of audits carried at least one impact ratio above 1 — the majority of those being DCI audits of HireVue tools deployed by JetBlue, Citizens, Pfizer or Burlington, an artifact of the aggregation and comparator-group method — and that every identified 'silent duplicate', meaning identical quantitative results republished across or within reports, appeared in DCI-conducted audits, all but one describing HireVue tools. Its authors could not determine the cause of all the duplicates. These are observations about a measurement regime; they are not findings of discrimination and not findings of audit fraud. An impact ratio is a ratio of selection rates over the applicants an audit could classify, and it is not an error rate — no error rate for this engine has ever been published by anyone.

Sources: schellmann2021, maurer2021a, dciconsultinggroup2023, gerchick2025

Appears on: /domains/cases/hirevue-video-assessment

Three external channels reached the vendor and produced three different kinds of result. On 6 November 2019 the Electronic Privacy Information Center filed a complaint with the Federal Trade Commission alleging that the AI-based candidate assessments constituted unfair and deceptive practices under Section 5, that the company falsely denied using facial recognition, and that its results were biased, unprovable and not replicable; no public FTC enforcement action against the company is on the record as of August 2026, so what the record supports is that the vendor was the subject of an advocacy complaint and nothing stronger. Six Illinois residents filed Deyerler v. HireVue, Inc. on 27 January 2022 in the Northern District of Illinois (No. 1:22-cv-01284), alleging that the software collected facial geometry and voice data during virtual job interviews without the disclosures and written consent the Illinois Biometric Information Privacy Act requires; on 26 February 2024 Judge Jeremy C. Daniel granted in part and denied in part the motion to dismiss, letting claims under sections 15(a), (b) and (d) proceed, dismissing the section 15(c) profit claim on the ground that selling software is not selling biometric identifiers, and rejecting the argument that the Illinois Artificial Intelligence Video Interview Act precludes BIPA claims — holding the two statutes impose different but concurrent obligations. A claim that survives a motion to dismiss is an allegation held plausible and never a finding of violation. On 25 June 2026 the Circuit Court of Lake County, Illinois (No. 2026LA00000141, Hon. Daniel L. Jasica) granted PRELIMINARY approval of a 3,750,000 dollar class settlement covering an estimated 91,305 people who completed an interview involving the challenged voice and facial biometrics technology while in Illinois between 27 January 2017 and 25 June 2026, with an estimated 150 dollars per valid claimant subject to pro rata reduction, a claims deadline of 13 October 2026 and a final approval hearing on 28 October 2026. Nothing has been paid, the class was conditionally certified for settlement purposes only, the settlement is expressly no admission of wrongdoing, and the vendor denies that it collected or possessed biometrics or any other information subject to BIPA. On 19 March 2025 the ACLU of Colorado filed charges with the Colorado Civil Rights Division and the EEOC on behalf of a Deaf, Indigenous employee, alleging that an automated video interview relying on automated speech recognition disadvantaged her and that a request for human-generated captioning was denied; both companies dispute the charges and the vendor's chief executive called the complaint entirely without merit and stated that the employer did not use one of its AI-based assessments. No court and no regulator has ever found this vendor violated the biometric statute, the FTC Act, or any discrimination law.

Sources: electronicprivacyinformation2019, memorandumorder2024, noticeofproposedclassactions2026, aclu2025a

Appears on: /domains/cases/hirevue-video-assessment

Both statutes that reach this deployment place their duties on EMPLOYERS rather than on the vendor, and that placement is the structural finding of the case. The Illinois Artificial Intelligence Video Interview Act (820 ILCS 42, effective 1 January 2020) requires employers using AI analysis of video interviews to notify applicants before the interview, explain how the AI works and what general types of characteristics it uses to evaluate them, and obtain consent to be evaluated; it restricts sharing of the video, requires deletion within 30 days of an applicant's request, and from 2022 requires employers relying solely on AI analysis to report applicant race and ethnicity data annually. Its text states no express enforcement mechanism and no private right of action — which is why the Deyerler plaintiffs sued under the biometric statute instead, and why the February 2024 holding that the video-interview statute does not displace the biometric one mattered: it kept open the only channel in this record with teeth. New York City Local Law 144 likewise places its annual independent-bias-audit and publication duty on employers. The vendor holds the pooled data and engages the auditor; the employers hold the duty and publish the result — which is how four named employers came to publish the same vendor-level numbers as their own audits. The vendor commissioned the audits voluntarily and argued publicly, in announcing the January 2023 engagement, that vendors should also bear audit responsibility. Nothing in this record describes either statute as regulating the vendor directly.

Sources: artificialintelligencevideoi2020, memorandumorder2024, gerchick2025, hirevue2021

Appears on: /domains/cases/hirevue-video-assessment

Independent scholarship bounds what automated video assessment can validly claim, and it measures the construct class rather than any one product. Hickman and colleagues, in the Journal of Applied Psychology in 2022, investigated automated video-interview personality assessments across a development sample of 1,073 and a retest sample of 99: models trained on self-reports showed little evidence of reliability or validity, while models trained on interviewer reports performed better but with mixed cross-sample reliability, and the authors cautioned vendors and adopting organizations accordingly. Raghavan and colleagues, at ACM FAT* in 2020, analysed what algorithmic pre-employment vendors publicly claim about validation and bias mitigation and found those claims largely unverifiable from what vendors disclose. Against that, a technology-press review of both audit regimes reaching this deployment recorded that neither addressed whether the products improve hiring at all. So the record contains no measurement of this engine's accuracy from any source: the mandated audits report selection-rate ratios, the commissioned audit did not examine the technical design or the training data, and no error rate has ever been published.

Sources: hickman2022, raghavan2020a, schellmann2021

Appears on: /domains/cases/hirevue-video-assessment

On 19 March 2025 the ACLU, the ACLU of Colorado, Public Justice and Eisenberg & Baum LLP filed a Complaint of Discrimination with the Colorado Civil Rights Division and the U.S. Equal Employment Opportunity Commission on behalf of D.K., a Deaf Pawnee woman who communicates in English with a deaf accent and in ASL, against BOTH Intuit, Inc. and HireVue, Inc. It alleges violations of the Colorado Anti-Discrimination Act (Colo. Rev. Stat. § 24-34-402), the Americans with Disabilities Act and Title VII in the denial of her promotion to Seasonal Manager, and pleads HireVue as an employment agency, an agent of the employer, an indirect employer and an aider and abettor under state law. None of those theories has been tested. Everything the charge asserts is an allegation and no probable-cause determination, dismissal, right-to-sue notice, court filing or settlement has been made public.

Sources: complaintofdiscriminationaga2025, aclu2025a, publicjustice2025

Appears on: /domains/cases/intuit-hirevue-promotion-screen

This is an internal-promotion case rather than a point-of-hire case, and the employer held years of direct performance evidence about the person the screen was assessing. Per the complaint and D.K.'s own published account she was hired as a seasonal Tax Associate in late 2019, became a Tax Expert Lead supporting a team of about four hundred Tax Associates, held that role for three seasons, earned a bonus every year, was praised by supervisors for her communication, joined Intuit's Accessibility Team in 2023, and was encouraged to apply for Seasonal Manager by her own manager, who sat on the hiring team. The promotion ladder ran Tax Associate to Tax Expert Lead to Seasonal Manager, with Tax Expert Leads trained for the next step and, in the complaint's words, many being promoted each year. She applied in spring 2024 through the employee dashboard and was invited on 21 June 2024 into an assessment of roughly three hours: about a dozen timed recorded video questions, mostly management scenarios with a few tax-law questions, plus essay and multiple-choice sections, with instructions and questions delivered audibly by a recording of a person speaking.

Sources: complaintofdiscriminationaga2025, acluofoklahoma2025

Appears on: /domains/cases/intuit-hirevue-promotion-screen

The accommodation sequence as alleged: the interview invitation offered only a technical-support email address and carried no accommodations information, so D.K. initiated a request herself from prior familiarity with Intuit's process. She asked for Communication Access Real-time Translation (CART) — human-generated real-time captioning — deliberately not ASL interpretation, because interpreters she had worked with previously could not handle tax concepts, and she did not ask to be excused from the video interview because it was never offered and she feared that asking would count against her. The request was denied and she was told the platform's built-in subtitles could be enabled. When she began the assessment, she recounts, no subtitle option existed, and she completed roughly three hours of it relying on her browser's automatic captions, which she describes as less accurate and particularly poor on complicated words. The alleged failure therefore includes a control that was pointed to and was not present, rather than only a degraded substitute channel. Intuit's on-record position is that the allegations are entirely without merit and that it provides reasonable accommodations to all candidates.

Sources: complaintofdiscriminationaga2025, acluofoklahoma2025, aclu2025b

Appears on: /domains/cases/intuit-hirevue-promotion-screen

The complaint alleges Intuit was on notice twice over, through two channels of its own. First, after the 2023 season D.K. — by then a member of Intuit's Accessibility Team — told the Team's Chair that the HireVue step was inaccessible and could exclude deaf applicants; she was told it would be looked into, no action was ever reported back, and Intuit required the same step of her the following year. Second, in 2020 Intuit's OWN automated call-monitoring system, which scored how closely agents followed scripts from speech-recognition transcripts of customer calls, allegedly read her deaf accent as deviation from script and produced one artificially low indicator against otherwise strong customer-satisfaction, resolution-count and response-speed measures. Intuit's alleged response was to reassign her from phone calls to the chat channel rather than to correct the metric; the low indicator remained in her record, and the complaint pleads it may also have been considered in the denial of her promotion. That call-monitoring system is Intuit's, not HireVue's, and the charge uses it as evidence of awareness rather than as the promotion screen.

Sources: complaintofdiscriminationaga2025, acluofoklahoma2025

Appears on: /domains/cases/intuit-hirevue-promotion-screen

On 13 August 2024 D.K. received an automated rejection stating that Intuit had decided to move forward with other candidates. On 25 September 2024 she received a feedback email — which she believed from its return address and generic language to have been generated by HireVue's automated analysis — advising her to give more concise and direct answers, to adapt her communication style to different audiences, and to practise active listening; the complaint says those recommendations map directly onto her being Deaf. On 31 October 2024 she made a personnel-file request under Colorado law, and the complaint filed the following March alleges that only minimal documents were produced. The complaint pleads HireVue's platform mechanism in a three-stage form — speech recognition of the spoken answers, a second system interpreting the transcript, and machine-learning scoring against job competencies — but pleads the scoring allegations about her specific interview on information and belief, inferred from that feedback email. That inference is exactly what HireVue disputes.

Sources: complaintofdiscriminationaga2025, proskauerrosellp2025, aclu2025a

Appears on: /domains/cases/intuit-hirevue-promotion-screen

The technical premise the charge builds on is a published measurement of general-purpose speech recognition, and it belongs to that study rather than to any deployment. A 2025 paper in The Laryngoscope (135:191-197) measured four widely used commercial speech-recognition services against a twenty-four-speaker corpus and reported a mean word error rate of 52.6 percent for d/Deaf and hard-of-hearing speakers versus 5.0 percent for normal-hearing speakers — per service 45.1 to 57.3 percent against 3.8 to 5.9 percent, which the authors describe as ten times higher — rising to 85.9 percent for speakers with the lowest speech-intelligibility classification, 80.5 percent for prelingual hearing-loss onset and 70.2 percent for sign-primary communicators. The complaint cites that study, together with peer-reviewed findings of higher speech-recognition error rates for Black speakers and for ethnicity-related dialects, and extends them on information and belief to speakers of Indigenous dialects of English. The study measured four general-purpose services. It did not measure HireVue's system or this interview, and the inference from that literature to this platform is contested.

Sources: zhao2025a, complaintofdiscriminationaga2025

Appears on: /domains/cases/intuit-hirevue-promotion-screen

Both respondents dispute the charges, and the vendor's dispute goes to the factual premise rather than to the legal conclusions. HireVue's chief executive Jeremy Friedman stated that the complaint 'is entirely without merit and is based on an inaccurate assumption about the technology used in the interview. Intuit did not use a HireVue AI-based assessment.' An Intuit spokesperson stated: 'The allegations in the complaint are entirely without merit. We provide reasonable accommodations to all candidates.' Whether any automated assessment ran on this interview at all is therefore in dispute, and the candidate-side evidence bearing on it is the return address and register of one feedback email. As of 28 August 2026 the matter remained a pending, non-public administrative investigation: no Colorado Civil Rights Division or EEOC determination, no right-to-sue notice, no court complaint and no settlement had been publicly reported, and an April 2026 legal-commentary treatment of the case records no post-filing developments. Administrative charge proceedings are confidential, so that silence establishes no public development rather than that nothing occurred.

Sources: aclu2025a, undergraduatelawreviewatflor2026, aclu2025b

Appears on: /domains/cases/intuit-hirevue-promotion-screen

The U.S. Equal Employment Opportunity Commission alleged that iTutorGroup, Inc., Shanghai Ping'An Intelligent Education Technology Co., Ltd. and Tutor Group Limited — three integrated companies providing English-language tutoring to students in China through United-States-based tutors working fully remotely from their homes — had programmed their tutor application software to automatically reject female applicants aged 55 or older and male applicants aged 60 or older, rejecting more than 200 qualified United-States applicants because of their age. After conciliation failed the Commission sued under the Age Discrimination in Employment Act on 5 May 2022, No. 1:22-cv-02565 (E.D.N.Y.); then-Chair Charlotte Burrows framed the filing for the agency's algorithmic-enforcement agenda with the words 'Even when technology automates the discrimination, the employer is still responsible.' The parties filed a joint notice of settlement with a proposed consent decree on 9 August 2023 and the court approved the decree in September 2023, announced by the Commission on 11 September 2023. iTutorGroup pays $365,000 to be distributed among the more than 200 rejected applicants through a claims process, without admitting liability; per-claimant amounts were not made public. iTutorGroup denied the allegations and disputed that the tutors were employees at all, characterizing them as independent contractors, a question the settlement resolved without adjudication.

Sources: u2023a, u2022c, hrdive2023

Appears on: /domains/cases/itutorgroup-age-screening

The screen at issue was an authored decision boundary rather than anything learned, and the distinction is the case's doctrinal value. The EEOC's own releases describe 'tutor application software' that was 'programmed to automatically reject female applicants aged 55 or older and male applicants aged 60 or older' — a deterministic, sex-differentiated threshold computed from the birthdate field collected on the application form, with no score, no training data and no published error rate anywhere in the record. It sat at the top of the hiring funnel, upstream of any human reviewer: rejection was automatic at application intake, no human review point between the rule and the rejection notice is documented, and the employer's own hiring staff saw only applicants who came through the screen. Rejected applicants received no statement of the reason. The settlement was widely described in the legal press as the EEOC's first workplace artificial-intelligence settlement, a framing that belongs to that press and not to the agency, whose releases never use the term; the case's significance is that the regulator treated an automated screen as ordinary actionable age discrimination regardless of mechanism, which is why it proceeded as intentional disparate treatment. All of the conduct described here is the Commission's allegation, resolved by a decree carrying no admission of liability.

Sources: u2023a, u2022c, greenbergtraurigllp2023

Appears on: /domains/cases/itutorgroup-age-screening

The practice surfaced through a single applicant's experiment on the input rather than through any internal control. The charging party applied with her real birthdate and was rejected immediately, then re-applied about a day later with an application identical in every respect except a more recent birthdate, and was offered an interview. That comparison required no inside access, one changed field and roughly a day, and it is the only comparison of a rejection against a counterfactual anywhere in the public record. The decree's reapplication invitations cover applicants rejected in March and April 2020, fixing the documented rejection window at two months, and roughly three and a half years then ran from that window to an enforceable remedy: rejections March to April 2020, a charge and failed conciliation, suit on 5 May 2022, joint notice of settlement 9 August 2023, decree approved September 2023. No public record identifies who authored the rule, why the thresholds differed by sex, or whether any internal review approved or missed it.

Sources: hrdive2023, greenbergtraurigllp2023, u2023a

Appears on: /domains/cases/itutorgroup-age-screening

The consent decree acted on the rule's input and on the oversight structure rather than on any decision logic. Its terms: injunctions against hiring discrimination based on age or sex; a prohibition on requesting applicants' birth dates before an offer, which removes from the intake form the field the rule computed on; a new anti-discrimination policy and an internal memo; multiple anti-discrimination trainings for those involved in hiring tutors; written notice to the EEOC of discrimination complaints, converting a formerly internal channel into a regulator-visible one; invitations to the applicants rejected in March and April 2020 to reapply; and, because iTutorGroup had already ceased hiring tutors in the United States, an obligation to notify and interview those applicants if it resumes United States operations, so part of the decree's machinery has never had occasion to operate. The Commission's own release and Seyfarth's 2024 recap give the duration as monitoring compliance 'for at least the next five years or longer if iTutorGroup resumes hiring tutors in the United States' — a floor with a conditional tail, not a flat five-year term. No post-decree enforcement activity in this case was identified.

Sources: u2023a, greenbergtraurigllp2023, seyfarthshawllp2024

Appears on: /domains/cases/itutorgroup-age-screening

The federal enforcement channel this decree instantiated changed measurably after the decree was entered, and the change is about the agency rather than about this case. In January 2025 the EEOC removed its Artificial Intelligence and Algorithmic Fairness Initiative content along with its May 2023 Title VII technical assistance on artificial intelligence and its May 2022 guidance on the Americans with Disabilities Act, following Executive Order 14179; the Department of Labor and the Office of Federal Contract Compliance Programs made parallel removals. Removing guidance repeals no law: Title VII, the Age Discrimination in Employment Act and the Uniform Guidelines on Employee Selection Procedures are unchanged, and no federal safe harbor was created. Executive Order 14281 of 23 April 2025 directs that 'All agencies shall deprioritize enforcement of all statutes and regulations to the extent they include disparate-impact liability' and orders a ninety-day evaluation of existing disparate-impact consent judgments; this decree rests on intentional disparate treatment, so that directive does not reach its theory, and nothing in the public record suggests the decree is threatened. The Commission lacked a quorum from January 2025 until 7 October 2025, when Commissioner Brittany Panuccio was confirmed; Andrea Lucas was designated Chair on 5 November 2025, and the restored commission's published 2026 priorities centre on investigations of diversity programmes, religious accommodation and national-origin cases, with artificial intelligence in hiring absent from the stated agenda.

Sources: cooleyllp2025, executiveorder2025, hollandknightllp2025

Appears on: /domains/cases/itutorgroup-age-screening

McHire is the chatbot-first hiring front end for approximately 90 percent of McDonald's franchisees, powered by Paradox.ai's conversational assistant Olivia. An applicant supplies contact details and shift preferences in natural-language chat, answers screening questions, is routed into a personality assessment administered by a further third party, Traitify.com, presented as agree-or-disagree phrase items, and self-schedules an interview before advancing to a human review stage where franchisee owners and managers manage them through the hiring stages. The employment-decision surface here is pre-screening and flow control rather than final selection: the assistant screens and schedules, and a person decides. Paradox is a conversational-hiring vendor whose client list extends well beyond this brand to other high-volume frontline employers including Aramark, Lockheed Martin, Lowe's and Pepsi, and in acquisition coverage Wendy's, 7-Eleven and General Motors. No error rate, completion rate or complaint rate for the assistant has been published by anyone, and no independent evaluation of this deployment exists; what the record does contain is public complaints that the assistant was answering nonsensically, which is what prompted two outside researchers to examine the platform at all.

Sources: carroll2025, bleepingcomputer2025, krebsonsecurity2025, workday2025

Appears on: /domains/cases/mchire-paradox-chatbot

Every conversation and every form field on McHire persisted into a single vendor-held lead record. Each record contained the applicant's name, email address, phone number, home address, shift preferences, the application's status and state-change history, the full raw chat transcript of the exchange with the assistant, and an authentication token permitting login to that applicant's own consumer interface — an impersonation surface stored alongside the record it identifies. Records accumulated without any purge described in the record: identifiers could be walked backward across the platform's history. The researchers' own test application received a lead_id of approximately 64,185,742, so application records numbering to roughly 64 million were REACHABLE through the flawed endpoint. That figure is the size of an identifier space and BleepingComputer states it represents the total number of job applications on the platform rather than unique applicants; it is a derivation from an identifier and is not a count of records exfiltrated or of people harmed. What makes this store a hiring object rather than a generic personal-data store is its content: who applied where, what they told a recruiting agent in their own words, the assessment step they were routed through, and their application status history. No documented model-retraining loop reads from these records.

Sources: carroll2025, bleepingcomputer2025, csoonline2025

Appears on: /domains/cases/mchire-paradox-chatbot

Security researchers Ian Carroll and Sam Curry found two flaws in what their writeup describes as a cursory review, prompted by public complaints that the assistant was answering nonsensically. First, the McHire administration login for restaurant owners accepted the default credentials 123456:123456 on a Paradox test-restaurant account, granting administration-panel access for that instance including all in-progress conversations. Paradox states that this legacy test account had not been logged into since 2019 and, in its own words, should have been decommissioned. Second, the endpoint PUT /api/lead/cem-xhr performed no authorization check on its lead_id parameter, returning any applicant's unmasked record to a caller who simply named the identifier; decrementing the identifier returned other applicants' records. Before 30 June 2025 nothing in the record surfaced either flaw: no employment regulator, no privacy regulator, no contractual security review, no audit and no internal review is documented examining the custody surface, McDonald's is not documented exercising its principal-to-vendor authority over the platform before the disclosure email arrived, and franchisees running hiring on the platform had no visibility into vendor credential practice and no documented channel through which to acquire any.

Sources: carroll2025, csoonline2025, paradox2025

Appears on: /domains/cases/mchire-paradox-chatbot

The disclosure-to-remediation sequence is documented to the minute. The researchers emailed Paradox.ai and McDonald's at 5:46 PM Eastern on 30 June 2025; McDonald's acknowledged at 6:24 PM, 38 minutes later; the default 123456 credentials were disabled by 7:31 PM the same evening, under two hours after the report; and Paradox confirmed the insecure direct object reference (IDOR) fix at 10:18 PM Eastern on 1 July 2025, approximately 29 to 30 hours after disclosure. On 9 July 2025 Paradox published a Responsible Security Update accepting responsibility in terms ('We take responsibility for this issue. Full stop.'), stating that five candidate records containing personally identifiable information — names, emails, phone numbers and IP addresses, all US-based candidates — plus two chat records containing no candidate information had been viewed and exclusively by the two researchers, that the store contained no Social Security numbers, that only the one client instance was affected and that no candidate information was leaked online, and announcing a bug bounty programme and a dedicated security contact at security@paradox.ai. That vendor position is uncontradicted and has never been verified by any independent forensic report, so potential exposure and actual access remain two separate quantities with nothing reconciling them. McDonald's statement, given to Wired and quoted verbatim in trade coverage, placed the failure with its supplier: 'We're disappointed by this unacceptable vulnerability from a third-party provider, Paradox.ai. As soon as we learned of the issue, we mandated Paradox.ai to remediate the issue immediately, and it was resolved on the same day it was reported to us.' The governance channel that fired was the security-research responsible-disclosure norm, operated by two people holding no contract, no mandate, no statutory standing and no access; detection and correction were both external to the deployment.

Sources: carroll2025, paradox2025, bleepingcomputer2025, csoonline2025

Appears on: /domains/cases/mchire-paradox-chatbot

No litigation, enforcement action or state attorney-general breach-notification filing tied to the McHire exposure was located as of 28 August 2026. That is stated as verified absence and never as exoneration, and the limits of the check are stated with it: the Maine attorney general's breach portal was offline at check time, reporting an apparent abuse of its data-breach reporting system, and the California attorney general's published breach list returned no entries, so the notification-registry finding rests on searches plus those partial registry checks rather than an exhaustive sweep. One search-engine summary asserted a Maine attorney-general filing; no underlying record could be found and it is recorded as unsubstantiated. Paradox's position — that only the two researchers viewed a handful of records and that nothing was leaked online — is the stated factual predicate under which broad breach-notification duties would not attach, and no independent adjudication of that position exists. A separate thread is kept separate: two weeks after the disclosure, Krebs on Security reported that infostealer malware on Paradox administrator and developer machines in Vietnam, identified as Nexus Stealer, had exposed weak seven-digit numeric passwords reused across multiple customer accounts along with Okta single-sign-on and Atlassian tokens valid into December 2025, and that a second developer compromise in late 2024 involved GitHub credentials. Paradox disputed the currency of the exposed passwords in part, attributing them to legacy password-manager migrations and stating that few remained active. That reporting concerns the company's credential hygiene generally rather than the McHire authorization chain specifically, and the dispute travels with it.

Sources: krebsonsecurity2025, paradox2025, csoonline2025

Appears on: /domains/cases/mchire-paradox-chatbot

Workday, Inc. announced a definitive agreement to acquire Paradox.ai on 21 August 2025, seven weeks after the disclosure, and completed the acquisition on 1 October 2025, folding the Olivia assistant into its talent-acquisition suite. Workday's own completion release describes the assistant as a conversational candidate-experience agent handling applications, screening support, self-scheduling and round-the-clock chat for frontline high-volume roles, and discloses no price. Trade coverage of the transaction carried the McHire incident and Paradox's dispute of its scope as due-diligence context, and recorded Paradox clients including Wendy's, 7-Eleven and General Motors. The effect on this case is that custody of the same applicant records moved inside a much larger human-capital software company three months after the exposure was disclosed. That acquirer is separately the subject of a different deployment record in this atlas on an unrelated question, and the two are not merged: this case is documented from the June and July 2025 record, when Paradox was independent.

Sources: workday2025, hrdive2025

Appears on: /domains/cases/mchire-paradox-chatbot

Meta Platforms' advertising system decides, for every job ad, which users inside the advertiser's eligible audience actually receive impressions, optimising predicted relevance and engagement against auction economics — so the allocation runs before anyone applies and the people it does not reach generate no application, no rejection and no adverse-action record anywhere. The platform's own published data, as recorded in the December 2022 class charge and the December 2023 joinder release, gives the scale: roughly 239 million US Facebook users, more than 30,000 US job ads published daily, women 54 percent of users interested in job hunting and people 55 and over more than 28 percent of them. The governance record runs on two layers that do not coincide: a restricted advertiser portal created by private settlement in March 2019, and a delivery-optimisation stage that independent peer-reviewed audits measured weeks later.

Sources: realwomenintruckingv2022, aarpfoundation2023, nationalfairhousingalliance2019, ali2019

Appears on: /domains/cases/meta-job-ad-delivery

On 19 March 2019 Facebook settled five coordinated legal actions — brought by the National Fair Housing Alliance and three regional fair-housing organizations with Emery Celli Brinckerhoff & Abady LLP, the Communications Workers of America, Outten & Golden LLP, the American Civil Liberties Union and individual plaintiffs — by agreeing to structural platform changes recorded in a joint statement the company co-signed: housing, employment and credit ads confined to a separate restricted portal; no gender, age or multicultural-affinity targeting; a fifteen-mile minimum geographic radius and no postal-code targeting; Lookalike Audiences for those ads stripped of gender, age, religion, postal-code and group-membership inputs; detection and rerouting of covered ads created outside the portal; advertiser anti-discrimination certification; plaintiff testing rights and regular implementation meetings; and a commitment to engage researchers on algorithmic bias, with implementation by 30 September 2019. The joint statement discloses no monetary terms; the roughly five-million-dollar figure attached to these settlements comes from contemporaneous press reporting rather than from the document.

Sources: nationalfairhousingalliance2019

Appears on: /domains/cases/meta-job-ad-delivery

In April 2019, weeks after the settlements took effect, Ali, Sapiezynski, Bogen, Korolova, Mislove and Rieke published a peer-reviewed study (CSCW 2019) showing that paired employment and housing ads carrying identical neutral targeting were delivered to strongly gender- and race-skewed audiences by the delivery stage itself, driven by the platform's own relevance predictions plus budget and market effects — with the budget channel operating independently of the content channel, because under a budget constraint the demographics that are cheaper to reach take more of the impressions. The method used ordinary advertiser accounts and was reproducible by outside auditors. In 2021 Imana, Korolova and Heidemann (WWW 2021) controlled for qualification distributions by running paired ads for the same job at companies with different de facto workforce gender mixes, confirmed gender skew in Facebook's job-ad delivery that qualification differences do not justify, and found no comparable skew on LinkedIn — making the behaviour platform-specific rather than labour-market-inevitable. Both studies measured the platform in the 2019-to-2021 era, before any remedial delivery controller existed even for housing ads.

Sources: ali2019, imana2021

Appears on: /domains/cases/meta-job-ad-delivery

On 21 June 2022 the United States sued Meta Platforms under the Fair Housing Act over housing-ad delivery, following a HUD Secretary-initiated complaint and charge, and on 27 June 2022 the court entered a settlement in United States v. Meta Platforms, Inc., No. 22-cv-5187 (S.D.N.Y.): Meta stopped using the Special Ad Audience tool, paid the $115,054 civil penalty that was then the statutory maximum, and agreed to build a Variance Reduction System to shrink the gap between an ad's eligible audience and its actual audience by sex and estimated race or ethnicity, under a compliance-metrics agreement of 9 January 2023, four-monthly reporting, an independent third-party reviewer and court oversight through 27 June 2026. The system samples eligible-audience demographics, estimates race and ethnicity by Bayesian Improved Surname Geocoding at a 50 percent threshold with differential-privacy noise added to the aggregates, retains no individual-level race data, and adjusts ongoing delivery in flight. Guidehouse Inc. issued five verification reports between June 2023 and 30 October 2024 (the fifth updated 19 December 2024); in its period of 1 May to 31 August 2024, for housing ads above 1,000 impressions, 95.1 percent met the ten-percent sex-variance threshold against a 91.7 percent requirement and 85.5 percent met the ten-percent estimated race and ethnicity threshold against an 81.0 percent requirement, with the reviewer's independent recomputation matching Meta's reported coverage at a 0.0 percent difference. All of the court-supervised metrics and all published verification cover HOUSING ads. Meta reports in its own newsroom having extended the same system to US employment and credit ads by October 2023, an operator-reported extension with no published compliance metrics and no third-party verification. The court-oversight term expired on 27 June 2026; the Department of Justice case page was last updated 21 January 2025 and lists no verification report after 30 October 2024, and no public post-expiry disposition was located as of 28 August 2026.

Sources: ua, guidehouseinc2024, metaplatforms2023

Appears on: /domains/cases/meta-job-ad-delivery

PLEADING, NOT A FINDING. On 1 December 2022 Real Women in Trucking filed a class charge of discrimination with the Equal Employment Opportunity Commission (No. 570-2023-00655), through Gupta Wessler PLLC and Upturn, Inc., alleging that Meta's ad-delivery algorithm discriminates by sex and age in deciding who receives job ads and pleading disparate treatment, disparate impact, the advertising-discrimination provisions and an employment-agency theory under Title VII and the ADEA and under state and local law; the charge alleges the post-2019 era inverted the mechanism, so that where job ads were once unlawful because advertisers expressly excluded older people and women, they are alleged to violate federal law since 2020 because the delivery algorithm itself relies on gender and age to limit who sees them. Its exhibits are tables from Facebook's own public ad library for ads whose eligible audiences were all-gender and all adult ages: a commercial-driving ad shown 94 percent to men and 5 percent to women, a mechanic ad above 99 percent to men, another blue-collar ad at 1 percent women and 3 percent users 55 and over, a school administrative-assistant ad at 2 percent men and 6 percent users 55 and over, a health-facility hospitality ad at 83 percent women, and a correctional-nursing ad at 22 percent men and 18 percent users 55 and over. AARP Foundation joined the charge on 19 December 2023 on behalf of older workers, citing more than 75 job postings as the evidentiary corpus. Meta's public response is that 'Addressing fairness in ads is an industry-wide challenge,' citing collaboration with civil-rights groups, academics and regulators. No tribunal has found Meta's delivery algorithm unlawful in employment. Counsel's live case page, accessed 28 August 2026, lists the charge as pending with no public EEOC determination. Its enforcement environment changed in 2025: the executive order of 23 April 2025, 'Restoring Equality of Opportunity and Meritocracy,' directed agencies to deprioritise disparate-impact liability, and a reported internal EEOC directive ordered disparate-impact-only charges administratively closed by 30 September 2025 with right-to-sue letters by 31 October 2025, from which charges also alleging intentional discrimination reportedly proceed on that theory alone.

Sources: realwomenintruckingv2022, upturn2022, aarpfoundation2023, prflawpeterromerfriedmanlaw2026, franczekp2025

Appears on: /domains/cases/meta-job-ad-delivery

Facebook's public ad library publishes per-ad demographic delivery tables only for ads classified as 'Issues, elections or politics.' Employment ads receive no such disclosure unless they carry that classification, so neither employers nor job seekers can routinely observe the demographic composition of a job ad's delivery — and the December 2022 charge's own per-ad evidence exists precisely because some job ads were so classified, as does the corpus of more than 75 postings the AARP Foundation cited when it joined in December 2023. The consequence recorded across this case's sources is that every delivery-layer finding in the record originated outside the operator: paired-ad academic audits run on ordinary advertiser accounts, and advocate harvesting of the ad library.

Sources: realwomenintruckingv2022, aarpfoundation2023, ali2019, imana2021

Appears on: /domains/cases/meta-job-ad-delivery

ATTRIBUTED TO THE EEOC, AND AGAINST ADVERTISERS. On 25 September 2019 the American Civil Liberties Union announced that the Equal Employment Opportunity Commission had issued reasonable-cause determinations against seven employers — Capital One, Edward Jones, Enterprise Holdings, Drive Time Auto, Nebraska Furniture Mart, Renewal by Andersen and Sandhills Publishing — finding they unlawfully excluded women and older workers from the audiences of their Facebook job ads. The determinations form part of roughly 66 charges filed since 2018 by the Communications Workers of America, the ACLU and Outten & Golden (56 age-only, 10 age and gender), addressed advertiser TARGETING choices rather than the platform's delivery algorithm, produced no platform remedy, and moved to conciliation with no public outcomes since. Reasonable cause is an administrative finding, not a court judgment, and the verified release does not itself date the determinations' issuance — 25 September 2019 is the date of the announcement.

Sources: americancivillibertiesunion2019

Appears on: /domains/cases/meta-job-ad-delivery

In March 2020 Northeastern University and pymetrics, inc. signed a sponsored research agreement that fixed, before any work began, the scope of a source-code audit of pymetrics' candidate-screening model-generation pipeline, the auditors' remuneration, their publication rights, a non-compete and a thirty-day responsible-disclosure window. Four Northeastern auditors conducted the audit in summer 2020 inside a pymetrics-provisioned AWS virtual machine, with access to source code, eight Jupyter notebooks, one engagement's full training and evaluation data, confidential technical, fairness-testing and job-analysis documents, staff time and a closing live demonstration of a production model trained, tested and deployed. The independence instruments were real and are documented: pymetrics paid $104,465 to the university, $64,813 of it salaries for the team, structured as a grant and paid in full BEFORE findings were delivered so it could not be conditioned on results, and supplied the audit compute at its own expense; the auditors kept their test methods secret from pymetrics throughout; and they held final editorial discretion including the contractual right to publish results that might reflect negatively on pymetrics. The contract, the audit and data-sharing protocol, the non-compete, the budget spreadsheet and the final report with section 3.2 redacted for proprietary information were posted publicly by the auditors. The paper was published at ACM FAccT in March 2021 with eight authors, four of them Northeastern auditors and four pymetrics employees including the company's chief executive as last author. The auditors coined the term 'cooperative audit' for the arrangement and explicitly placed it outside the existing internal/external taxonomy: they were not pymetrics employees, yet held privileged access to source code, data and staff, so by the field's own prior definitions the audit was not an external one. The transparency address printed in the 2021 paper no longer resolves; the live page is at cbw.sh/research/audits/, verified 2026-08-28.

Sources: wilson2021a, wilson2021b, wilson2021, schellmann2021

Appears on: /domains/cases/pymetrics-cooperative-audit

The control the audit examined is a pre-deployment four-fifths gate with abandon-if-noncompliant semantics. Per client role, pymetrics built a screening model from an in-group of 50 to 100 high-performing incumbents against an out-group sampled from a player database of more than two million, then tested the candidate model against a held-out 'bias group' — typically more than 10,000 players engineered to equal proportions across the seven EEOC categories, drawn from a pool of more than 600,000 players with self-reported demographics at over 75% survey completion per the vendor. Recommendation tiers sit at the 50th and 70th score percentiles, and deployment requires an impact ratio of at least 0.8 at BOTH thresholds on the bias group. If no compliant model is found, the engagement is reworked or no model ships. The audited code used support-vector-machine models over 64 features produced by 12 core games, filled gaps by median imputation, and dropped players missing more than two games.

Sources: wilson2021a, wilson2021b

Appears on: /domains/cases/pymetrics-cooperative-audit

The auditors found pymetrics' code correctly implemented four-fifths testing over the seven EEOC categories; that demographic data was not used as a training feature and no overt proxies such as zip codes were present; that they could not construct manipulated incumbent data producing a biased model without it being flagged, because every control-flow path reached the adverse-impact tests; that the 100-plus-item completion checklist per model plus the mandatory second-data-scientist review before production was a reasonable safeguard against negligence and against a single malicious insider, leaving collusion between two insiders as the residual path they explicitly named; and that median imputation did not substantively alter adverse-impact results despite statistically significant demographic differences in missing data. The auditors also recorded that no programmatic back-end re-check of adverse impact occurs after the data-scientist stage. Their own summary is that pymetrics 'passed the audit, subject to the qualifications and limitations we state'. No litigation, regulator enforcement action, consent decree or documented harm involving pymetrics or the audit was identified anywhere in this record.

Sources: wilson2021a, wilson2021b

Appears on: /domains/cases/pymetrics-cooperative-audit

The audit's exclusions were pre-agreed with the audited party in the contracted scope before the work began, not discovered as gaps afterwards. Excluded by that agreement: the choice of fairness objective and metric, the EEOC category set, intersectional groups, construct validity of the games, the newer reasoning games, post-deployment back-testing, customised client processes, security, and privacy compliance. Whether the games measure anything job-relevant — the assessment's reason for existing — was not assessed; the auditors state it was 'beyond our capabilities' as computer scientists. Intersectionality was excluded because such groups are not EEOC-recognised and all parties agreed the legal risk of optimising to a non-regulatory standard barred it, a reason the record states rather than adjudicates; pymetrics itself expressed interest in it. The audit is a summer-2020 snapshot of the standard, non-customised pipeline, and the auditors disclaim any claim about earlier practice, later practice or customised client engagements.

Sources: wilson2021a, wilson2021b

Appears on: /domains/cases/pymetrics-cooperative-audit

At ACM FAccT 2022, Young, Katell and Krafft made this engagement the lead case of what they term 'publishing and certifying corporate apologia': four of the paper's eight authors were pymetrics employees including the chief executive as last author; the company funded the work and set the scope, 'precluding various scenarios from scrutiny'; and 'it seems unlikely that this paper would have been submitted had the results negated the firm's validation claims.' They also documented the downstream loop: pymetrics had publicised the engagement in a company post of 13 May 2020, before any results existed, and after publication marketed the product as 'independently audited' — a term the audit paper's own cited definitions do not support, since the auditors classify the arrangement as cooperative and place it outside both the internal and external categories. They further recorded that a co-author sat on the FAccT Executive Committee when the paper was accepted and that 2021 FAccT review did not mandate funding disclosure. These are the critics' characterisations and arguments in a peer-reviewed position paper, attributed to them; no court, regulator, board or conference process has adjudicated them. The 'audited' framing carried into the acquisition: Harver announced its acquisition of pymetrics on 11 August 2022 with terms undisclosed, describing the product as 'mitigating multiple forms of bias through an audited AI platform'.

Sources: young2022, harverviaprnewswire2022

Appears on: /domains/cases/pymetrics-cooperative-audit

The audited practice migrated into a statutory channel. Under New York City Local Law 144, in effect from July 2023, the same game-based assessment platform is audited annually by a paid third-party auditor, BABL AI Inc., and the summary is published by the party deploying the tool: a V2.0 summary dated 29 June 2023, addressed to 'pymetrics inc. (Harver)', was posted by the employer Paramount, and a V1.0 summary dated 17 July 2025 for what is now Harver's Soft Skills Platform was posted by Harver. The platform retains the same three recommendation tiers and a 50th-percentile selection threshold. The 2025 summary reports every calculated impact ratio at or above 0.8, the lowest non-intersectional ratio being 0.914 for Asian candidates, computed across 164,014 male and 119,242 female candidates with known gender. It also reports what the calculation excluded: 191,455 candidates of unknown gender, 317,513 of unknown race or ethnicity — roughly twice the included known-race population — and 331,177 with at least one unknown demographic, with groups under two percent of the sample reported as not applicable. These are recorded external observations from a mandated annual audit: selection-rate ratios over candidates who disclosed demographics, not harm findings, and not evidence of platform-wide compliance. What the annual audits verify is bias-testing assertions and reported statistics, not the continued existence or exact mechanics of the pre-deployment abandon-if-noncompliant gate.

Sources: bablaiinc2023, bablaiinc2025

Appears on: /domains/cases/pymetrics-cooperative-audit

An ACLU-led study published at ACM FAccT 2025 analysed all 116 publicly available Local Law 144 bias audits from July 2023 to November 2024 and found the mandated audits to be incomplete evaluations of bias: missing demographic data, opaque aggregation, problematic test data, and metrics that do not represent how the tools are actually used. It warns of audit washing, corporate capture of auditors, and 'discrimination-hacking', records that the four-fifths rule is a rule of thumb rather than the legal standard, and shows that tools reporting four-fifths compliance could be in violation once missing-demographic impacts are considered. It cites the pymetrics cooperative audit among the proposed audit standards the field built on, and the corporate-capture critique among its warnings. The finding characterises the REGIME across all 116 audits and not this product's audits specifically, which were among the more complete in disclosing intersectional tables and unknown-group counts.

Sources: gerchick2025

Appears on: /domains/cases/pymetrics-cooperative-audit

The baseline this engagement stood out against is documented in the prior literature: a survey of 18 algorithmic pre-employment assessment vendors published at ACM FAT* 2020 found their publicly stated validation and bias-mitigation claims largely unverifiable from outside. The pymetrics engagement was the first publicly documented case of an assessment vendor opening its source code to outside auditors under pre-negotiated publication rights.

Sources: raghavan2020a, raghavan2020b, wilson2021a

Appears on: /domains/cases/pymetrics-cooperative-audit

The audited pipeline ran at scale: more than 600 active client engagements between January and October 2020, covering 16 of 23 major O*NET occupation groups, with players from 191 countries and roughly 40% of them in the United States. pymetrics also open-sourced its adverse-impact testing framework as the audit-ai library, cited in the audit paper, which implements the four-fifths rule alongside Fisher's exact test, a z-test, chi-squared and Cochran-Mantel-Haenszel tests; the repository remains public with low activity after the acquisition. No error rate, accuracy figure or defect rate for the model-generation pipeline is published in any source in this record, and no measurement compares this deployment with a human counterfactual on the same candidates.