Domain Atlas / Child welfare & family services
Allegheny Family Screening Tool
Evaluation evidence on the Allegheny Family Screening Tool found that screener overrides of the tool's recommendations reduced racial disparity in screen-in rates relative to the tool alone.[3]
What happened
Allegheny County's Family Screening Tool scores child-maltreatment referrals to support call-screening decisions, and is unusual for the depth of its public documentation: methodology reports, an independent county-commissioned impact evaluation, and sustained academic and journalistic scrutiny. That record includes findings on racial disparity in screen-in rates — and, notably, evaluation evidence that screener overrides of the tool's recommendations reduced the disparity relative to the tool alone. An independent audit of the tool's first years (2016-2018) put a mechanism under that finding: run without human override, the score would have recommended screening in about 68% of Black children versus 50% of white children (an 18-point gap), while the screeners actually screened in 51% and 43% (a 7-point gap) — the narrower gap came from workers disagreeing with the score about a third of the time.
Which direction the tool moved equity is itself contested and source-dependent. The county-commissioned Stanford evaluation reported that the tool and accompanying policy changes reduced racial-disparity gaps in investigation and case-opening rates; the Associated Press reported that the developers' own unpublished analysis had found no statistically significant effect of the algorithm on the disparity. Other critiques widened the frame. An ACLU and Human Rights Data Analysis Group analysis found that 97% of Black referral-households were affected by at least one permanent "ever-in" variable drawn from public-benefits data sources, versus 80% of non-Black households — casting the tool as poverty and permanent-record profiling. And the U.S. Department of Justice's Civil Rights Division was reported to be scrutinizing the tool after civil-rights complaints filed in fall 2022 raised concerns that its use of disability, mental-health, and Supplemental Security Income data may discriminate against parents with disabilities; families are not shown their scores, and no public findings have been reported.
The sociotechnical reading
AFST reframes "human in the loop" from a checkbox into a measurable component: here, the operator network demonstrably changed the system's equity behavior, in the protective direction. That cuts both ways — a system whose fairness depends on engaged overrides inherits every fragility of the humans doing the overriding (workload, deference drift, deskilling). It is the Atlas's clearest bridge to the deskilling and vigilance material in the Field Guide: the safeguard is alive, so it must be maintained like something alive.
Two structural features sharpen that reading. The map runs a memory loop — the score is computed from administrative records, and many inputs are permanent "ever-in" flags, so prior contact with jails, benefits, or behavioral-health systems durably raises a family's future score; yesterday's records and decisions become today's inputs, the loop the Lab's contamination stressor exercises. And because even the direction of the tool's equity effect is contested across well-resourced studies, the productive question the map asks is not whether "the tool" helped or harmed but where any equity effect is produced — and the independent audit locates it in the override step, not the model. That is also why opacity bites: a score families cannot see and cannot contest loads the equity burden onto that same override step.
The concepts used in this reading are defined in the Field Guide; the governance responses live in the Practice Library. The model organization for this case can be stress-tested in the PAN Lab.