Child welfare & family services
Predictive screening and profiling where the cost of both false alarms and misses lands on families — and where the human override layer has measurably mattered.
Use cases
What AI is doing here
Maltreatment call screening
PredictiveRisk scores supporting screen-in/screen-out decisions on child-maltreatment referrals.
Family risk prediction
PredictiveLongitudinal risk models over family and administrative data to prioritize investigation or services.
Early-help profiling
PredictiveMining council/agency data to flag families for preventive outreach before crisis.
Case notes as training data
PredictiveUsing narrative case records to train predictive models — importing the biases and errors those records contain.
Case files
What has gone wrong, and right
Documented deployments, presented as model organizations calibrated to the evidence, with full citations.
Allegheny Family Screening Tool
Allegheny County, Pennsylvania, USAThe most-studied predictive risk score in child welfare — and evidence that the human override layer is where equity was won or lost.
Stress-test this shape in the PAN Lab →Illinois Rapid Safety Feedback
Illinois, USAA child-welfare risk tool that flagged thousands of children at extreme risk while missing actual fatalities — discontinued in 2017.
Stress-test this shape in the PAN Lab →Oregon Safety at Screening
Oregon, USAAn AFST-derived screening tool that Oregon shelved in 2022 amid equity concerns — a rare pre-crisis discontinuation.
Stress-test this shape in the PAN Lab →Hackney / Xantura Early Help Profiling
London Borough of Hackney, UKA council's vendor-built family-profiling pilot, run without telling the families, quietly scrapped in 2019 after data-quality problems meant it surfaced few genuinely new cases.
Stress-test this shape in the PAN Lab →Allegheny Hello Baby
Allegheny County, Pennsylvania, USAA predictive model scores every newborn in the county to offer prevention services, not investigations — benevolent intent riding on a maltreatment-removal prediction, delivered by mandated reporters behind a deliberate firewall.
Stress-test this shape in the PAN Lab →Douglas County Decision Aide
Douglas County, Colorado, USAAn independently-trialed, race-excluded child-welfare screening score that a rigorous evaluation found sped decisions but barely changed outcomes — because workers heeded only the extremes.
Stress-test this shape in the PAN Lab →Eckerd Rapid Safety Feedback: origin and spread
Hillsborough County, Florida, USA (spread to multiple states)A child-welfare risk tool whose Florida success was the vendor's own claim — endorsed by a federal commission and copied across states years before an independent evaluation found it changed nothing.
Stress-test this shape in the PAN Lab →ProKid (Netherlands)
Netherlands (national police; four pilot regions)Dutch police risk-profiling that scored children under 12 — including those recorded only as victims — from up to twelve years of police records; the government's own pilot evaluation found no well-functioning instrument in any of its four regions.
Stress-test this shape in the PAN Lab →Insight Bristol / Think Family Database
Bristol, England, UKA population-scale police-and-council database whose child-exploitation risk models were quietly switched off as 'not fit for operational use' — leaving auditors unable to find the code, the variables, or any record of why.
Stress-test this shape in the PAN Lab →Sistema Alerta Niñez (Chile)
Chile (national; Ministry of Social Development and Family)A national predictive-risk score that ranks children for early-help outreach from data families gave to receive benefits — its operational accuracy never published, its independent bias audit never disclosed.
Stress-test this shape in the PAN Lab →Los Angeles County Project AURA
Los Angeles County, California, USAA proprietary child-abuse risk model that a 95.6% false-positive rate stopped in retrospective testing — shelved before it ever scored a live case.
Stress-test this shape in the PAN Lab →What Works for Children's Social Care ML pilots
England, United KingdomA government-funded evidence centre built child-welfare prediction models, pre-registered a success bar, missed it on every model, and published the failure — the rare case where the evidence gate fired before deployment.
Stress-test this shape in the PAN Lab →System map
Who is in the system, and what pushes on it
Who is in the system
- Frontline workers. Caseworkers, screeners, eligibility staff — the operator network whose judgment the system augments or erodes.
- Supervisors & QA. The institutional correction layer: overrides, second reads, quality review.
- Agency leadership. Owns procurement, policy, and the authority map; answers for the system publicly.
- Served people & families. Those the decisions land on. Deliberately outside the PAN dynamics — their outcomes are measured, never simulated.
- Regulators & oversight bodies. Boards, auditors, data-protection officers, inspectorates — external correction capacity.
- Advocates & community organizations. Surface harms institutions do not see; historically the earliest accurate signal.
Dominant pressures
- Caseload surge. Demand outruns staffing; per-case attention shrinks and review becomes triage.
- Deadline pressure. Statutory or managerial timeliness rules reward fast approval of machine output over slow disagreement.
- Staff turnover. Experienced skepticism leaves; new staff calibrate their trust on the tool itself.
- Data & policy drift. The world, the intake process, and the rules change under a system trained on how things used to be.
- Compliance over substance. Paper controls (sign-offs, checklists) satisfy audits while the behavior they describe erodes.
Governance
Questions leaders should be asking
- 1. What does a risk score change about a worker's next action — and is that mapping written down anywhere?
- 2. Are overrides tracked, and does anyone know whether they are improving or degrading equity?
- 3. If the tool were saturating workers with alerts, how would leadership find out?
- 4. What would trigger discontinuation, and who holds the authority to trigger it?
For the actions behind these questions, see the Practice Library.
Seeing your organization in this domain? Mapping its actual pathways, pressures, and correction capacity is engagement work.
Work With 1023AI →