Domain Atlas / Public benefits & eligibility
Udbetaling Danmark data-driven control (Denmark)
Work with this case in the PAN Lab ↗
Denmark's Udbetaling Danmark (UDK), administered by ATP, runs a data-driven welfare-fraud operation that as of 2019 used up to about 60 AI and machine-learning models to score benefit recipients into a 'wonderlist' of high-risk people, which a human control team filters into control cases for investigation. In UDK's own 2023 control statistics (three documented models), the 'Model Abroad' foreign-affiliation model sent 511 cases for control but recovered money in only 36 -- about 7%, with roughly nine in ten resulting in no further action -- and UDK confirmed that 54% of the 'Really Single' household-outlier cases its unit opened were in fact legitimate. Those 'revenue' outcomes conflate deliberate fraud with honest error, which UDK does not separate, so they are not pure fraud rates. Amnesty International characterised the system as mass surveillance and prohibited social scoring under the EU AI Act; UDK, ATP and the ministry (STAR) rejected that characterisation, the system was not suspended, and as of this writing no court had ruled.[4]
What happened
Udbetaling Danmark (UDK), established in 2012 to centralise municipal welfare payments -- child allowance, pensions, housing benefit, unemployment, maternity and sick pay -- runs a data-driven fraud-control operation administered by ATP (Arbejdsmarkedets Tillaegspension). ATP is a quasi-public body that administers on UDK's behalf and subcontracts private firms, including NNIT A/S, to build some of the fraud-control models; that public-private split is itself part of the accountability critique. In 2021 UDK paid about DKK 241 billion (about EUR 32.3 billion) to roughly 2.4 million benefit recipients -- on the order of half of Denmark's adult population. As of 2019 the operation used "up to 60" AI and machine-learning algorithms; Amnesty International's Algorithmic Accountability Lab obtained redacted freedom-of-information documentation on only four of them, with input weights and many inputs withheld.
At the centre is a "Joint Data Unit" that merges and links the personal data of millions of residents from around ten national registers -- civil registration (CPR), buildings and dwellings (BBR), business (CVR), income, tax (R75), regional health data, VAT, cash and sickness benefits (STAR), education grants (SU) and the motor-vehicle register -- alongside a "Joint Data Unit Abroad" that pulls residence, entry-exit, marital, children, property and foreign-benefit data from foreign authorities. The models output a "wonderlist" (a question list) of statistically high-risk people, which a human control team (the HOK unit) and municipal control units filter into "established control cases" before investigation. Three documented models carry 2023 outcome figures. "Really Single" is a household-outlier model that polices single-person supplements, heavily weighting a "housing score" and a "rel atypical resident score" drawn from CPR and BBR, so that people whose living arrangements look statistically atypical are flagged. "Model Abroad" scores a beneficiary's "foreign affiliation" as a relative strength of ties to non-EEA countries and prioritises people with "medium and high" ties for investigation, with citizenship a direct input; internal documentation states that "there is thus good sense in removing cases from the heaps with High or Medium non-EEA affiliation." A third "Fictitious Employment" and parental-leave income control completes the quantified set; a fourth documented model was obtained only in redacted form and is not separately quantified.
In UDK's own 2023 control statistics (its Table 5), the funnel runs from "requested cases" (the algorithm's output) to "established control cases" (caseworker-accepted) to "cases with revenue": Really Single produced 412 requested, 292 control cases and 135 with revenue (a 33% revenue rate; UDK confirmed 54% of the cases its unit opened were in fact legitimate); Model Abroad produced 511 requested, 351 control cases and 36 with revenue (a 7% revenue rate, roughly 90% no further action); and the Fictitious Employment control produced 491 requested, 207 control cases and 72 with revenue (a 15% revenue rate). "Revenue" here means money found to be owed whether through deliberate fraud or honest error -- UDK does not split the two -- so these are not pure fraud rates. About 30% of investigated fraud cases originate from the models; the control team handles roughly 5,000 to 6,000 cases a year, of which about 1,800 come from the algorithms, the rest from tips that Danish law requires be processed. UDK assesses model quality by the share of provided cases the control team deems worth control, and describes its bias approach as equalising the "true positive rate." As of 2019, 90 of Denmark's 93 municipal control teams used social-media data in fraud checks; in one documented case a woman was ordered to repay DKK 12,500 (about EUR 1,650) after a Facebook-based cohabitation finding, a decision the National Appeals Board (Ankestyrelsen) overturned.
Oversight is thin and fragmented. The Danish Data Protection Authority (Datatilsynet) told Amnesty it can generally act only on complaints (GDPR Art. 57) and that data-protection impact assessments are the controller's own responsibility (Art. 35), with no proactive investigatory power; because flagged people rarely know an algorithm selected them, complaints are rare. The Ministry of Employment and its agency STAR said their oversight is limited to efficiency and fraud-prevention goals and that they cannot instruct UDK on case processing. Crucially, the discrimination Amnesty describes is a design-level risk rather than a measured outcome: UDK and ATP denied all requests for the demographic data needed to test the models for bias, saying they do not hold it, so no statistical disparate-impact figure exists in the record. Amnesty's report -- "Coded Injustice: Surveillance and Discrimination in Denmark's Automated Welfare State" (index EUR 18/8709/2024) -- was launched on 13 November 2024 (its document index is dated 12 November) after more than two years of research; researcher Hellen Mukiri-Smith said the mass surveillance risks "targeting the very people it was meant to protect," and an interviewee described life under the system, in Danish, as "Det er som at sidde for enden af pistolen. Vi er altid bange" ("It is like sitting at the end of the gun. We are always afraid").
Amnesty argues the system functions as prohibited social scoring under the EU AI Act (Art. 5(1)(c)) and should be banned. UDK rejected that assessment, insisting human caseworkers always review flagged cases; STAR (in a written response of 1 November 2024) rejected the oversight findings, and UDK rejected the discrimination-by-design finding without further explanation; subcontractor NNIT responded to Amnesty on the same date. The legal characterisation is disputed and unresolved. The system was not suspended: in the 2024-25 parliamentary session, the Folketinget's Digitalisation and IT committee (DIU) received a briefing (Bilag 32) in which the Minister of Employment forwarded UDK's board's account of "the factual conditions in the data-driven control" -- a legislative follow-up, with no regulator-ordered halt, court ruling, or data-protection enforcement action confirmed as of this writing. (Historical context, from AlgorithmWatch's 2020 reporting and kept separate from the algorithms' own record: UDK's 2017-18 controls found about DKK 0.5 billion (about EUR 70 million) in erroneous payments, not split between honest error and deliberate fraud, and 91 individuals were referred to police for welfare fraud in 2017; earlier state maladministration, unrelated to these models, underpaid about 300,000 pensioners in 2013 and mis-set housing benefit for about 325,000 households in 2015. The separate 2018 municipal "Gladsaxe model" early-warning pilot is a distinct case, not one of UDK's fraud models.) No AI model identifier is documented in the record.
The sociotechnical reading
Udbetaling Danmark is the Atlas's standing-surveillance-by-data-linking case, and its lesson sits upstream of any single flag. The harm is the linkage itself: a Joint Data Unit that merges around ten national registers -- and pulls foreign-authority data -- over a population on the order of half of Denmark's adults, holding them all inside one suspicion apparatus before the first model even runs. That is a different failure from the domain's other fraud-scoring cases. Rotterdam's model was pried open by an outside audit, measured for bias, and suspended; France's CNAF score was provable-on-its-own-arithmetic and quantified by the operator's own internal study; Sweden's risk profile was secret and then decommissioned once its measured disparities surfaced. Denmark's is live, contested, and unmeasured -- because the operator withheld the very demographic data that would let anyone compute a disparity. The map's honest reading has to hold that absence: the discrimination is a design-level risk, argued from the model's structure, not a proven error-rate gap.
Two structural features carry that risk. The first is a relative metric: "Model Abroad" scores "foreign affiliation" as a strength of ties to non-EEA countries and always selects some "medium and high" group, so its threshold moves with the population rather than with anyone's behaviour, and citizenship enters directly as an input. The second is outlier-as-suspicion: "Really Single" treats a statistically atypical household as a reason for scrutiny, which structurally falls on non-mainstream family forms. Both are compounded by low precision at scale -- in UDK's own figures most flagged-and-opened cases were legitimate, roughly nine in ten for the foreign-affiliation model -- so the burden of being investigated lands overwhelmingly on people who did nothing wrong. And the models are retrospectively calibrated on prior control cases, so whoever was targeted before feeds forward into who is suspected next.
What the framework flags as the missing control is not an absent human -- the control team's two-stage filter is a real check, and it is the safeguard UDK points to when it rejects the "social scoring" label -- but the absence of everything that would test the apparatus from the outside or the inside. There is no minimisation or authorisation gate on the linkage (the connection-authorisation and data-minimisation levers), no proactive oversight (the data-protection authority acts only on complaints, and opacity ensures the people who could complain never learn they were selected), and no independent precision or fairness check on the wonderlist itself (the cross-model lever), only a weak human second look that no scheduled challenge sharpened (the challenger lever). This inverts the usual audit-as-actor story: the scrutiny came from an outside investigation using freedom-of-information law, and even then the system did not stop. The deference question is live too -- a wonderlist rank becomes a verdict exactly when the control team treats it as the answer. The contested framing is part of the record and kept in it here: Amnesty asserts mass surveillance and prohibited social scoring, UDK, ATP and the ministry reject that characterisation, a parliamentary committee took up the matter, and no court has ruled.
The concepts used in this reading are defined in the Field Guide; the governance responses live in the Practice Library. The model organization for this case can be stress-tested in the PAN Lab.