Inside a Misfiring Government Data Machine

Last week, WIRED published a series of in-depth, data-driven stories about a problematic algorithm the Dutch city of Rotterdam deployed with the aim of rooting out benefits fraud.

In partnership with Lighthouse Reports, a European organization that specializes in investigative journalism, WIRED gained access to the inner workings of the algorithm under freedom-of-information laws and explored how it evaluates who is most likely to commit fraud. 

We found that the algorithm discriminates based on ethnicity and gender—unfairly giving women and minorities higher risk scores, which can lead to investigations that cause significant damage to claimants’ personal lives. An interactive article digs into the guts of the algorithm, taking you through two hypothetical examples to show that while race and gender are not among the factors fed into the algorithm, other data, such as a person’s Dutch language proficiency, can act as a proxy that enables discrimination.

The project shows how algorithms designed to make governments more efficient—and which are often heralded as fairer and more data-driven—can covertly amplify societal biases. The WIRED and Lighthouse investigation also found that other countries are testing similarly flawed approaches to finding fraudsters.

“Governments have been embedding algorithms in their systems for years, whether it’s a spreadsheet or some fancy machine learning,” says Dhruv Mehrotra, an investigative data reporter at WIRED who worked on the project. “But when an algorithm like this is applied to any type of punitive and predictive law enforcement, it becomes high-impact and quite scary.”

The impact of an investigation prompted by Rotterdam’s algorithm could be harrowing, as seen in the case of a mother of three who faced interrogation. 

But Mehrotra says the project was only able to highlight such injustices because WIRED and Lighthouse had a chance to inspect how the algorithm works—countless other systems operate  with impunity under cover of bureaucratic darkness. He says it is also important to recognize that algorithms such as the one used in Rotterdam are often built on top of inherently unfair systems.

Source : Wired