• In a first anywhere in the world, a court in the Netherlands recently stopped a digital identification scheme for reasons of exclusion.
• This has a context for similar artificial intelligence (AI) systems worldwide, especially at a time when identity, citizenship and privacy are pertinent questions in India.

SyRI

• Last week, a Dutch district court ruled against an identification mechanism called SyRI (System Risk Indicator), because of data privacy and human rights concerns.
• It held SyRI was too invasive and violative of the privacy guarantees given by European Human Rights Law as well as the EU’s General Data Protection Regulation.
• The Dutch Ministry of Social Affairs developed SyRI in 2014 to weed out those who are most likely to commit fraud and receive government benefits.
• Legislation passed by Dutch Parliament allowed government agencies to share 17 categories of data about welfare recipients such as taxes, land registries, employment records, and vehicle registrations with a private company.
• The company used an algorithm to analyse data for four cities and calculate risk scores.

What were the arguments in court?

• After taking into account community concerns, civil society groups and NGOs launched a legal attack on this case of algorithmic governance.
• Legal criticism mounted, alleging that the algorithm would begin associating poverty and immigrant statuses with fraud risk.
• The Dutch government defended the programme in court, saying it prevented abuse and acted as only a starting point for further investigation instead of a final determination.
• The government also refused to disclose all information about how the system makes its decisions, stating that it would allow gaming of the system.
• The court found that opaque algorithmic decision-making puts citizens at a disadvantage to challenge the resulting risk scores.
• The Netherlands continuously ranks high on democracy indices.

How relevant is this for India?

• Similar to the Supreme Court’s Aadhaar judgment setting limits on the ID’s usage, the Hague Court attempted to balance social interest with personal privacy.
• However, the Aadhaar judgment was not regarding algorithmic decision-making; it was about data collection.
• The ruling is also an example of how a data protection regulation can be used against government surveillance.
• India’s pending data protection regulation, being analysed by a Joint Select Committee in Parliament, would give broad exemptions to government data processing in its current form.
• India’s proposed regulation is similar to the US in the loopholes that could be potentially exploited.

Get an IAS/IPS ranker as your 1: 1 personal mentor for UPSC 2024

Attend Now