Could the EU's AI Act increase mass surveillance?

Last December, EU negotiators reached a preliminary agreement on the AI Act, a world-first attempt to regulate the emerging technology that includes new rules on the use of biometric identification systems such as facial recognition.

But civil society organisations fear there are loopholes in the planned law. 

“They have set very broad conditions for the police to use these systems. What we fear is that this will have a legitimising effect,” said Ella Jakubowska of Reclaim Your Face, a coalition calling to ban biometric mass surveillance.

Jakubowska says that until now it had been “possible to challenge” these systems and argue that they were not wanted “in a democratic society.” She fears they will now be harder to reject, and more likely to be adopted by other countries worldwide under the impression they have received the EU seal of approval.

The new regulation, which is awaiting final approval by the Council of the EU and the European parliament, would set different conditions for whether facial recognition systems are used directly or remotely. In both cases, it would be subject to judicial authorisation and would only be available in specific contexts. 

Live use should be limited in time and space and should concentrate on the prevention of specific terrorist threats, the identification of suspects of crimes such as terrorism, human trafficking or kidnapping, or to anticipate a terrorist attack.

In remote use, it would focus on the location of persons convicted of or suspected of having committed a serious crime.

Parliament and member states clash

The European Parliament had called for a full ban on facial recognition, but softened its red line in response to the demands of countries such as France.

Paris was among the capitals that pushed hardest for exceptions that would allow wider use. It has even announced the use of AI to monitor suspicious activity during the 2024 Olympic Games to be held in the country.

Digital rights organisations decry the legislation for failing to put an end to mass surveillance. “What we can expect is a potential increase in the use of facial recognition systems in our public spaces, especially when these systems are used live,” Jakubowska said.

“While you are walking in a public space, going to the shops, to school, to the doctor, to a demonstration,” she explained, “there could be increased powers for law enforcement agencies to use this live facial recognition technology to track you through time and place, wherever you go.”

But for some MEPs, the AI Act strikes the right balance between security and civil rights. “I think it is a very good way of balancing integrity and security,” argued, Arba Kokalari, an MEP for the centre-right European People’s Party (EPP).

“If we had banned this technique, two things would have happened. Why should the integrity of a terrorist be more important than the security of our citizens? And secondly, this technique would continue to be developed by other countries and especially by China,” said Kokalari.

EU countries will vote on the final text of the law on Friday. Some of them have not yet decided how they will vote, but fears that the legislation could fall were allayed on Tuesday after Germany announced that it would vote in favour. Parliament will then have to approve the text.

Video editor • Vassilis Glynos