1.1. Scope and research questions

Artificial intelligence (AI) seems to be disrupting most fields of life in one way or the other. For the field of law enforcement, as in many other sectors, it is nothing new to use the potentials of electronic computation, data analysis and – more generally – integrating scientific advancements into police investigations (Fontes & Perrone, Reference Fontes and Perrone2021; Navarro, Reference Navarro2023). From data scraping on the internet and lie detectors to bulk telecommunication surveillance, biometric identification or instant translation (Donohue, Reference Donohue2014; Edri, 2020; Khder, Reference Khder2021): what is possible with AI is a next step in a long-lasting technological development in computer science rather than the quantum leap, or even an “AI revolution” (Awwad, Reference Awwad2024; Chataut et al., Reference Chataut, Nankya and Akl2024; Makridakis, Reference Makridakis2017), that stakeholders in industry and media seem to suggest at times (Cebral-Loureda et al., Reference Cebral-Loureda, Rincón-Flores and Sanchez-Ante2023; Chowdhury & Sadek, Reference Chowdhury and Sadek2012). Yet, many new AI tools are currently flooding the market, and some have the potential to modernize and improve the work of police and other security forces, while others allow surveillance in a dimension that might come in conflict with digital human rights and the rule of law (Berk, Reference Berk2021; Casaburo & Marsh, Reference Casaburo and Marsh2024; Cataleta, Reference Cataleta2021; Muller, Reference Muller2020; Završnik, Reference Završnik2020).

This article takes a closer look at the instrument of regulatory sandboxing in AI regulation (Charisi and Dignum, Reference Charisi, Dignum, Régis, Denis, Axente and Kishimoto2024; Ruschemeier, Reference Ruschemeier and Steffen2025) – the central measure in support of innovation laid down in Chapter VI of the AIA. The legal instrument aims to “ensure a legal framework that promotes innovation, is future-proof and is resilient to disruption” by allowing the “development and testing of innovative AI systems under strict regulatory oversight before the systems are placed on the market” (Recital 138 AIA). The focus of this article lies on the potential of regulatory sandboxes to develop legally compliant, technologically sovereign and democratically controllable AI systems for law enforcement agencies. Recitals 138, 139 AIA give the first impression that regulatory sandboxes are mainly drafted as a tool to test innovative products of SMEs and start-ups in an early stage (Boura, Reference Boura2024; Yordanova, Reference Yordanova2022) and reduce their administrative and regulatory burden. Yet, the legislator has foreseen the potential of this “limited regulatory space” (DeMeola, Reference DeMeola2021) as a protected environment – a “balanced solution” (Christoph, Reference Christoph2023) – for law enforcement agencies to co-create technical tools, potentially alongside precise legal provisions, including necessary safeguards and technical as well as organizational measures of data protection (Ballestrem et al., Reference Gausling, Ballestrem, Bär, Gausling, Hack and von Oelffen2020; Helminger, Reference Helminger2022; Jandt, Reference Jandt2017; Müller-Quade & Houdeau, Reference Müller-Quade and Houdeau2023) and supervisory structures. By analyzing the provisions of the AIA, I want to find out whether the instrument of regulatory sandboxing – as Christoph (Reference Christoph2023) has put it the context of US law – can contribute to bringing “coherence and discipline to the use of new technology in law enforcement and criminal proceedings” where the status quo of procuring private sector solutions “has led to uncertainty, inconsistency, and the danger of manifest injustice within local, state and federal justice systems” (Christoph, 2023). This article analyzes the hypothesis that regulatory sandboxing can be an effective way out of the conflicting paths of (quickly) modernizing policing by allowing the use of AI systems on the one hand and drafting legal provisions and implementing AI systems that comply with high fundamental rights standards on the other. It takes the discussion around biometric internet scans, as introduced as part of a federal “security package” in the Federal Republic of Germany (section 1.3. of this article), as a case study to outline the advantages of the instrument of regulatory sandboxing in the field of law enforcement compared to hasty legislative processes.

1.2. AI systems in law enforcement under rule of law and the AIA

In the future, those who provide (Art. 3[3] AIA) or deploy (Art. 3[4] AIA) AI systems in the EU will have to comply with product safety provisions of the AI Act. This “uniform legal framework (…) for the development, the placing on the market, the putting into service and the use of artificial intelligence systems” ultimately aims at “ensuring a high level of protection of (…) safety, fundamental rights (…), including democracy, the rule of law” (Recital 1[1] AIA). That the use of technology needs to be supported by a legal basis is nothing new in the field of public administration. Already under the rule of law, law enforcement agencies are not free to integrate technology in their investigative toolbox (Greenstein, Reference Greenstein2022; Hesse, Reference Hesse1999; Kommers and Miller, Reference Kommers and Miller2012;Weber, Reference Weber2008). The AI Act follows this basic understanding of fundamental rights doctrine.

The AIA follows a risk-based approach and densely regulates the category of “high-risk AI systems” (Art. 6 AIA). According to Recital 46 AIA, such systems should only be placed on the Union market, put into service or used if they comply with certain mandatory requirements in order to ensure that they “do not pose unacceptable risks to important Union public interests.” Certain AI systems used by law enforcement agencies are considered high-risk AI systems (Art. 6[2], Annex III [6][a-e] AIA) due to their potential impact on fundamental rights of citizens. Also, the field of “migration, asylum and border control management” widely falls into the high-risk category (Art. 6[2], Annex III[7] AIA). The lists of Annex III, Nr. 6 and 7 show different forms of AI systems that are considered high-risk: from assessing “the risk of a natural person becoming the victim of criminal offences” (Nr. 6[a], polygraphs (Nr. 6[b]), “assess personality traits and characteristics or past criminal behaviour “(Nr. 6[d]) to AI systems in migration meant for “detecting, recognising or identifying natural persons” (Nr. 7[d]).

Apart from whether an AI system is banned or considered high-risk according to the AIA, the limits of the national constitution must be respected (Almada and Petit, Reference Almada and Petit2025; Masala, Reference Masala2024; Micklitz et al., Reference Micklitz, Pollicino, Reichman, Simoncini, Sartor and De Gregorio2021), especially if the use of technology has an impact on fundamental rights of citizens. Every encroachment of fundamental rights must be based on a legally certain provision in national law that is proportionate. The AIA acknowledges the necessity of a legal basis by clarifying in Annex III(6) that their use must additionally be “permitted under relevant Union or national law.” Against this background, it becomes apparent that the use of AI systems by law enforcement agencies will be strictly regulated under the AIA based on a complex interplay of national and EU legislation in the field of AI, data protection and national security law.

1.3 The German “security package” as an example for political and legal debates around the legality and constitutionality of high-risk AI systems in the realm of law enforcement

As an example, I will introduce a case from Germany that has attracted significant public attention. For decades, police forces were not able to track down the alleged leftist terrorist Daniela Klette, who had gone underground in the 1990s and is suspected of having committed a series of robberies afterwards (The Times, 2023). At the end of 2023, a team of journalists was able to find new evidence about Klette’s whereabouts in the course of their research for a podcast: They fed the official mugshot being displayed at airports and train stations across the country into the AI-Tool PimsEyeFootnote ¹ and found her face on pictures of a Capoeira group in Berlin on Facebook (Bovermann et al., Reference Bovermann, Fink and Mutter2024; Großekemper et al., Reference Großekemper, Höfner, Rosenbach and Wiedmann-Schmidt2024). Police forces followed the lead of those journalists and were able to arrest Klette a few weeks later. It turned out she had lived and freely moved in the heart of Berlin for years without ever being recognized. Security agencies felt embarrassed and got the impression that the success of the journalists was dancing on their noses. As a result, a public debate sparked: How can it be possible that a small group of journalists can track down an alleged terrorist gone underground with a freely available software application, while police have been unable – and not allowed – to do so?

And: What are the limits of security agencies in the EU to use AI systems that are available on global markets? Legal questions surrounding the use of AI software, especially by some non-EU companies, have arisen in manifold ways. In the case of Palantir, a series of judgments of the German constitutional court (Bundesverfassungsgericht) have circled around the question if and under which circumstances law enforcement authorities in the states of Hessen, Bayern and Hamburg (Bäuerle, Reference Bäuerle2024) are allowed to automatically analyse large databases of state and federal police authorities; with Palantir also being used in Nordrhein-Westfalen. The law enforcement authorities had procured a software application from Palantir and tried to adjust it to the procurement conditions. So far, however, no constitutional possibility has been found, and the Bundesverfassungsgericht asked for adjustments (see Judgment of the First Senate of 16 February 2023, file numbers: 1 BvR 1547/19, 1 BvR 2634/20). Similarly, in the case of Daniela Klette, law enforcement authorities must have felt inclined to procure Clearview AI or PimsEyes. However, Clearview AI had already been heavily fined by Italian and Dutch data protection authorities for stark infringements of data protection law (Der Hamburgische Beauftragte für Datenschutz und Informationsfreiheit, 2020; Martini & Kemper, Reference Martini and Kemper2023a; Martini & Kemper, Reference Martini and Kemper2023b; Rezende, Reference Rezende2020; Jung and Kwon, Reference Jung and Kwon2024).

Working with selected SME or larger companies in the protected experimental environment of a regulatory sandbox would allow for a path that follows the political goal of “digital sovereignty”, now often used as a shorthand for an ordered, value-driven, regulated and therefore reasonable and secure digital sphere (Bellanova et al., Reference Bellanova, Carrapico and Duez2022; European Commission, 2023; European Parliamentary Research Service (EPRS), 2020; Floridi, Reference Floridi2020; Heidebrecht, Reference Heidebrecht2024; Pohle and Thiel, Reference Pohle and Thiel2020). Both the market surveillance authorities (Art. 70 AIA), authorities for the protection of fundamental rights (Art. 77 AIA), and the law enforcement authorities would gain insights into the technical design, configuration and main components of the used AI systems – instead of having to deal with proprietary systems of large tech companies which have been developed in a different regulatory environment. For a regulatory sandbox for AI systems in the field of law enforcement, the federal and state levels could encourage certain companies – especially SME and European-based companies – to figuratively throw their hat in the ring – with the additional (implicit) prospect to later have a top-runner position in procurement procedures to acquire suitable AI systems for the police.

Interestingly, the arrest of Klette coincided with the final negotiations of the European Union’s landmark AI Act. In the final trilogue, Members of the European Parliament had pushed for a ban on biometric internet scans by PimEyes, Clearview AI and the likes by implementing a ban in Art. 5 lit. e AIA (accompanied by Recital 43), forbidding the “placing on the market, the putting into service for this specific purpose, or the use of AI systems that create or expand facial recognition databases through the untargeted scraping of facial images from the internet or CCTV footage.” Nonetheless, the German Federal Government – under the influence after a terrorist attack in the city of Solingen (Euractiv, 2024; Kurmayer, Reference Kurmayer2024) – pushed for a “security package” (Sicherheitspaket) to react to threats of supposed Islamist terrorism. The ministerial draft included the legal competence of migration authorities to compare a photograph taken of foreign nationals upon entry biometrically with publicly accessible personal data from the internet by means of an automated data processing application in case there are no other means of identification.Footnote ² Similar provisions implemented for the Federal Police (Bundeskriminalamt and Bundespolizei) to use the same technology under certain additional requirements – expanding the possibility to not only the face, but also the voice of a person.Footnote ³

The draft law sparked a public debate, including an outcry of civil society organizations who had been campaigning against all forms of biometric identification in public space for years (Reclaim Your Face, n.d.; Reuter, Reference Reuter2024). Under the pressure of the federal government, which wanted to show a swift reaction to the terrorist attack, and after intense negotiations in parliament that added additional safeguards, the Bundestag finally approved the two laws of the Sicherheitspaket.Footnote ⁴ An expert hearing on the first draft had previously led to media coverage, after several experts held the law in breach of German constitutional law, GDPR and also the AIA. The Federal Commissioner for Data Protection and Freedom of Information, Prof. Dr. Louisa Specht-Riemenschneider, criticized that the legal provisions did not even cursorily sketch the technical functionality and implementation of biometrical internet scans, thus not setting up specific safeguards and limitations, opening law enforcement a too wide scope that can lead to unproportionate infringements of the right to informational self-determination (Bundesbeauftragte für den Datenschutz und die Informationsfreiheit, 2024). While the competence of migration authorities to carry out biometric scans of asylum seekers – particularly in relation to internet data – came into force (with substantial changes during the parliamentary process), the legal provisions concerning the federal police were ultimately rejected by the second chamber of federal legislation, the Bundesrat. The opposing conservative party declared that – in their opinion – the legislative changes in the Bundestag, the Sicherheitspaket had been “softened and watered down” (Welt, 2024).

These developments in recent legislation in Germany show: if law enforcement agencies want to implement novel AI tools in areas sensitive to digital human rights, it is vital not to observe the drafting of legal provisions and their technical implementation as something separate. A provision that is intended to permit the use of an AI system that is not precise enough to give legal clarity, but rather leaves the details of implementation largely to the law enforcement agencies themselves, will necessarily get in conflict with the principle of statutory reservation.

The new provision of Section 15b Asylgesetz – with the competence of migration authorities to conduct biometric scans with internet data – will most likely end up in front of the Federal Constitutional Court (Bundesverfassungsgericht) or even the European Court of Justice (for infringement on the ban in Art. 5 lit. e AIA, see for more details section 5 of this article). In the worst-case scenario (from the perspective of law enforcement agencies), they will have by then acquired an (expensive) AI system infringing upon digital human rights, and would need to restart from scratch.

The case study raises the question: Could the instrument of a regulatory sandbox prevent the possibility of implementing digital technologies in the field of law enforcement that bear the risk of falling short of the requirements laid down in the AIA and fundamental rights doctrine?

2.1. Joint regulatory sandboxes in law enforcement and the overall intention of chapter VI

As far as regulatory sandboxes are supposed to open a controlled field of experimentation and testing under the supervision of competent authorities in the “pre-marketing phase” (Recital 139 S. 1 AIA), they can also be interpreted as a vehicle that widens the playbook for public administration on procuring market-ready products that are designed alongside with European values and principles as laid out in the European Charter of Fundamental Rights.Footnote ⁶ Before this background, regulatory sandboxes can open a sweet spot from both the perspective of digital sovereignty and digital human rights: Instead of being inclined to procure “quick fixes” for the intended investigation methods from other regulatory environments (e.g., biometrical scans of suspected persons with internet data with Clearview AI or PimEyes – or cross data-basis analysis and visualization with Palantir products), a regulatory sandbox could bring law enforcement authorities and European SMUs together to develop a tailored solution.

A secondary goal of regulatory sandboxes has a rather introspective angle for institutions of AI governance and legislation: With the instrument, a gap in the policy circle can be closed as the close ties between providers and supervisory authorities can “facilitate regulatory learning for authorities and undertakings, including with a view to future adaptations of the legal framework” (Recital 139 AIA). This intention can be further made fruitful in the context of AI systems for law enforcement agencies. There is a chance to combine the instrument of regulatory sandboxing with funding schemes of research and development projects, with the process of drafting new legislation in police law. Recital 142 AIA points out the “principle of interdisciplinary cooperation between AI developers, experts on inequality and nondiscrimination (…) and digital rights, as well as academics.” This regulatory idea can be realized in the process of shaping AI tools for police agencies that will comply with constitutional law. If the federal and state levels decide to set up a joint regulatory sandbox in the field of law enforcement, they can potentially also include ministries and parliamentarians who draft and decide on legal provisions allowing such AI systems for police forces – regulatory learning could take place on the spot. The legal provision, a suitable AI system fulfilling the intended purpose under the sole control of public authorities, and its factual implementation in police IT infrastructure and investigative methods could be tailored altogether – reducing the risk of lawsuits, disproportionate infringements of digital human rights and legal uncertainty.

2.2. Further processing of personal data acquired by law enforcement authorities (art. 59[2] AIA)

According to Article 57(1) of the AI Act, each Member State must establish at least one regulatory sandbox by August 2nd 2026. This may involve setting up a new sandbox or significantly adapting an existing one to support the development, testing and validation of AI systems. Yet, the regulation is open for all kinds of sectoral (Moraes, Reference Moraes2023) and particularly for more specific regulatory sandboxes on different state levels (Art. 57[2] AIA). Especially, in federal states such as Germany, where law enforcement and policing are largely regulated and carried out on the state and not the federal level, the option of “AI regulatory sandboxes at regional or local level, or established jointly with the competent authorities of other Member States” might open room for specialized and joint regulatory sandboxes in the field of law enforcement. By setting up such regulatory sandboxes, the goal of “evidence-based regulatory learning” (Art. 57[9] lit. d) could be expanded to comparative legal insights as well.

Regulatory sandboxes can be more than a support mechanism for SMEs and a fast lane to successful conformity assessments. This is underlined by a closer look at Art. 59 AIA. Even though the provision raises manifold legal questions on further processing of personal data in regulatory sandboxes, the article focuses only on those parts that are relevant for the field of law enforcement : The provision shows that the EU legislator had law enforcement in mind when drafting Chapter VI – even if only as a subordinate goal. It allows for the further processing of “personal data collected for other purposes for developing certain AI systems in the public interest” (Recital 140[1] AIA). What comes across solely as an authorization for further processing of personal data under data protection law at first sight in Art. 6 (4) GDPR and Art. 5(1)(b) GDPR, opens a door in Art. 59(2) for further processing of personal data for law enforcement authorities that has been acquired for “the prevention, investigation, detection or prosecution of criminal offenses or the execution of criminal penalties, including safeguarding against and preventing threats to public security” in regulatory sandboxes. So, the provision points to law enforcement agencies using their data pools to develop AI systems together with potential providers – from the private or public sector – to enhance their digital capacities and ultimately enable them as deployers. Recital 140(4) AIA underlines that (prospective) providers “should ensure appropriate safeguards and cooperate with the competent authorities, including by following their guidance and acting expeditiously and in good faith to adequately mitigate any identified significant risks to safety, health and fundamental rights.”

Interestingly, though, the corresponding recitals do not specify the scope and effects of Art. 59(2) AIA in the field of law enforcement. Recital 142 rather revolves around other fields of application of Art. 59(1) AIA and explicitly points out “socially and environmentally beneficial outcomes” (Recital 142 p. 1) as use-cases. There is no recital that can help specify the normative scope of Art. 59(2) AIA. A deeper look at legislative history shows that the whole provision was neither mentioned in the AIA’s draft of the European Commission nor in the European Parliaments Mandate, yet was only part of the EU Council’s position. The provision – that most likely goes back to ministries of the interior of national governments – seems to have been added without changes to the final text of the AIA in the – historically long (Bertuzzi, Reference Bertuzzi2023; Waem and Demircan, Reference Waem and Demircan2023) – political trilogue without any further explanation in recitals (den Heijer et al., Reference den Heijer, Abeelen and Maslyka2019). However, the clear structure of the provision makes it possible to interpret it without any further explanation of legislative intent.

Overall, the provision puts a high threshold: the processing of personal data must (1) stay “under the control and responsibility of law enforcement authorities,” (2) “be based on a specific Union or national law” and (3) “subject to the same cumulative conditions as referred to in paragraph 1” which constitutes a total of 10 conditions that must be fulfilled aimed to ensure an alignment with legitimate public interest (see for a full overview Textbox 1 below).

2.3. Can public (law enforcement) authorities play an active part in regulatory sandboxes?

A combined look at Art. 57(2) and Art. 59(2) AIA shows that the EU legislator has foreseen and intended the possibility of setting up joint regulatory sandboxes that involve different state levels (e.g., federal and state level in Germany) where also data sets from law enforcement authorities can be further processed. This opens a wide regulatory space for federal collaboration – to develop, modify or update AI systems in the field of policing, from threat prevention over law enforcement to the execution of criminal offenses.

It is clarified in Art. 58(2)(b) AIA that “providers may also submit applications in partnerships with deployers and other relevant third parties.” In practice, law enforcement agencies can play an active role as (future) deployers of a high-risk AI system (Art. 3[4] AIA). They can also participate as “relevant third party” in cases where the only or most likely deployer of a specific AI system is a police agency. It can be in the interest of the provider of a high-risk AI system to file an application together with a law enforcement agency, without it being clear if a future deployment will take place. Furthermore, Art. 59(1) clearly shows that also “a public authority” can be an active participant in a regulatory sandbox where personal data are processed to “safeguard substantial public interest.” This is underlined by Art. 59(3) AIA which states that the personal data must stay “under the control and responsibility of law enforcement authorities” – how would this be possible without them being involved in the experimental setting and the oversight mechanism of a regulatory sandbox?

Recital 139 S. 5 underlines that “where appropriate,” competent authorities “should cooperate with other relevant authorities, including those supervising the protection of fundamental rights, and could allow for the involvement of other actors within the AI ecosystem,” thus allowing a broad participation of actors who are not directly involved in the product design and marketing of an AI system. However, law enforcement authorities cannot get involved in regulatory sandboxes as authorities supervising the protection of human rights. As deployers of AI-driven surveillance software, they are the object of supervisory activities – and fall short of the necessity to “exercise their duties independently” (Art. 70[1] AIA). According to Art. 74(8) AIA, it is the data protection authorities, “or any other authority designated pursuant to the same conditions laid down in Articles 41 to 44 of Directive (EU) 2016/680” who carry out the task of market surveillance. A police authority cannot be appointed as a national competent authority; it is rather supervised by them while using high-risk AI systems in the field of law enforcement.

According to their respective role, data protection authorities will focus on fundamental rights questions, suggesting privacy-preserving technologies and legal safeguards, while law enforcement agencies will focus on usability and effectiveness of AI systems for carrying out their duties. In an ideal scenario, these opposed views can be mitigated within a regulatory sandbox through dialogue, technical and organizational means and legal safeguards – and lead to AI systems that suit the needs and technical infrastructure of police authority while respecting fundamental rights.