6.4.1 The Blurring Boundaries of Personal Data

The scope of application of the GDPR is firmly dependent on the notion of personal data. As already observed, such a personalistic approach characterises the European legal framework of protection in the field of data. In the algorithmic society, the economic value of Big Data comes from the processing of personal and non-personal data. Therefore, in order to trigger the machine of European data protection law, it is necessary to understand when the link between information and individuals leads to defining data as ‘personal’.

The GDPR only applies to the processing of ‘personal data’ as ‘any information concerning an identified or identifiable natural person’.Footnote ⁷⁷ While the notion of ‘identified natural person’ does not raise particular concerns for defining personal data, the notion of identifiability deserves more attention, especially when artificial intelligence technologies are involved. The GDPR provides a comprehensive approach concerning the identifiability of the data subject which can be identified by ‘all means … which the data controller or a third party can reasonably use to identify said natural person directly or indirectly’.Footnote ⁷⁸ The assessment concerning the reasonableness of these means should be based on objective factors ‘including the costs and the time required for identification, taking into account both the technologies available at the time of treatment and the technological developments’.Footnote ⁷⁹

Within this framework, the ECJ has extensively interpreted the notion of personal data extending its boundaries also to information apparently outside this definition. For instance, in YS,Footnote ⁸⁰ the ECJ clarified that the data relating to an applicant for a residence permit contained in an administrative document, and the data in the legal analysis contained in that document, are personal data, while the analysis per se cannot be considered within this notion. Likewise, in Digital Rights Ireland,Footnote ⁸¹ the ECJ recognised the relevance of metadata as personal data since they could make it possible ‘to know the identity of the person with whom a subscriber or registered user has communicated and by what means, and to identify the time of the communication as well as the place from which that communication took place’.Footnote ⁸² Therefore, the ECJ extended the notion of personal data considering also the risk of identification deriving from the processing of certain information.

The same approach was adopted in Breyer.Footnote ⁸³ The dispute concerned the processing and storing of dynamic IP addresses of visitors to institutional websites by the German federal institutions to prevent cyber-attacks. The domestic court asked the ECJ whether the notion of personal data also included an IP address which an online media service provider stores if a third party (an access provider) has the additional knowledge required to identify the data subject. In Scarlet,Footnote ⁸⁴ the ECJ had already found that static IP addresses should be considered personal data since they allow users to be identified. In this case, the attention is on dynamic IP addresses that cannot independently reveal the identity of a subject as they are provisional and assigned to each Internet connection and replaced in the event of other accesses. Therefore, the primary question focused on understanding whether the German administration, as the provider of the website, was in possession of additional information that would allow the identification of the user. The ECJ identified such means in the legal instruments allowing the service provider to contact, precisely in case of cyber-attacks, the competent authority, so that the latter takes the necessary steps to obtain this information from the former to initiate criminal proceedings. As a result, firstly, this case shows that, for the purpose of the notion of personal data, it is not necessary that information allows the identification of the data subject per se. Secondly, the information allowing identification could not be in the possession of a single entity.

The ECJ addressed another case enlarging the scope of the notion of personal data in Novak.Footnote ⁸⁵ The case concerned the Irish personal data authority’s refusal to guarantee access to the corrected copy of an examination test due to the fact that the information contained therein did not constitute personal data. After reiterating that the notion of personal data includes any information concerning an identified or identifiable natural person, the ECJ observed that, in order to answer the question raised by the national court, it is necessary to verify whether the written answers provided by the candidate during the examination and any notes by the examiner relating to them constitute information falling within the notion of personal data. The ECJ observed that the content of those answers reflects the extent of the candidate’s knowledge and competence in a given field and, in some cases, his intellect, thought processes and judgment as well as graphological information. The collection of these responses also has the function of assessing the candidate’s professional skills and their suitability to exercise the profession in question. Finally, the use of such information, which translates into the success or failure of the candidate for the exam in question, can have an effect on their rights and interests, as it can determine or influence, for example, their ability to access the desired profession or job. Likewise, with regard to the examiner’s corrections, the content of these annotations reflects the examiner’s opinion or evaluation on the candidate’s individual performance during the examination, and, precisely, on their knowledge and skills in the field in question. Together with Breyer, this case shows an extensive approach to the notion of personal data with the result that it is not possible to foresee in any case when information should be considered ‘personal’ but it is necessary to examine the context through a case-by-case analysis.

In the algorithmic society, this overall picture would lead to consider a fortiori how the dichotomy between personal and non-personal data looks less meaningful. Even if the processing of personal data through artificial intelligence technologies does not always involve personal data such as, for example, climatic and meteorological data, the potentiality of artificial intelligence technologies to find correlation through a mix of related and unrelated as well as personal and non-personal data, broadens the cases in which the scope of application of the GDPR covers the processing of information which would not fall within the notion of personal data at first glance. For instance, big data analytics aims to identify correlations based on originally unrelated data.Footnote ⁸⁶ It is the processing of different types of data that could lead to discovering or redefining data or information as personal.Footnote ⁸⁷ Therefore, it could be impossible to find information that cannot be potentially transformed into personal data,Footnote ⁸⁸ precisely because the economic value of Big Data encourage to process vast amounts of personal and non-personal data.

This consideration could be extended even to the process of anonymisation of personal data. The GDPR does not apply to anonymous data or information that does not refer to an identified or identifiable natural person or to personal data made sufficiently anonymous to prevent or disallow the identification of the data subject. Consequently, anonymised data would not fall within the scope of application of the GDPR. However, it could be easy to define the cases in which the anonymisation process is not reversible or apparently anonymous data are instead personal when mixed with other information. Therefore, there is no single definition of anonymous data, but this notion should be considered in the framework in which the data controller operates, taking into account ‘all objective factors, such as the costs of and the amount of time required for identification, taking into consideration the available technology at the time of the processing and technological developments’.Footnote ⁸⁹

The primary criterion to assess whether data are anonymous comes from a mix of factors and refers to the reasonable usability of the available means to reverse the process of anonymisation referring precisely to ‘all the means reasonably likely to be used, such as singling out, either by the controller or by another person to identify the natural person directly or indirectly’.Footnote ⁹⁰ According to Finck and Pallas, this complexity is linked both to technical and legal factors. On the one hand, ‘[f]rom a technical perspective, the increasing availability of data points as well as the continuing sophistication of data analysis algorithms and performant hardware makes it easier to link datasets and infer personal information from ostensibly non-personal data’. On the other hand, ‘[f]rom a legal perspective, it is at present not obvious what the correct legal test is that should be applied to categorise data under the GDPR.Footnote ⁹¹

Therefore, even data that would lead to the identification of individuals could be considered anonymous, due to the absence of reasonable means to obtain personal data from that information. Nonetheless, as underlined by Stalla-Bourdillon and Knight, the approach to anonymisation would be idealistic and impractical.Footnote ⁹² This is because the phase of analytics plays a crucial role in the anonymisation of personal data. It is possible to observe how the quantity and quality of elements identifying personal data influence the number of resources needed for anonymisation. There is a point where the resources available no longer allow the identification due to the number of data to be anonymised. The anonymisation process is effective when it can prevent anyone using reasonable means from obtaining personal data from anonymised data consisting of irreversible de-identification.Footnote ⁹³ According to the WP29, ‘the outcome of anonymisation as a technique applied to personal data should be, in the current state of technology, as permanent as erasure, i.e. making it impossible to process personal data’.Footnote ⁹⁴ The concept of anonymous data still creates ‘the illusion of a definitive and permanent contour that clearly delineates the scope of data protection laws’.Footnote ⁹⁵ Anonymising data could not mean that we are not dealing with personal data any longer. Even when the data controller makes it almost impossible to identify the data subject, evidence shows that the risk of re-identification is concrete.Footnote ⁹⁶ The WP29 has already underlined that the advance of new technologies makes anonymisation increasingly difficult to achieve. Researchers have underlined the fallacies of anonymisation in different fields,Footnote ⁹⁷ especially when Big Data analytics are involved.Footnote ⁹⁸

Furthermore, even when focusing on pseudonymisation, the GDPR still applies.Footnote ⁹⁹ Pseudonymisation consists of ‘the processing of personal data so that personal data can no longer be attributed to a specific data subject without the use of additional information, provided that such additional information is stored separately and subject to technical and organizational measures intended to ensure that such personal data is not attributed to an identified or identifiable natural person’.Footnote ¹⁰⁰ The GDPR explicitly promotes the use of this technique as a risk-management measure but not as an exception to its scope of application. Unlike anonymisation, the data controller can reverse pseudonymised data, and this is why this information falls within the scope of personal data.

Pseudonymisation consists just of the replacement of data with equally univocal, but not immediately, intelligible information. Therefore, on the one hand, as long as data can be considered anonymous, this information can be processed freely by using Big Data analytics techniques, provided that, as already underlined, the processing does not lead to the identification of the data subject. On the other hand, in the case of pseudonymisation, the discipline of the GDPR applies and, as a result, the data controller is responsible for assessing the risks of this processing and relying on the appropriate legal basis. Furthermore, even if it cannot be excluded that, in some cases, pseudonymised data could be close to the notion of anonymity, they could fall under the processing of the GDPR allowing the data controller not to maintain, acquire or process additional information if the purposes for which a controller processes personal data do not or do no longer require the identification of data subjects.Footnote ¹⁰¹

Therefore, on the one hand, the GDPR would increase the protection of data subjects by extending the scope of the notion of personal data. The more the notion of personal data is broadly interpreted, the more the processing of data through artificial intelligence technologies falls under data protection laws and, therefore, the processing of information through these technologies is subject to the GDPR’s safeguards. However, the impossibility to foresee when this technique could lead to the reidentification of data undermines legal certainty, thus constituting a brake to the development of digital technologies in the internal market.

6.4.2 Clashing General Principles

The implementation of artificial intelligence technologies to process personal information does not just contribute to blurring the gap between non-personal and personal data but also to broadly challenge the general principles governing the GDPR. Once information falls within the category of personal data, the relationship between the GDPR and algorithmic processing is far from being exhausted. The challenges concern not only the scope of application of European data protection law but also its founding principles. It would be enough to look at the Charter underlining that ‘data must be processed fairly for specified purposes and on the basis of the consent of the person concerned or some other legitimate basis laid down by law’.Footnote ¹⁰² Together with other grounding values, the GDPR has introduced these principles representing the expression of the constitutional dimension of privacy and data protection as fundamental rights of the Union.

The GDPR’s general principles can be considered the horizontal translation of constitutional values guiding data controllers when ensuring the compliance with data protection rules and the protection of the data subject’s rights. General principles play a crucial role in avoiding that the processing of personal data leads to serious interference with the data subjects’ fundamental rights. At the same time, they constitute axiological limits to the exercise of powers based on the discretionary processing of personal data.

Generally, the analysis of large quantities of data through opaque processing leading to outputs that are not always predictable are just some elements to consider when assessing the compatibility of Big Data analytics with the general principles of European data protection law. Such a multifaceted analysis of data for multiple purposes raises serious concerns about, but not limited to, the principles of lawfulness, fairness and transparency. These principles require natural persons to be made ‘aware of risks, rules, safeguards and rights in relation to the processing of personal data and how to exercise their rights in relation to such processing’.Footnote ¹⁰³ The obligations for the data controller to inform data subjects about the processing of their personal data,Footnote ¹⁰⁴ or the legal basis for processing personal data, are just two examples expressing (or implementing) the general principles.Footnote ¹⁰⁵ As observed by Gutwirth and De Hert, while the right to privacy is an instrument of opacity for the protection of the individual, data protection plays the role of a transparency tool.Footnote ¹⁰⁶

These principles are challenged by algorithmic processings whose decision-making processes are often opaque.Footnote ¹⁰⁷ These techniques do not always allow to explain to data subjects the consequences of processing their personal data through such systems. For example, Big Data analytics often involve the re-use of data and lead to the creation of other information through inferences.Footnote ¹⁰⁸ Therefore, it would not always be possible to predict from the beginning all the types of data processed and potential uses.Footnote ¹⁰⁹ Therefore, the process of mandatory disclosure required by the GDPR would de facto fail before the characteristics of these technologies. It is no coincidence that Richard and King have defined this situation as a ‘transparency paradox’.Footnote ¹¹⁰ On the one hand, Big Data analytics promise new levels of knowledge by defining models and predictions. On the other, the mechanisms by which these systems reach a new degree of knowledge are obscure. In other words, the price to access more knowledge is accepting a certain degree of data ignorance.

The information asymmetry between the data subject and data controller leads to questioning not only the principle of transparency but also those of lawfulness and fairness. The lack of transparency in the processing may not always allow the data subject to express a valid consent.Footnote ¹¹¹ Artificial intelligence technologies challenge how data subjects express their free and informed consent. In this situation, where the data controller cannot explain the potential use of data transparently, the data subject is not aware of the risks when giving their consent to access products and services. Such information asymmetry is even more problematic when the data subject needs, for example, to access public services which are provided by a data controller or the data controller in a position of monopoly or oligopoly. According to the GDPR, the legal basis of consent should not be valid for processing personal data where there is a clear imbalance between the data subject and the data controller.Footnote ¹¹²

Besides, the principle of lawfulness is undermined not only by the low level of transparency in the field of artificial intelligence but also by how information about the processing of personal data is shared with data subjects through privacy policies. This issue is not only relating to the use of long and complex explanations about the processing of personal data undermining de facto the possibility for data subjects to really understand how their personal data are used and for which purposes.Footnote ¹¹³ Another primary issue concerns the spread of daily life applications (i.e. Internet of Things) collecting personal data in public and private places without the awareness of data subjects.Footnote ¹¹⁴ The strict rules to obtain consent and the burden of proof can prevent discretionary determinations over personal data but also encourage data controllers to rely on other legal bases beyond consent.Footnote ¹¹⁵

This trend could be problematic for the principle of lawfulness also because the legal bases for the processing of personal data do not apply when the data controller processes particular categories of data, namely ‘those personal data that reveal racial or ethnic origin, political opinions, religious beliefs or philosophical, or union membership, as well as genetic data, biometric data intended to uniquely identify a natural person, data relating to the health or sexual life or sexual orientation of the person’.Footnote ¹¹⁶ As already observed, the analysis of a vast amount of data from heterogeneous datasets can lead to the discovering of new data (i.e. inferences) which could require a different legal basis to process them.Footnote ¹¹⁷

In the algorithmic society, the rationale behind the distinction between ‘ordinary’ and ‘particular’ categories of data tends to be nullified by the way in which the data are processed for at least two reasons. Firstly, Big Data analytics are based on a high volume of structured and unstructured data, which usually do not rely on the distinction between categories of data. Secondly, data on health, race or sexual orientation can be obtained from the processing of unstructured data. For example, the content of a social network account can reveal health or racial origin data that inevitably become part of the analysis process that leads to profiling or an automated decision. In other words, even non-particular categories of data can constitute a vehicle for the deduction of information of a particular nature. As noted by Zarsky, ‘the rise of big data substantially undermines the logic and utility of applying a separate and expansive legal regime to special categories’.Footnote ¹¹⁸

Such a consideration also shows how artificial intelligence technologies challenge the principle of purpose limitation, precisely due to the multiple and unpredictable re-use of data.Footnote ¹¹⁹ It would not be by chance if the WP29 focused on the need to respect this principle in the field of Big Data by ensuring that the purposes for which the data is processed can be known or foreseen by the data subjects.Footnote ¹²⁰ In order to comply with the principle of purpose limitation, it is necessary to inform the data subject of the processings whose purposes differ from the initial ones at the time of data collection and analysis. Therefore, the aim of this principle is to protect data subjects against the unforeseeable extension of processing purposes. The general use of Big Data analytics implies that data is not just held and used by a certain and predetermined number of third parties for a specific purpose. On the contrary, as observed by Mittelstadt, data ‘travels with the person between systems and affects future opportunities and treatment at the hands of others’.Footnote ¹²¹

Besides, the relevance of the principle of purpose limitation deserves to be examined not only by looking at the protection of data subjects’ rights but also by considering the effects that such a principle can produce on the internal market. It could constitute a barrier to the development of monopolies and dominant situations in the context of data analysis by limiting the possibility for data controllers to use data for any contingent purpose. Nevertheless, as Hildebrandt observed, a narrow interpretation of this principle could limit the potentialities of analytics which, usually, rely on creating models and previsions based on unrelated data and purposes.Footnote ¹²² The principle of purpose limitation can indeed constitute a barrier to data-driven innovation, especially for data sharing. However, what is defined as ‘purpose limitation’ could be more precisely described as ‘non-incompatibility’.Footnote ¹²³ Since it is not possible in some cases to foresee all the potential uses, the principle of purpose limitation would apply only in relation to that processing which is incompatible with those disclosed to the data subject.

Nonetheless, the challenges to the principles of transparency, lawfulness and fairness do not exhaust the concerns about the relationship between algorithmic technologies and the GDPR’s general principles. The collection and analysis of vast amounts of data can affect the principle of data minimisation. Bygrave has described this principle as an instrument to ensure proportionality and necessity without exceeding the quantity of data to be processed.Footnote ¹²⁴ Unlike the processing of data through analogical means, new automated processing techniques allow extracting value even from apparently unrelated data. This feature has been facilitated by the possibility of storing and analysing increasing amounts of data according to the so-called ‘N = all’ model according to which the collection and analysis of information are not based just on relevant data but on the whole.Footnote ¹²⁵ The processing and accumulation of a vast amount of data also threaten the principles of integrity and confidentiality due to the increasing risks in handling large volumes of information to be managed.Footnote ¹²⁶ The more data are processed and stored, the more the risk of facing serious data breaches will be amplified. Likewise, the trend towards data accumulation could also clash with the principle of data retention and security.Footnote ¹²⁷ Dealing with large amounts of data processed for multiple purposes could make retention policies complex to implement and security measures subject to increasing layers of risks because of the amount of information involved.

Likewise, the principle of accuracy also plays a primary role because the result of automated decision-making is strongly influenced by the quality of data. Data mining techniques rely on various sources such as social media and other third-party sources that are known for not always being accurate. The pluralism of data sources increases the risk of dealing with inaccurate data.Footnote ¹²⁸ This problem does not only occur ex ante when collecting and analysing data but also ex post due to the distorted effects that inaccurate data can have on the outputs.Footnote ¹²⁹ According to Tene and Polonetsky, ‘in a big data world, what calls for scrutiny is often the accuracy of the raw data but rather the accuracy of the inferences drawn from the data’.Footnote ¹³⁰

All these principles should be read in light of the principle of the data controller’s accountability, which is the ground upon which the GDPR’s risk-based approach is built. The data controller should be able to prove compliance with general principles. The meaning of the principle of accountability can be better understood when focusing on the dynamic definition of the controller’s responsibility based on the nature, scope, context and purposes of processing as well as the risks of varying likelihood and severity for the rights and freedoms of natural persons.Footnote ¹³¹

On this basis, the data controller is required to implement appropriate technical and organisational measures to guarantee, and be able to demonstrate, that the processing is carried out in accordance with the GDPR and, especially, its principles. According to the principle of privacy by design and by default,Footnote ¹³² the data controller is required to set adequate technical and organisational measures, such as pseudonymisation, to implement the principles of data protection effectively and to provide the necessary guarantees by design and ensure that, by default, only the personal data necessary for each specific purpose are processed. For example, as far as the principles of transparency or purpose limitation are concerned, data processing should allow the data subject to be aware of the modality of processing even when artificial intelligence technologies are involved, thus requiring these technologies to take into consideration the requirement established by the GDPR. In other words, these principles would require data controllers to ensure ex ante that the implementation of technologies processing personal data complies with the general principles of European data protection law. However, there is a tension with general principles when data controllers rely on algorithmic technologies to process personal data.

These considerations could be enough to explain the clash between artificial intelligence and European data protection. Nevertheless, the implementation of algorithmic technologies for processing personal data is also relevant for the protection of data subjects’ rights, precisely when these systems lead to significant legal effects on their rights and freedoms.

6.4.3 The Freedom from Algorithmic Processing

One of the primary constitutional challenges for privacy and data protection in the age of Big Data consists exactly of dealing with the lack of transparency and accountability in automated decision-making processes and their effects on individual fundamental rights and freedoms as well as democratic values. As already stressed, the involvement of algorithmic processing for purposes of profiling and automated decision-making challenges privacy and data protection.Footnote ¹³³

Automated decision-making could be defined as the process of making decisions without human intervention. According to the GDPR, this process consists of a decision based solely on automated processing.Footnote ¹³⁴ Usually, these processes involve the use of artificial intelligence technologies. These techniques can lead to binding decisions also depriving individuals of legal rights such as accessing credit.Footnote ¹³⁵ It is in this case that the GDPR aims to introduce safeguards to protect individuals against the discretionary use of personal data for purposes of automated decision-making. In order to empower data subjects to maintain control over their data and mitigate the asymmetry between the data controller and subject, the GDPR provides the so-called data subjects’ rights.Footnote ¹³⁶

The GDPR is particularly concerned by profiling which consists of ‘any form of automated processing of personal data consisting in the use of such personal data to evaluate certain personal aspects relating to a natural person, precisely, to analyse or foresee aspects concerning professional performance, the situation economic, personal health, preferences, interests, reliability, behaviour, location or movements’.Footnote ¹³⁷ Against such processing, the data subject has the right to object at any time, for reasons connected with their particular situation. However, this right is not absolute since it can only be exercised when the processing is necessary for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller,Footnote ¹³⁸ or for the purposes of the legitimate interests pursued by the controller or by a third party, except where such interests are overridden by the interests or fundamental rights and freedoms of the data subject.Footnote ¹³⁹ Therefore, the scope of such a right is narrow and does not apply when profiling occurs based on the consent of the data subject or any other legal basis provided for by the GDPR.

Once the right to object has been exercised, the data controller cannot process personal data unless it demonstrates the existence of legitimate reasons prevailing over the interests, rights and freedoms of the interested party or to ascertain, exercise or defend a right in court. Furthermore, if personal data is processed for direct marketing purposes, the data subject has the right to object at any time to the processing of personal data for these purposes, including profiling. In both cases, the data controller is explicitly required to present this information clearly and separately from any other information at the time of the first communication with the data subject.

Such a right aims to empower users who can complain about the processing of their personal data when it is made by a public authority or it is the result of the choice of data controllers to rely on the legitimate interests as a legal basis of the processing, which, in any case, needs to balance the interest of the controller with the fundamental rights of the data subject. In this case, the right to object allows users to intervene in this balancing which, otherwise, would be left in the hands of data controllers. In this case, the right to object protects data subjects against profiling by artificial intelligence technologies, even if the scope of this right is narrow.

Together with this safeguard, under the GDPR, individuals can rely on their right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects that concern him or her, or that significantly affects his or her person.Footnote ¹⁴⁰ The WP29 has clarified that the reference to the expression ‘right’ not to be subject to a decision based exclusively on automated processing does not imply that this guarantee only applies when the subject invokes this right, since ‘individuals are automatically protected from the potential effects this type of processing may have’.Footnote ¹⁴¹ As pointed out by Mendoza and Bygrave, it is more appropriate to think of this safeguard as a prohibition rather than a right.Footnote ¹⁴² In this context, the principle of transparency would require the data controller to provide information to the data subject ‘on the logic used, as well as the importance and the expected consequences of this treatment for the data subject’, regardless of whether the data is collected by the data subject,Footnote ¹⁴³ in line with the spirit of the GDPR which requires a high level of transparency in the processing of personal data.

By arguing a contrario, the lack of such a right would produce negative effects not only for individuals but also for democratic values since it would leave data controllers to fully rely on artificial intelligence technologies to make decisions affecting the rights of data subjects without providing any safeguards such as transparency and accountability for these outcomes. The lack of these safeguards is particularly evident when looking, for instance, at the framework of content moderation as examined in Chapter 5. This freedom can be considered as the positive translation of constitutional rights within the legal regimes of data protection and, therefore, it applies to private actors without the need to rely on the horizontal application of fundamental rights. In this sense, the right not to be subject solely to automated decision-making processes increases the possibility for data subjects to receive information about the automated decisions involving them and, therefore, fosters the level of transparency and accountability.

Therefore, even if the relevance of this right within the framework of the GDPR is clear, the remaining question concerns the degree of transparency which the data controller should ensure. According to the GDPR, the data controller should provide meaningful information about the logics involved in the decision-making process.Footnote ¹⁴⁴ In order to ensure transparency and fairness, these logics should take into account the circumstances and context of the processing, implementing appropriate mathematical or statistical procedures for the profiling, technical and organisational measures appropriate to minimise errors and inaccuracies, as well as safe procedures for personal data to prevent, inter alia, discriminatory effects.Footnote ¹⁴⁵

The right not to be subject solely to automated decision-making has triggered a debate on whether the GDPR provides an effective legal basis for data subjects to avoid potentially harmful consequences deriving from the implementation of algorithms, most notably by relying on a ‘right to explanation’ in respect of automated decision-making processes.Footnote ¹⁴⁶ Some argue that the GDPR introduces it.Footnote ¹⁴⁷ Others underline that such a right fosters qualified transparency over algorithmic decision-making,Footnote ¹⁴⁸ deny the existence of such a right,Footnote ¹⁴⁹ or doubt that the GDPR provisions provide a concrete remedy to algorithmic decision-making processes.Footnote ¹⁵⁰

It is not by chance that transparency is one of the most debated issues when focusing on algorithmic technologies.Footnote ¹⁵¹ The threats to individuals are intimately, even if not exclusively, connected with the impossibility to ensure transparent outcomes of automated decision-making processes.Footnote ¹⁵² Despite the criticisms of the process of mandatory disclosure,Footnote ¹⁵³ these obligations constitute a first step to mitigate the asymmetries between data subjects and data controllers. The GDPR aims to empower data subjects by mitigating the technical opacity of automated decision-making.Footnote ¹⁵⁴ The data controller should not only disclose the data used and the purposes of the processing, but it has also the duty to inform the data subjects about the use of automated decision-making and explain the logic of this process. These safeguards constitute a shield against potential predetermined and discretionary decisions against which the data subject would not have any remedy.

A further guarantee for data subjects against automated decision-making is provided by the limitation to the processing of particular categories of data provided for by the GDPR, without prejudice to the cases of explicit consent of the data subject and if the processing is necessary for reasons of significant public interest on the basis of Union or Member State law, which must be proportionate to the aim pursued, respect the essence of the right to data protection and provide for appropriate and specific measures to protect the fundamental rights and interests of the data subject.Footnote ¹⁵⁵ In the field of Big Data analytics, profiling aims to create clusters of individuals based on their characteristics. Often, processing telephone numbers or names and surnames would not be enough to develop predictive models since profiling focuses on the individual characteristics which constitute particular categories of data such as health information, political ideas or even biometric data. Even in these cases, adequate measures have to be in force to protect the rights, freedoms and legitimate interests of the data subject.

Nevertheless, this data subjects’ right is not absolute. The general notion of ‘legal or similarly significant effects’ limits its general applicability.Footnote ¹⁵⁶ The WP29 has also specified that this freedom applies just in cases of ‘serious impactful effects’ and when the automated decision could ‘significantly affect the circumstances, behaviour or choices of the individuals concerned; have a prolonged or permanent impact on the data subject; or at its most extreme, lead to the exclusion or discrimination of individuals’.Footnote ¹⁵⁷ For example, this provision would apply when the data subject is applying for a credit card as well as accessing to education or health services.

Moreover, several exceptions limit the scope of data protection safeguards. Unlike the case of the notion of personal data and general principles, the GDPR provides a clearer set of exceptions to the application of this data subjects’ right against automated decision-making processes. This liberty does not apply when the automated decision is necessary for the conclusion or execution of a contract between the interested party and a data controller as well as when it is authorised by Union or Member State law to which the data controller is subject, which also specifies appropriate measures to protect the rights, freedoms and legitimate interests of the data subject. Moreover, this safeguard also does not apply when the processing is based on the explicit consent of the data subject. However, when the processing is based on a contract or the explicit consent of the data subject, the data controller is required to implement suitable measures to safeguard the data subject’s rights and freedoms and legitimate interests. In this case, this prohibition turns into a right when the GDPR recognises that the data subject should at least have the right to obtain human intervention on the part of the controller, to express his or her point of view and to contest the decision. This data subject’s safeguard cannot lead to ‘fabricating human involvement’ since human involvement and oversight should be meaningful.

Furthermore, the data controller may limit the boundary of the right to explanation by invoking its interest to protect the trade secrets and intellectual property rights,Footnote ¹⁵⁸ or, more generally, its freedom of economic initiative that would be frustrated by complying with transparency obligations requiring unreasonable resources.Footnote ¹⁵⁹ For instance, when the techniques of data analysis through machine learning are involved, it is possible to highlight the so-called black box effect consisting of the impossibility to reconstruct the steps from the beginning of the processing up to the final output.Footnote ¹⁶⁰ Bathaee underlined that this issue ‘poses an immediate threat to intent and causation tests that appear in virtually every field of law’.Footnote ¹⁶¹

This scenario is made even more opaque and fragmented by the limits that Member States establish to these data subjects’ rights.Footnote ¹⁶² Member States can restrict such rights to the extent that limitations are established by EU law or the Member State, provided that this restriction respects the essence of fundamental rights and freedoms and a necessary and proportionate measure in a democratic society to safeguard interests such as, for example, national security.Footnote ¹⁶³ Therefore, on the one hand, the rights to data subjects against automated processing can mitigate the interferences coming from processing of personal data through algorithmic technologies. On the other hand, the scope of these rights could undermine the concrete enforcement of this safeguard, thus increasing the possibility for data controllers to rely on automated decision-making technologies to process personal data. Besides, the lack of legal certainty around the scope of this safeguard could also affect the consistent application of this safeguard as also demonstrated by the complexity of multi-state profiling.Footnote ¹⁶⁴

Within this framework, the challenges raised by automated decision-making processes are another example of the clash between algorithmic technologies and the protection of fundamental rights and democratic values. This case is another example of how European digital constitutionalism is called to reframe the role of European data protection in the algorithmic society.

6.5.1 Recentring Human Dignity

The evolution of the algorithmic society has contributed to underlining the relevance of data as a personal piece of information. Increasingly, public and private actors rely on machines to make decisions on individual rights and freedoms based on the processing of data. While public actors trust algorithmic technologies to improve public services and perform public tasks such as biometric surveillance, private actors implement automated decision-making to process data to attract revenues following the logic of digital capitalism.Footnote ¹⁶⁹ Within this framework, as underlined by Gutwirth and De Hert, ‘humans have become detectable, (re)traceable and correlatable’.Footnote ¹⁷⁰ Personal data disseminated in daily lives are raw materials for artificial intelligence systems which then are trained to cluster this data based on correlation. Nonetheless, since, in the age of Big Data, even generic pieces of information could be considered personal, clustering data also mean profiling individuals.

In Europe, personal data are ‘personal’ since they are connected to the individual. This focus is not only because the notion of personal data extends far beyond the notion of identified natural persons but also because data protection law without personal data would lose its constitutional meaning within the European framework. It is not by chance that the scope of the GDPR does not extend to legal persons or deceased,Footnote ¹⁷¹ or non-personal data.Footnote ¹⁷² This characteristic underlines how, in Europe, personal data are not only relevant for the circulation of information or the extraction of value. As stressed in Chapter 3, the rise of European digital constitutionalism has shed light on the constitutional dimension of privacy and data protection complementing the internal market goals.

This constitutional framework is the reason why personal data cannot be seen just as an object of property rights but also as data ‘extra commercium’.Footnote ¹⁷³ The ‘propertisation’ of personal data contributes to their commodification under the logic of digital capitalism with the result that any data would be considered as tradable as goods and not as a piece of individual identity. It is true that the circulation and exchange of personal data constitute the pillars of the algorithmic society. Nonetheless, the unaccountable and discretionary commodification of personal data would lead to considering consumer protection or contract law as the primary instrument to deal with the commercial exploitation of data.Footnote ¹⁷⁴ However, these concurring regimes would fail to protect personal data as an expression of the individual and, therefore, this is also why personal data ‘cannot be considered as a commodity’.Footnote ¹⁷⁵ Likewise, the EDPS has underlined that personal data cannot be conceived as mere economic assets.Footnote ¹⁷⁶ As Floridi underlined, ‘“My” in my data is not the same as “my” in my car, but it is the same as “my” in my hand’.Footnote ¹⁷⁷ Therefore, protecting the right to privacy should be considered as a matter of personal identity and integrity since it determines the evolution of human personality and therefore of human dignity. In a different way, the right to be forgotten exactly showed this face of the right to privacy even before the rise of online platforms, and the Google Spain case.Footnote ¹⁷⁸

Even if human dignity is almost invisible in the GDPR,Footnote ¹⁷⁹ the human-centric approach in European data protection law comes from the ability of human dignity to permeate in the core of European fundamental rights.Footnote ¹⁸⁰ The Charter opens up the catalogue of rights stating ‘human dignity is inviolable. It must be respected and protected’.Footnote ¹⁸¹ The central position of this value within the Charter is not a formal recognition of constitutionality,Footnote ¹⁸² but it plays the role of a pillar for the entire system of fundamental rights. This approach mirrors the Universal Declaration of Human Rights which enshrines human dignity in its preamble.Footnote ¹⁸³ Therefore, as stressed in Chapter 1, human dignity should not be seen as a clashing value but as the core of each fundamental right laid down in the Charter. Human dignity therefore is a necessary piece of the puzzle to be considered and safeguarded in the balancing process. This is part of the European constitutional roots which look at dignity as the pillar against any human annihilation.

Therefore, the mission of data protection law would be to ensure that its human imprinting does not fall apart while ensuring democratic values of transparency and accountability. Even in this case, the role of dignity could be considered as a primary trigger for the consolidation of data protection as the positive dimension of the right to privacy, similarly to how human dignity could contribute to fostering the positive dimension of the right to freedom of expression to address the challenges of content moderation as examined in Chapter 5. According to the EDPS, ‘[p]rivacy is an integral part of human dignity, and the right to data protection was originally conceived in the 1970s and 80s as a way of compensating the potential for the erosion of privacy and dignity through large scale personal data processing’.Footnote ¹⁸⁴

The notion of personal data is not the only point showing the role of human dignity in the GDPR. Individual consent is the primary pillar of European data protection law, thus representing the centrality of individual self-determination.Footnote ¹⁸⁵ As mentioned in Chapter 1, the first decision of the German Constitutional Court on data protection has shed the light on the role of dignity in the processing of personal data. It is indeed the autonomous choice of the data subject which would allow the data controller to legally process personal data. This is why, even if imbalances of power question the meaning of consent in European data protection law, the GDPR still focus on consent as a primary legal basis defined as ‘any freely given, specific, informed and unambiguous indication of the data subject’s wishes by which he or she, by a statement or by a clear affirmative action, signifies agreement to the processing of personal data relating to him or her’.Footnote ¹⁸⁶

Likewise, the distinction between personal data and particular categories of data provides another clue about the human-centric approach of European data protection law. This double track of protection for personal data aims to protect personal information which can reveal intimate aspects of human lives. Such a difference, already introduced in the Data Protection Directive, has been fostered by the GDPR which has not only extended the categories of data falling under the scope of such a special regime but also provides a general ban of the processing of this type of data even though it foresees conditions of lawfulness as exceptions.Footnote ¹⁸⁷ For instance, biometrics and DNA data have been included within the broader protection of particular categories of data, being it information able to represent humans as they are. Precisely, in a phase where biometric technologies are expanding and intertwining with artificial intelligence to pursue different tasks,Footnote ¹⁸⁸ such a safeguard reflects the need to avoid that personal data are subject to automated decisions without the ‘explicit consent’ of data subjects. In this case, it is not enough to rely on the conditions for processing personal data, but it is necessary to ground the processing on specific legal bases.Footnote ¹⁸⁹ Even in this case, the core of the entire system is the data subject’s consent, which, in this case, has to be ‘explicit’.

Such a personalistic approach also affects the framework of automated decision-making processing. The GDPR does not expressly clarify the constitutional values underpinning its structure. Therefore, a literal or systemic interpretation of data protection law could not provide a full picture of the values which the prohibition to subject individuals to these systems would protect. Dreyer and Schulz have underlined that the goal of this rule is beyond the mere protection of personal data.Footnote ¹⁹⁰ Even if not exclusively, the primary goal of this rule is the protection of human dignity. The right not to be subject to automated decision-making deals with the ability of machines to make determinations about human lives. Even in this case, the rise of the Internet has underlined how digital technologies can perform activities in a more efficient way than humans. The same is true for algorithmic technologies that are able to see correlations that humans do not perceive, or predict the future which is one of the abilities that humans have always tried to reach.

What does not actually change is the risk of error. Even if machines were more efficient than humans, they could still fail and reproduce the biases of their programmers. At first glance, algorithms would appear as neutral technologies which can extract values from information that are useful for businesses and society. However, from a technical perspective, algorithms are far from being neutral. They are not just mathematical models providing outcomes in a certain form based on the processing of information.Footnote ¹⁹¹ Algorithms transform inputs into outputs, thus expressing a value judgement. Automated decision-making systems are therefore value-laden.Footnote ¹⁹² The human role in the programming and development of these technologies contributes to reflecting the biases and values of programmers into the technological design.Footnote ¹⁹³ This issue is not a novelty since all technologies are the result of certain design choices. Reidenberg and Lessig have already clarified how much the architecture of technology is a critical piece of the regulatory jigsaw.Footnote ¹⁹⁴ In the case of algorithms, the role of design is even more critical since these technologies can produce decisions on which humans ground their activities, or even largely rely.Footnote ¹⁹⁵

Besides, machines are still not entirely able to interpret real dynamics and exactly understand contexts and emotions,Footnote ¹⁹⁶ or translating legal concept into machine determinations.Footnote ¹⁹⁷ This limit also explains why so frequently the implementation of artificial intelligence technologies has led to discrimination.Footnote ¹⁹⁸ The right to equality can be considered another expression of human dignity. Without being considered equal, there are multiple layers of protection for different categories of ‘humans’. The right to non-discrimination is one of the fundamental principles of European constitutional law. The right to equality is the basic pillar of democratic constitutionalism as shown by its relevance in the Charter and the Convention.Footnote ¹⁹⁹ Discriminatory outcomes of algorithmic processing can originate from the low level of data quality or embedded bias in the programming phase like in the case of discrimination based on ethnicity.Footnote ²⁰⁰

Therefore, the GDPR shields data subjects against the interference to their legal rights coming from the errors automated decision-making can produce. This prohibition recognises that machines cannot be fully trusted. In other words, such a rule clarifies that efficiency cannot prevail over fundamental rights and freedoms. At the same time, artificial intelligence technologies can also foster fundamental rights, thus allowing humans to escape from paths of marginalisation. Even in this case, the GDPR has not introduced a general ban for this type of processing but has tried to limit the serious effects that these technologies can produce on data subjects. Likewise, the GDPR has introduced the so-called human-in-the-loop principle to ensure that human decisions are not affected by decisions taken just by unaccountable systems. This approach is firmly connected with the acknowledgement that machines err and are (still) not able to distinguish the complexity of human lives. The attempts to digitise human lives to a mere calculation would annihilate the role of humans, leading towards a process of dehumanisation. In other words, the human being is dignus. Any attempt to digitise humanity would clash with the nature of human beings.

Within this framework, human dignity constitutes the primary beacon for data controllers and courts when focusing on the challenges of automated decision-making. This focus does not mean that this right should confer privacy and data protection a quasi-absolute protection in any case. On the opposite, privacy and data protection would acquire a predominant role when there is the need to ensure that individual rights are not so compressed that autonomy and self-determination are effectively compromised. The limit established by the GDPR to the processing of personal data through automated decision-making processes is not a mere data subject right which can be overcome easily by ensuring security measures or opaque forms of explanation. It is an instrument of freedom against the techno-determinism established by predominant private and public actors.Footnote ²⁰¹ This rule horizontally connects human dignity, as the basic pillar of European constitutionalism, with algorithmic technologies, thus making the promises of a more constitutional sustainable innovation. The focus on human dignity would be the primary reference for lawmakers and judges in approaching this safeguard, thus implying a strict interpretation of the exceptions and limitations to this ‘human’ right.

6.5.2 Conflicting Positions and Proportionality

Human dignity is the primary but not the only underpinning value of the GDPR. Another constitutional principle grounding European data protection is proportionality which can be considered the foundation of the risk-based approach based on the principle of accountability. As in the case of human dignity, different angles can show how this value is expressed by the GDPR.

Proportionality is a pillar of democratic constitutionalism.Footnote ²⁰² Even if this principle is declined in different ways on a global scale,Footnote ²⁰³ proportionality expresses the need to internally limit the exercise of public and private powers, thus safeguarding individuals against excessive interferences.Footnote ²⁰⁴ The structure of European data protection is a paradigmatic example of the principle of proportionality. As already stressed, personal data enjoy a broad margin of protection in the Union.

Although the ECJ has recognised a high degree of protection to personal data, there is not a rigid hierarchy between fundamental rights and freedoms. Data protection is not an absolute right even when focusing on legitimate interests according to the tests established by the Convention and the Charter. The protection of this fundamental right cannot lead to the destruction of other constitutional interests such as freedom to conduct business as enshrined in the Charter.Footnote ²⁰⁵

Therefore, when interpreting the obligations of the GDPR, it is crucial not to forget that the interests of the data controller and of the data subject represent nothing but the constitutional clash between the protection of personal data with other fundamental rights and freedoms or legitimate interests in the case of public authorities. In other words, the general principles, safeguards and obligations of the GDPR need to be framed within such a context of balancing rather than axiology. It is not by chance that the ECJ has relied on the principle of proportionality since its first cases on data protection,Footnote ²⁰⁶ and this balancing logic is at the core of the GDPR’s structure.

Moving from the constitutional level to the GDPR, the principle of accountability of the data controller could be considered the constitutional translation of a risk-based approach based on the notion of balancing. This principle requires the controller to prove compliance with the GDPR’s principles by establishing safeguards and limitations based on the specific context of the processing, primarily the risks for data subjects.Footnote ²⁰⁷ The Data Protection Directive had already tried to introduce such an approach focused on the risk of processing, for instance, concerning the implementation of security measures. Likewise, the WP29 stressed the role of a risk-based approach in data protection underlining how risk management is not a new concept in data protection law.Footnote ²⁰⁸ Even the Council of Ministers of the Organisation for Economic Co-operation and Development implemented a risk-based approach when revising the Guidelines Governing the Protection of Privacy and Transborder Flows of Personal Data, first adopted in 1980.Footnote ²⁰⁹

From a formal perspective, despite the open clauses, the move from minimum to full harmonisation has been a powerful boost for legal certainty in the internal market. Such a move has not only led to strengthening the protection of privacy and personal data as fundamental rights of the Union but has also allowed a more balanced approach between rights and obligations. The principle of accountability reflects such a mix between certainty and proportionality. The data controller has been considered responsible (and not only liable) to ensure that the protection of data subject’s privacy and data protection are ensured and protected. And this role comes from the respect not only of the GDPR’s obligations but also of general principles.

The GDPR modulates the obligations of the data controller according to the specific context in which the processing takes place, namely ‘taking into account the nature, scope, context and purposes of processing as well as the risks of varying likelihood and severity for the rights and freedoms of natural person’.Footnote ²¹⁰ For instance, when looking at legitimate interest as a condition for lawfully processing personal data, the GDPR provides a limitation balancing ‘the interests or fundamental rights and freedoms of the data subject which require protection of personal data, in particular where the data subject is a child’.Footnote ²¹¹ This focus extends also to the principle of privacy by design and by default as an expression of the general principle of accountability.Footnote ²¹² As observed by Macenaite, ‘risk becomes a new boundary in the data protection field when deciding whether easily to allow personal data processing or to impose additional legal and procedural safeguards in order to shield the relevant data subjects from possible harm’.Footnote ²¹³ It would be enough to focus on the norms concerning the Data Protection Impact Assessment or the appointment of the Data Protection Officer to understand how the GDPR has not introduced mere obligations to comply but a flexible risk-based approach which leads to defining different margins of responsibility on each data controller depending on the context at stake.Footnote ²¹⁴

Fundamental rights are the parameters on which the risk-based approach, as a system where data controllers’ responsibility is assessed on a case-by-case basis, is grounded. This system represents nothing but the expression of a principle of proportionality reflecting the lack of a rigid axiology in the European constitutional framework. The risk-based approach reflects nothing else than the balancing of the conflicting interests of data subjects and controllers. In other words, the GDPR has led to the merge of a rights-based approach where the fundamental rights of data subjects play the role of a beacon for compliance.

From the perspective of data controllers, the high standard of compliance required by the GDPR could however affect small or medium controllers which can be required to adopt higher safeguards, primarily when data processing operations could lead to high risks for the data subjects. This approach could affect the freedom to conduct business and development of the internal market. Even if the GDPR’s approach could favour multinational corporations in the process of compliance,Footnote ²¹⁵ nevertheless, it introduces a mechanism which does not focus only on rigid obligations but also on the concrete framework of the processing. This margin of discretion could promote the development of artificial intelligence technologies while protecting individual fundamental rights. This shift from theory to practice introduces certain flexibility allowing the data controller to determine the measures to apply according to the risks connected to data processing, while maintaining the duty to justify the reasons for these decisions. The GDPR would increase the discretion of the data controller in determining which safeguards apply to the data collected and processed in a certain context.

Likewise, from the data subjects’ standpoint, the risk-based system is complemented by a rights-based system coming from the broad extension of fundamental rights in the European framework. Individuals have the right to access and limit the processing of their data, ask about their erasure or portability based on the conditions established by the GDPR for each data subject’s right. Scholars have underlined that ‘from the user perspective, the impact of data portability is evident both in terms of control of personal data (and in general in the sense of empowerment of control rights of individuals), and in terms of a more user-centric interrelation between services. At the same time, it is a challenge to third data subjects’ rights’.Footnote ²¹⁶ This approach underlines how the GDPR does not provide users with absolute rights. While empowering data subjects would increase the control over the processing of data, the implementation of their rights is a burden requiring data controllers to invest resources and define procedures to implement them.

When framing such considerations in the field of artificial intelligence, the GDPR does not establish an absolute prohibition in relation to automated decision-making, even if it bans the processing of particular categories of data except for the case where the data subject has given his or her explicit consent. The GDPR introduces exceptions according to which, despite potential legal or similarly significant consequences, data subjects cannot rely on this right. Their presence should not come as a surprise when focusing on the characteristics of European constitutionalism which, as already stressed, does not recognise absolute protection to fundamental rights. The ECJ underlined that the right to the protection of personal data does not enjoy absolute protection but is subject to balancing with other interests.Footnote ²¹⁷ In any case, limitations shall be strictly necessary to genuinely meet the objectives of general interest pursued, subject to the principle of proportionality.Footnote ²¹⁸ Moreover, Member States can introduce exceptions to limit the right not to be subject to automated decision-making processes.Footnote ²¹⁹ In any case, the protection of fundamental rights cannot lead to the annihilation of any other rights and freedoms recognised in this Charter.

Therefore, the principle of accountability is not only a burden for data controllers but also a threatening delegation of responsibility concerning the protection of fundamental rights and freedoms. This way, the GDPR leads data controllers to become the arbiters of privacy and data protection. The limit to the exercise of this power is the principle of proportionality which, together with human dignity, are guidance for lawmakers and judges when addressing the balancing between the accountability of data controllers and the fundamental rights of data subjects. Therefore, the principle of accountability can play an important role in the development of the internal market without leaving fundamental rights behind. As a general principle, the more the discretion exercised by the data controller, the more the data subjects should be protected. This principle would leave data controllers to perform their activities considering that their beacon of compliance is not simply represented by the GDPR’s material and organisational requirements but also coincides with the protection of individuals, precisely their dignity.

Therefore, the principle of human dignity is relevant within the framework of proportionality. Although the GDPR’s exceptions to data subjects’ rights and freedoms may find their legitimation in the need to balance conflicting interests, however, justifying exceptions to data subjects’ rights against automated decision-making processes would betray the aim to protect human dignity. It would be worth wondering how exceptions could be tolerated in this case if these technologies could lead to a process of dehumanisation in the long run. The answer to such a concern can be found by looking at due process safeguards which would aim to preserve human dignity while promoting a sustainable solution to foster innovation.

6.5.3 Enhancing Due Process

The question is therefore how human dignity can be protected against potential disbalances in the exercise of conflicting rights and freedoms. Limitations to individual rights reflecting the principle of proportionality should not be considered as a threat to human dignity when due process safeguards are in place. The possibility to rely on procedural safeguards would mitigate disproportionate effects resulting from the exercise of public powers or private determinations. Due process would play a crucial role even beyond the boundaries of public powers.Footnote ²²⁰

Together with the personalistic principle, European data protection law is an example of due process safeguards. Since the adoption of the Data Protection Directive, European data protection law has primarily provided substantive obligations and procedural safeguards regulating the entire process of data processing from analysis of risks (e.g. DPIA), to rules on notice (e.g. mandatory disclosure), collection (e.g. consent), processing (e.g. purpose limitation), safeguards (e.g. data subject rights) and remedies (e.g. judicial enforcement). These norms represent the expression of the right to self-determination of individuals who, without knowing how data are processed, cannot be aware of the processing of their personal data. These ex ante safeguards increase transparency and accountability, thus making the individual more aware of how personal data are used to make even automated decisions which could affect their legal rights. Put another way, such an approach would meet that principle of self-determination which makes humans dignus rather than subject to public and private determinations.

By promoting transparency and accountability in automated decision-making processes through procedural safeguards, the GDPR fosters human dignity. Therefore, due process is an essential tile of the constitutional mosaic of the GDPR. This constitutional architecture is also evident when focusing on the safeguards relating to artificial intelligence technologies. The data controller is required to inform data subjects about the existence of a process of automated decision-making, its logic, significance and consequences,Footnote ²²¹ while the data subject has the right to ask the data controller to access their personal data.Footnote ²²² In the case of the right not to be subject to automated decision-making, the GDPR recognises a procedural safeguard consisting of the right ‘to require human intervention, to express her point of view and to contest the decision’.Footnote ²²³ Therefore, apart from when the processing is authorised by Union or Member State law to which the controller is subject and which also lays down suitable measures to safeguard the data subject’s rights and freedoms and legitimate interests, individuals have the right to ask for human intervention to assess the machine’s outcome.Footnote ²²⁴

The principle of human-in-the-loop in the context of algorithmic decision-making is a paradigmatic attempt to introduce procedural safeguards. Minimal due process becomes a precondition to mitigate the asymmetry of powers between individuals and data controllers in the context of automated decision-making.Footnote ²²⁵ In this sense, due process is an inalienable right in the algorithmic society,Footnote ²²⁶ or why individuals should have a right to contest artificial intelligence systems.Footnote ²²⁷ This constitutional value raised within the realm of state actor is horizontally extended to the private actors through the obligation to ensure human intervention. It is not by chance that this principle is stated only when the processing involved automated decision-making technologies. This is because algorithmic decisions can produce serious effects on individual rights and freedoms. To remedy the lack of transparency oversight on algorithmic technologies, the GDPR requires that this processing deserves to be complemented by an adversarial principle and redress mechanism based on human intervention.

By recognising this right, the GDPR also seems to suggest that the last word over individual rights and freedoms should be human. A machine should not play this function without the support of humans that need to be in the loop. This is what the Commission already underlined in 1992 when stating that ‘human judgment must have its place’.Footnote ²²⁸ Therefore, due process safeguards can protect human dignity complementing the general prohibition of full automated decision-making systems for the processing of personal data. This principle does not just recognise the role of humans in automated decision-making but also the primacy of human assessment over the efficiency of machines. Paradoxically, the inefficiency and irrationality of human beings is the last safeguard against the true interpretation of its nature.

The principle of human-in-the-loop cannot be considered as a general solution for the challenges raised by artificial intelligence. By looking at such a principle under the lens of proportionality, it can be observed that, while enhancing due process safeguards, it could potentially disregard other interests requiring protection. A broad extension of this rule can undermine the freedom to conduct business of private actors or the performance of public tasks. Besides, as already stressed, relying on human intervention as a procedural safeguard does not always ensure better decision-making.

These drawbacks are just a small price to pay to ensure that humans are not marginalised by opaque algorithmic technologies and asymmetries of powers. These concerns are compensated by the critical role which due process plays against the unaccountable development of artificial intelligence technologies and the rise of private powers in the algorithmic society. The development of automated systems is not always driven by public purposes but usually by business interests focused on profit maximisation. Design choices could be not neutral and answer to opaque business logics which transform human life in technical norms of processing and extraction of values. In other words, the definition of transnational standards of automated systems outside any public scrutiny contributes to creating a para-constitutional environment competing with public values. This situation is not only relevant for due process, but also for the principle of the rule of law. If legal norms are replaced by technological standards, there will be no space for democratic constitutionalism to ensure the protection of public values against the rise of unaccountable technologies expressing private powers. Within this framework, the principle of human-in-the-loop is a shield not only as a due process safeguard, but also to protect democratic values.

The GDPR is fostering the principle of rule of law when the processing of personal data involves automated decision-making. This way, the GDPR bans any discretionary use of automated decision-making to process personal data. The principle of the rule of law is of a critical value to reduce the gap between the public and private sector involved in processing personal data. In the lack of any legal obligations, private actors are not required to give reasons justifying their policies or actions. While public actors are required to comply with constitutional principles, the private sector is not bound by constitutional principles and norms without a positive translation as it occurred with the GDPR. In the algorithmic society, private companies have demonstrated their abilities to acquire dominant positions in the market of data by extracting value from them. Within this framework, the data subject could be considered as a vulnerable actor whose protection of rights and freedoms should not only find its ground in the substantive rights but also in procedural safeguards to remedy the imbalance of power.

Within this framework, enhancing due process complements the relevance of human dignity and proportionality as expression of the constitutional values underpinning the GDPR. In this case, the GDPR obligations should not be seen as a mere instrument for requiring data controllers to comply with certain rules but as the constitutional expression of procedural safeguards aimed to avoid a disproportionate exercise of powers in the balancing between conflicting interests. In this sense, the obligations of the GDPR should be constitutionally interpreted as a means to ensure that human dignity and democratic values are not annihilated by the lack of transparency and accountability in the exercise of powers in the field of data.