I. Background

It is currently trendy to establish bodies to ponder upon the topic of artificial intelligence (AI) for policymakers. In the United Kingdom, this has been taken to a bizarre extreme. Attached to the public sector alone we can identify at least: the Centre for Data Ethics and Innovation;Footnote ¹ the AI Council;Footnote ² the Office for AI;Footnote ³ the House of Lords Select Committee on AI;Footnote ⁴ the House of Commons Inquiry on Algorithms in Decision-Making;Footnote ⁵ the Alan Turing Institute; the National Data Guardian;Footnote ⁶ the Information Commissioner’s Office; a proposed new digital regulator;Footnote ⁷ the various directorates of the Department of Digital, Culture, Media and Sport who took over from the Government Digital Service on drafting the Government Data (Science) Ethics Framework;Footnote ⁸ the Digital Catapult;Footnote ⁹ the Office for Tackling Injustices;Footnote ¹⁰ the Regulatory Horizons Council;Footnote ¹¹ proposed new algorithmic accountability powers against platforms for Ofcom;Footnote ¹² NHSX, a new health National AI Lab;Footnote ¹³ AI monitoring by the Health and Safety Executive’s (HSE) Foresight Centre;Footnote ¹⁴ AI analysis from the Government Office for Science;Footnote ¹⁵ the Office for National Statistics’ Data Science Campus;Footnote ¹⁶ and the Department of Health and Social Care’s code of conduct for data-driven health and care technology.Footnote ¹⁷ Keeping up with these organisations, initiatives and efforts is a full-time job in and of itself.

The European Commission has not set up quite so many initiatives as yet, but among those introduced, the most well-known is arguably the “High-Level Expert Group on Artificial Intelligence” (HLEG).Footnote ¹⁸ This group was formed through individuals nominating themselves as experts in this field, and was established in 2018. In the year following its formation, it has produced two main deliverables. The first was a set of ethics guidelines.Footnote ¹⁹ These ethics guidelines did not land, even in their revised post-consultation form, without controversy and accusations of “ethics washing” and industry capture of the process.Footnote ²⁰ The second deliverable was a set of “policy and investment” recommendations.Footnote ²¹ This article critiques that document, and contextualises that critique in a broader narrative about the trajectories of artificial intelligence, policy and society.

This document does come at a time of agenda-setting in Europe. In May 2019 the European Commission announced its intention to create a “robust regulatory framework [to] proactively address the ethical and legal questions surrounding artificial intelligence”.Footnote ²² This was reiterated in Commission President-elect Ursula von der Leyen’s “Agenda for Europe”, where she stated her intention to “put forward legislation for a coordinated European approach on the human and ethical implications of Artificial Intelligence”.Footnote ²³ It is therefore imperative that its recommendations should be subjected to maximum possible scrutiny.

This article focuses on several issues with the HLEG’s policy recommendations in the area of artificial intelligence. Firstly, the document is set up as to ignore and encourage the skipping of problem framing in areas where AI is suggested as being helpful. The capacities needed to assess the usefulness of computing in any given context, rather than just to increase its availability or intensity, go unconsidered. Secondly, the type of expertise and input recommended to cope with this issue – largely an increased, inward-looking focus on ethicists of artificial intelligence or stakeholders in an “AI” bubble – do not seem appropriate responses to factors and environments that foster unethical technology in practice today, and may even further marginalise the voices of affected communities. The structure and the composition of the HLEG only contributes to this problem. Lastly, the HLEG either ignores or under-emphasises two critical issues: the infrastructure underpinning optimisation using “AI”, and questions of power and control.

II. Framing issues

What is meant when organisations apply “AI” to a problem is often indistinguishable from the application of computing, statistics, or even evidence. The usage of the phrase has become so laughably ambiguous and general, it is almost like saying that to solve an urban infrastructure problem, one must “apply power tools”. Yet, unlike that analogy, “AI” is “intelligent”. This masks an important way in which no tool can be intelligent: tools cannot define the problem they are applied to. When holding a hammer, everything looks like a nail.

Because of this, a focus on AI incentivises fudging or skipping arguably the most important stage in any policy process – problem structuring and definition.Footnote ²⁴ Take the HLEG’s recommendation to “[f]oster the development of AI solutions that address sustainability challenges”.Footnote ²⁵ There is no blueprint for engineering societal sustainability. It is widely acknowledged as a “wicked”, poorly structured problem. There is limited political agreement on the ends of sustainability – who will sacrifice what for carbon reductions? – and further limited agreement on the means by which these disputed ends should be sought.Footnote ²⁶ Deploying artificial intelligence as a “solution” requires quite certain agreement on ends, which are incidentally limited to those compatible with rendering the world legible by machines.Footnote ²⁷ The difficulty in reaching agreement on ends does not, however, result from general dithering and indecision which a keen (and likely white cis male) engineer can efficiently resolve, coming up with ends suitably compatible to optimisation through “AI”. Agreement can only be sustainably reached by genuine processes of negotiation and participation.Footnote ²⁸ Bulldozing through or subverting that stage is not a recipe for sustainability, but for tension, conflict, and unintended consequences. The idea that exists as a subtext that AI is “intelligent” enough to abolish trade-offs, depoliticise extremely contested issues, and give all parties what they seek is at best unevidenced and naïve, and at worst a lie.

The HLEG recommendations, among many policy reports and views in this area, suffer from a deficit in recognising the importance of problem structuring and framing more broadly. We can take some lessons on how best to apply the “magic” of artificial intelligence from the application of actual magic by late fantasy author Terry Pratchett. Pratchett’s popular, comical Discworld series contains two main users of magic: wizards and witches. Bureaucratic, hierarchical wizards blindly frame all problems as soluble by magic, often bringing problematic consequences. Witches, who live independently in superstitious rural villages, see the most important feature of magic as knowing when not to use it. They are rarely seen casting spells, as they start with the problem in front of them, often passing off psychology, practical knowledge or social dynamics as “magic” to achieve their ends. The HLEG, in many recommendations, emphasises training and retaining AI specialists for the application of AI to societal problems. It misses the importance of having the capacity to know when not to use artificial intelligence. To what extent is “intelligent” surveillance a solution for anti-social youth behaviour in an area without enough alternative evening activities or public spaces? What use is predictive railway maintenance if the infrastructure was poorly specified and invested in further upstream? Will monitoring and prediction of employee workload in a search for productivity compensate for the mental stress of constant observation such a system might engender? Ben Green addresses these issues in his recent book The Smart Enough City, highlighting the need to see technology as just one tool in a toolkit: just one of many means that might potentially be used to head towards a complex and societally negotiated end.Footnote ²⁹ The focus should be on taking off the “tech goggles” to identify problems, challenges and needs, and to not be afraid to discover that other policy options are superior to a technology investment.

The HLEG recommendations are, at times, aiming at deficits that do not exist. Unscrupulous vendors do not need help to better locate businesses and public bodies apparently “in need” of an AI “solution”. Their PR efforts are doing a stellar job in that department as it stands. On the contrary, intervention might be needed in helping buyers (public and private) lacking in analytic capacity figure out whether there is truly a promising role for technology in their problems, and what it might look like. Those inside organisations – procurement officials, advisers, policy officers, and the like – must be able to cut through the advertising and lay out options in a fair and even way, placing technologies such as artificial intelligence alongside, not above, the many other tools at their disposal.

III. Low level experts needed

A similar issue can be taken with the surprising suggestion the report supports that “720 professorial chairs in applied ethics” are required across a network of “trustworthy AI” research.Footnote ³⁰ Is a lack of institutionalised ethicists the source of the problems being faced today? Or, more cynically, might this suggestion be a result of the interests of those sitting on the HLEG itself?

As the Tech Won’t Build it and Google Walkout movements are illustrating, organisational factors permitting the unethical use of technology seems to be located in the ways the business models of tech giants override the desires and concerns of individual expert workers. It would be quaint if the main or only barrier to reflection and change for a lumbering, bureaucratic multinational was the lack of a well-meaning ethicist to point out a quandary or unintended consequence. Quaint, but unrealistic. The business models that scale to create today’s multinational money-printing (or, at the very least, capital-attracting) machines generate externalities which are just the cost of doing business. The harms, risks and inconveniences they shift to society are in general inextricable parts of the business model, with some more amenable to cheap mitigation (often begrudgingly in the face of potential or actual, enforced legislation) than others.

If movements like Tech Won’t Build It and Google Walkout have taught anything, it’s that there is some, perhaps limited, hope in the morals of individual workers. Yet are ethicists the solution here? They seem extraordinarily poorly placed to train workers who are skilled and empowered to organise, object and influence business models from within – indeed, a training in assessing ethical issues says little about how to action the results of those assessments. What is needed is a challenge to corporate governance. Yet any such reforms which might shake-up corporate decision-making or board representation are conspicuously completely absent from HLEG recommendations.

An issue with both ethicists and employees, largely engineers, taking the lead surrounds the flaws in a engineering-centred approach to ethical issues. Seeta Peña Gangadharan and Jędrzej Niklas have recently critiqued the techno-centric nature of studies of fairness, accountability and transparency, drawing attention to how civil society voices analyse technology as part of systemic practices of injustice, rather than miss the role of broader systems.Footnote ³¹ Seeing a greater role for ethicists of technology or conscientious engineers as the correct response to injustice exacerbated by technology but not, at its root, caused by it risks further marginalising those with the clearest view of on-the-ground issues and the closest connection and legitimacy to affected communities. The HLEG is notable by its exclusion of such voices, seeing expertise in artificial intelligence as the domain of technical researchers, generalist ethics and governance scholars, industry lobby groups. This is greatly problematic, particularly in light of the extensive research on the factors for success of expert assessments in value-laden, technically challenging areas, which illustrate the importance of legitimacy in the success of these assessments.Footnote ³² Affected communities “judge legitimacy [of expert assessments] based on who participated and who did not, the processes for making choices, and how information was produced, vetted, and disseminated”, with “[p]otential users often reject[ing] assessments that lack legitimacy as ‘not invented here’”.Footnote ³³ As it stands, only two of the 51 members of the HLEG clearly represent civil society of any flavour, let alone in any of the specific domains the report discusses.Footnote ³⁴

IV. Lack of infrastructure

The HLEG recommendations are additionally deeply timid in respect to even discussion of the need to exercise legal control over data collection infrastructures.

The capabilities of the largest firms to build effective machine learning models is often more a result of the data collection and experimentation infrastructure they have established than any single static dataset. Signs of these infrastructures are everywhere, from the data-intensive exhaust of cloud services powering voice assistants or location services to the covert third-party trackers littering the WebFootnote ³⁵ and popular apps.Footnote ³⁶ They connect to the new way software is designed, through “agile” methods in “perpetual beta”, where the invisible experimentation, data collection and updating infrastructures mean that for many pieces of software the version you use today may not be the same as that of the person sitting beside you, and is unlikely to be the same version you use next week.Footnote ³⁷ These firms can build machine learning models of all flavours with this infrastructure, affording them informational power in a huge range of situations rather than just a handful.

Equally important is the role of infrastructure in deploying artificial intelligence. Models matter most when their results are used, and so those with the power to propagate the results of these models at scale are just as critical as points of policy intervention as those where models are created and trained. The report does not adequately address the notion that in some cases, artificial intelligence may change the delicate, societally negotiated power balances relating to existing technical infrastructures, such as financial institutions and online intermediaries. The current societal acceptance of CCTV cameras among some individuals could be argued to have been partly based on the notion that the footage was sufficiently labour intensive to examine and monitor such that it would not be considered constant surveillance. The application of machine learning techniques, such as machine vision, and the use of automatic classifiers such that CCTV is “always being watched” changes the power of those controlling previously “dumb” infrastructure. In these cases, the use of AI might require existing policies to be reconsidered with a view to retaining the same balance of societal interests and fundamental rights, which might mean limiting infrastructural expansion that previously seemed proportionate.

In general, tackling the policy challenges of AI cannot be divorced from tackling the policy challenges of tracking and sensing, both in digital and physical infrastructures. To this extent, it is disappointing that the HLEG does not discuss the lack of regulatory capacity to analyse, proactively monitor and police these areas.

The Information Commissioner’s Office (ICO), the UK’s data protection supervisory authority, is the largest in Europe. It receives no money from the Treasury, and its funding comes almost entirely from fees levied on data controllers, with a budget of approximately £40m.Footnote ³⁸ Given that all data controllers fall in scope as being subject to regulation, as well as the 66 million inhabitants of the UK, and any non-UK residents anywhere in the world who are data subjects in relation to a UK data controller, this is a huge undertaking for that sum of money. Other regulators of digital services, such as Ofcom, the broadcast regulator, have a budget nearly five times greater, and often spend above it.Footnote ³⁹ The ICO is furthermore in strong shape in terms of number of employees compared to other European data protection authorities. A recent report from the European Data Protection Board paints a dismal picture of supervisory authorities’ financial and human resourcing needs, with several regulators in the EU stating their need for 50% more or even double the resources they can currently muster.Footnote ⁴⁰

Without sufficient regulatory resource, it is not going to be possible to understand, monitor and assess the infrastructures that AI transforms and it used through in a way that can truly ensure they comply with the law. As it stands, infrastructure such as real-time bidding, an advertising system drawing heavily on machine learning-powered profiling that has been powering a significant proportion of adverts on the web since 2010, has only just been analysed and found to be significantly illegal by the Information Commissioner.Footnote ⁴¹ The illegality of this pervasive system is not new, but a ten-year wait for analysis is simply too long to wait to uncover significant areas of regulatory non-compliance.

The HLEG does not recommend increased regulatory and enforcement capacity. The main public capacity recommendation the report does provide surrounds a perceived need for data sharing infrastructures and open public data (which appear strangely ignorant of the 2019 Open Data Directive, its “open by design and by default” public sector provisionsFootnote ⁴² and particular requirements for “dynamic data” and “high-value” data being made available through APIs and bulk downloads). The HLEG should have instead taken the opportunity to recommend an increase in the funding and capacity for all regulators in light of technological challenges. The proper funding of regulators is a primary inhibitor of the enforcement of existing laws today, and the even and proper application of laws across the Single Market, without de facto regulatory arbitrage, relies on at least comparable capacity between Member States. Again, a cynic might note the presence of several large firms and notorious technology lobby groups such as DIGITALEUROPE in the HLEG as a reason why increased enforcement of commonly-flouted existing data law did not make it into the report.

V. Power outage

Power rarely makes a bold showing in the recommendations of the HLEG. Indeed, the few places where HLEG has mentioned redressing relative power and control in the document are easily identified by both being placed at the end of their respective sections, and by beginning with a non-committal recommendation to “consider” instead of “develop”, “set up” or “adopt”. The few that exist are almost all weak or ambitious in their content, too. For example, the public sector should “consider” mandating their contractors to return “data” (not models) produced in the context of a contract to a public body paying them – unless of course it was not of sufficient “general public interest” and/or it might “infringe the private company’s IP”.Footnote ⁴³

Recommendation 18.8 is one of the only daring recommendations in the entire document in this regard, suggesting that legally mandated interoperability requirements should be imposed on a sector-by-sector basis on both dominant and non-dominant firms to enable competitive market infrastructures.Footnote ⁴⁴ Such recommendations are far from new,Footnote ⁴⁵ but constantly shot down by firms who much prefer the framing of “data portability” (where a non-dynamic snapshot can be moved from provider-to-provider by a particularly determined consumer). The dominant players in AI are currently collaborating to ensure they can pre-empt legislative desires and point to functioning snapshot-style data portability, rather than risk seeing interoperability on the regulatory agenda.Footnote ⁴⁶

The only other recommendation concerning competition and market power is to “consider” revisions to competition law to consider “the volume of data or incumbency data advantages – the building block of many AI systems – in the assessment of market power”.Footnote ⁴⁷ This recommendation sounds worthy, and certainly would be impactful, but is already one step behind the current trajectory of the largest technology firms in the world.

To understand why, it is important to note that firms do not intrinsically care about data, but their ability to optimise.Footnote ⁴⁸ Governments and surveillance agencies are the main candidates for caring about the content of datasets beyond their optimisation potential, but most private firms operating with broadly lawful intentions do not have an interest in qualitatively snooping on individuals for their own business purposes. Typically, it has gone without saying that you cannot optimise without data. This left large platforms heavily vulnerable to co-optation by state surveillance, as the leaked Snowden documents illustrate.Footnote ⁴⁹

Today, a growing number of firms have taken steps to publicly tie their hands, practically and mathematically limiting their ability to view individual records of data whilst still controlling infrastructure, often cryptographic or hardware-based in nature, which allows them to run the optimisations or deliver the services that bring in revenue. The most high-profile case of this is the well-publicised struggle between Apple and the FBI in relation to accessing the contents of an encrypted iPhone used in a terrorist shooting at San Bernadino, CA in late 2015.Footnote ⁵⁰

Many technologies are being fast developed that permit optimisation without a firm being able to see individual records of data “in the clear”– in decrypted form. These include technologies which allow aggregate statistics or machine learning models to be computed without the pooling of data in a single place, or beyond devices (such as homomorphic encryption or secure multi-party computation) and techniques designed to establish datasets which do not reveal information about individual users (such as differential privacy).Footnote ⁵¹ These are good from the standpoint of privacy-as-confidentiality, for individuals whose threat models consist of qualitative information about themselves being leaked. They are not as good for ensuring individual or society control concerning design or optimisation decisions, particularly those designed to profile them, manipulate them, or to place different weightings or disregard aspects of society or the environment.Footnote ⁵² Those concerned about data-driven “nudging” as a mode of regulationFootnote ⁵³ will likely find little solace in the fact that confidentiality of data is mathematically assured despite firms be able to deploy “artificial intelligence” to influence individuals in effectively the same way as previously.

VI. Concluding remarks

The HLEG missed a range of opportunities in this report. The document is a marathon of recommendation, with no clear prioritisation and the more interesting of the recommendations buried deeply in a range of futile, confused or non-actionable suggestions. It foregrounds AI as a technological solution to completely inappropriate issues without considering the capacities needed to understand the problems which AI could potentially be applied to. It incorrectly portrays the reasons why unethical technologies are found used in the wild today, making them appear like bone fide oversights that ethicists might be able to highlight rather than intrinsic parts of business models which disregard their effects on societies and environments. By their composition and their recommendation, they further sideline and marginalise community and domain voices, and seek to reify an elite club of AI and society experts to the detriment of those with connection to harms and issues that technologies exacerbate. And finally, they largely ignore infrastructure and power, which should be one of, if not the most, central concern around the regulation and governance of data, optimisation and “artificial intelligence” in Europe going forwards.

There is limited time in which to shape tomorrow’s information infrastructures while they are still, even slightly, in flux. The European Commission needs challenging thinkers that are bold and perceptive in their understanding of the future trajectories of those firms intermediating so much of today’s information flows. Unfortunately, as a collective and in my opinion, the High Level Group has not demonstrated that it can provide that critical analysis.