1.1 Introduction
Clever Computing, the convergence of physical sensing through the Internet of Things (IoT) with data-driven AI systems, is emerging within many settings of everyday life, like homes, workplaces, and public spaces. The IoT has long involved networked devices that sense and gather data about humans, things, and daily activities. But AI is increasingly being integrated into IoT systems to adapt the nature of user interactions, shift how data is used, and extend system autonomy. Clever Computing is often sold on the premise of increased convenience, safety, and efficiency. From a user perspective, the reality is often quite different, where systems may increase data surveillance and corporate control, erode citizen digital rights, and create a raft of new socio-technical and legal harms.
I argue that Clever Computing is not designed in a way that sufficiently accounts for social, ethical, and, most critically, legal values. Instead, to build Clever Computing systems that we want to live with now and in the future, technology design processes need to become human-centred and better account for the law. I propose Accountable Design as a novel, practically orientated approach to doing this. Laws often aim to protect humans, their rights, relationships, and environments by establishing responsibilities, requirements, and rules for those who can impact or harm those interests. Yet, situating what law means on the ground for technology design practices is often unclear. This book explores not only the implications of folding legal values into human-centred design processes but also how to move towards building more legally accountable Clever Computing systems.
Accountable Design is underpinned by extensive interdisciplinary research. It draws together methods, concepts, and frameworks from technology law, human–computer interaction (HCI), design research, applied ethics, and science and technology studies. Accountable Design shows how to integrate user needs and the legal obligations of designers in developing technologies that mitigate socio-technical and legal harms. As I will demonstrate, technology designers are already directly implicated and accountable for responding to a wide array of legal requirements. Design is squarely at the centre of legislative endeavours to address risks stemming from Clever Computing systems. Measures aimed at protecting the societal values of privacy, security, trust, and safety – by design – exist within recent and upcoming legislation, particularly from the European Union (EU). Examples include:
Privacy: EU General Data Protection Regulation 2016 (GDPR)Footnote 1 requires privacy by design through ‘data protection by design and default’ requirements and The EU Data Act 2023Footnote 2 mandates fair default data access.
Security: The EU Amended Radio Equipment Directive 2014 (RED),Footnote 3 EU Cyber Resilience Act 2024 (CRA),Footnote 4 and the UK Product Security and Telecommunications Infrastructure Act 2023 (PSTI) ActFootnote 5 require increased security safeguards for the IoT by design.
Trust and Safety: The EU Artificial Intelligence Act 2024 (AIA)Footnote 6 and EU Machinery Regulation 2023 (MAR)Footnote 7 set design and development requirements for high-risk AI systems and health and safety obligations when interacting with automated workplace systems.
Designers are charged with demonstrating how they are addressing the regulatory requirements within these frameworks. As a result, law itself is becoming a design brief, but what this means in practice for Clever Computing requires further unpacking. I will show that Accountable Design is the much-needed response that bridges the gap between legal and technology design research and practitioner communities. To do this, I translate how law implicates design whilst charting how to build legally accountable Clever Computing systems by drawing on approaches and working practices from human-centred computing.
The book presents the socio-technical, design, organisational, and legal aspects of doing Accountable Design across different contexts of Clever Computing use through three detailed case studies. These show the need for an inherently interdisciplinary approach aligning conceptual, legal, design, and empirical analysis to ground and develop the concept. Through this analysis, I consider the necessity of reorientating technology design processes to integrate legal values to create more socially, ethically, and legally responsible Clever Computing systems. Ultimately, the book provides critical reflection on what it means to regulate through design whilst providing pathways to practical action for those designing systems on how to do it in a human-centred manner.
Each chapter has a detailed overview of its structure, outline, and conclusions, so I will only provide a brief roadmap here (summarised in Figure 1.1). The rest of this chapter develops the motivations behind the new term Clever Computing and outlines the history of similar technologies, showing that the increased integration of AI into IoT (and associated socio-technical and regulatory issues) is a long-standing trend. Chapter 2 examines the rationale behind the new concept of Accountable Design and provides an interdisciplinary analysis of the meanings of accountable, design, and the conceptual implications of regulating through technology design. Three case studies in Chapters 3–5 follow, which examine contexts of AI-enabled IoT use and apply the Accountable Design approach. Chapter 3 focuses on designing for privacy in Clever Urban Computing systems deployed in urban and city contexts. Chapter 4 examines how to design for security in Clever Domestic Computing in homes, whilst Chapter 5 explores design for trust and safety for Clever Biometric Computing, particularly emotion and facial recognition technologies being deployed in public spaces and workplaces. Chapter 6 establishes a set of eleven principles and a six-step process for doing Accountable Design, coupled with a range of design tools to operationalise the approach. Through the development of decks of ideation cards, board games, speculative fictions, and technology impact assessments, I provide methodological reflections on how to fold law into technology design practices. Critically, through using and evaluating these tools, I offer insights about how to support researchers and practitioners who should be doing Accountable Design.
Book overview.
Figure 1.1 Long description
Circle 1 text states Clever Domestic Computing and Designing for Security; Circle 2 states Clever Urban Computing and Designing for Privacy; Circle 3 states Clever Biometric Computing and Designing for Trust and Safety.
In the remainder of this chapter, Section 1.2 explores the emergence of the IoT through the earlier ‘ubiquitous computing’ (Ubicomp) research agenda. This analysis helps situate the risks associated with engineering technologically driven visions of the future. These often neglect current social and human needs in favour of techno-led solutions, highlighting the need for systems design to better account for user needs and, by extension, legal values. I then define the IoT through key aspects such as reliance on physical sensors, pervasive data gathering, and ongoing connectivity with an ecosystem of actors. In Section 1.3, I present the reasoning behind Clever Computing. It captures the convergence of physical sensing through the IoT with data-driven AI in six ways: changing and improving interaction techniques with the IoT; making sense of IoT data and enabling distributed data analysis techniques; and augmenting existing functionality and increasing autonomy of IoT systems. Section 1.4 then provides a chapter summary.
1.2 From Ubicomp to the IoT
1.2.1 Demonstrating the Need for Accountable Design: The Case of IoT Toys
Toys should be items that children can always trust. However, the integration of data-gathering sensors into IoT teddy bears, dinosaurs, and walkie-talkies puts their young owners at unnecessary risk due to privacy and security being poorly accounted for in their design (Figure 1.2).Footnote 8 This is a long-standing issue, where a well-known example from 2017 was ‘My Friend Cayla’ – a doll with an onboard Bluetooth speaker and microphone. Security professionals found that it could easily be compromised, allowing hackers to remotely control the doll and speak to children.Footnote 9 In response, regulators around the world stepped in. Cayla was banned from sale in Germany, where the government even advised parents to destroy any Caylas in their homes.Footnote 10 A Norwegian Consumer Council investigation found Cayla and another IoT toy robot from the same manufacturer, i-Que, contravened consumer and data protection rules.Footnote 11 For example, contract terms for the devices could be changed without notice, data could be shared with third parties for targeted advertising, and commercial brands could be mentioned in conversations with children (posing manipulation risks). Whilst Cayla and i-Que are no longer on sale, they were not unusual in their risky design and associated business models. Other household toy names like the rebooted version of the 1990s classic toys, the Furby Connect and the Hello Barbie doll, faced similar security issues. The latter was hacked too, giving access to speakers, microphones, and account information from children. Barbie could select from 8,000 lines of dialogue during conversations to personalise its relationship with child users, a trend Val Steeves terms the ‘dark side of customer relationship management’, enabling deceptive and unfair marketing to children as a particularly vulnerable user group.Footnote 12
AI companion bear toy.
AI is increasingly embedded in IoT toys too, posing new risks for children’s data and wellbeing.Footnote 13 The market is awash with products using emotion recognition technology (ERT) in their interactions with children to infer their ‘feelings’ and respond appropriately.Footnote 14 For example, Casio’s Moflin is a furry guinea pig esque companion pet robot that emotionally reassures, calms, and ‘grows with you’.Footnote 15 Many toy firms, large and small, have started using generative AI. Start-up Curio created a soft plushie that integrates OpenAI’s ChatGPT, providing children with a friendly face when conversing with the chatbot, and major toy manufacturer Mattel has partnered with OpenAI.Footnote 16 Embodied Inc.’s AI-powered Moxie robot fuses both AI approaches, positioning itself as a holistic ‘learning robot with heart’ that targets emotional and educational development of children using generative AI (again with ChatGPT), alongside face- and voice-based ERT.Footnote 17 The toy market mirrors wider shifts in educational technologies using AI to track the learning engagement of children,Footnote 18 which is prompting increased government concern in the UK.Footnote 19
There are many legal issues around ERT (explored in depth in Chapter 5), but the use of generative AI in toys raises significant privacy issues.Footnote 20 The Italian government temporarily banned ChatGPT in 2023, citing concerns about the processing of children’s data.Footnote 21 The European Data Protection Board (EDPB) ChatGPT Taskforce foregrounds the technology’s impact on the GDPR fairness principle in Article 5(1)(a), which requires data not to be processed in a manner that is detrimental to the subject. The EDPB state that, as a public-facing platform, data subjects are offering personal data to ChatGPT, and OpenAI, as the data controller, should not shift responsibility to data subjects for screening their own chat inputs. They are using chat data to further train their model, so they remain responsible for GDPR compliance.Footnote 22 However, in practice, toy companies are shifting responsibility precisely in this way. The Mozilla Foundation’s consumer protection initiative ‘*Privacy not Included’, which reviews privacy risks around new consumer products, examined the Moxie AI robot. They found that the privacy policy instructs parents to tell children not to share private information with the toy.Footnote 23 In late 2024, Moxie closed down, switching off their cloud reliant robots. Parents were left with an expensive piece of e-Waste and a difficult conversation with their child about where their artificial emotional support companion had gone.
Design of next-generation AI-enabled toys needs significantly stricter regulation, given the target audience of children, alongside critical reflection on to what extent designing fair and safe child-AI interactions is even possible. Remedial steps can and have been taken, including enforcement actions, regulator investigations, bans, and litigation against manufacturers.Footnote 24 Pre-emptive action to mitigate harm in the first place is needed. A turn to design can be a way of embedding measures that respond to legal responsibilities within the fabric of the technology itself. In the UK, the Information Commissioner’s Office (ICO) has already sought to improve the design of digital services for children more broadly. Their Age-Appropriate Design Code of Conduct mandates design steps that build from the UN Convention on the Rights of the Child,Footnote 25 including acting in the best interests of the child or conducting a data protection impact assessment (DPIA) to address risks for children at different ages and developmental stages. The Code suggests design responses to create more age-appropriate systems, including minimising the data collected in the first place, using high privacy-protecting defaults, and being transparent about what data is used (and presenting this information in accessible formats). Alongside design, it is crucial to change business models to avoid harmful commercial practices like sharing children’s data or profiling and nudging them to change their settings to be less restrictive. A combination of strict regulatory and design-led efforts is necessary to shift current practices, which are worryingly showing no sign of slowing. A 2025 US Public Interest Group showed developers continued integration of AI chatbots into many toys on the market, despite parent AI companies having policies against it on age grounds.Footnote 26
AI-enabled IoT toys are just one example of Clever Computing within this book that highlights the growing need to properly account for legal rights and responsibilities in design processes. I will now look backwards at the historical trajectory of Clever Computing to examine the emergence of embedded, data-driven, connected, and now AI-driven IoT systems in everyday life.
1.2.2 Learning from the History of Ubicomp
Early innovations in the networking of physical devices did not start with a grand vision of building a connected future. Instead, it was driven by computer science researchers’ need for caffeine. The first IoT device in 1982 was a vending machine at Carnegie Mellon University in the US. It was networked by David Nicols and fellow graduate students to the Advanced Research Projects Agency Network (ARPANET – i.e. the early Internet) to check on real-time stocks of Coca-Cola. This helped the students decide if it was worth taking a stroll downstairs to retrieve a cold can or not.Footnote 27 Similarly, in 1991, the Trojan Room Coffee Machine at Cambridge University’s Computer Lab in the UK set up the world’s first webcam. Instead of being used for making endless conference calls, as they are today, it took periodic photos of the coffee machine. These were then uploaded to a server which allowed creator Quentin Stafford-Fraser and his colleagues in the Systems Research Group to remotely check if there was still coffee left in the pot.Footnote 28
Around the same time, Mark Weiser proposed an ambitious future vision of ‘post-desktop computing’.Footnote 29 Pioneered at Xerox PARC in the US in the early 1990s, Ubiquitous Computing (Ubicomp) was the label for a programme of work to develop computers that ‘weave themselves into the fabric of everyday life until they are indistinguishable from it’.Footnote 30 It aimed to make embedded computing ‘invisible in use’ in the workplace and at home. Ubicomp would shift how users interacted with computers by going beyond conventional desktop-based peripherals such as a mouse and keyboard and instead move towards interactive ‘tabs, pads, and boards’.In contemporary terms, these are wearables, handheld mobile devices, and large interactive displays. Existing preconceptions around how humans interact with computers would be challenged, as interactions would no longer just be a one-to-one relationship with a device.Footnote 31 Instead, shared boards could enable interactions with multiple users in a room whilst ambiently sensing and supporting different tasks, for example, working collaboratively in an office or cooking at home in a Ubicomp kitchen. Many of the advances in Ubicomp were supported by concurrent shifts in ‘mobile computing’.Footnote 32 The ‘tabs and pads’ of Weiser’s vision were coming to life through portable phones and tablets with sophisticated on-device processing, screen-based graphical user interfaces, mobile networking, and global positioning system (GPS) that provided scope for new user interactions ‘in the wild’, from location-based games to map navigation apps.Footnote 33
Ubicomp was not the only ‘post-desktop’ vision in town. Similar trends emerged with slightly different labels like Pervasive Computing,Footnote 34 Calm Computing,Footnote 35 and Ambient Intelligence,Footnote 36 latterly followed by Everyware,Footnote 37 Cyber-Physical Systems,Footnote 38 and Smart Technologies.Footnote 39 Some emerged from different starting points, and others tweaked, refined, or extended Ubicomp. For example, calm computing recognised the value of calmness as a Ubicomp design principle because ‘if computers are everywhere they better stay out of the way, and that means designing them so that the people being shared by the computers remain serene and in control’.Footnote 40 The constant beeping, buzzing, and notifications of contemporary IoT systems suggest that calmness was not been the guiding principle in our current ‘attention capture’-driven economy.
Nevertheless, collaborative academic, industry, and government investment in the 1990s and 2000s underpinned the growth of many research programmes into physical, mobile, and embedded computer systems. This included Philips and Eindhoven University’s collaboration on Ambient Intelligence,Footnote 41 IBM and Carnegie Mellon’s concurrent research into Pervasive Computing,Footnote 42 the European Commission’s Disappearing Computer programme,Footnote 43 Intel’s work on Proactive Computing,Footnote 44 and numerous university projects like the University of Nottingham Equator Consortium,Footnote 45 MIT’s Project Oxygen,Footnote 46 and Georgia Tech’s Aware Home.Footnote 47 This led to a long period of research and development tackling difficult technical questions like:Footnote 48
How to design invisible interfaces that can ambiently collect data from users without constantly asking them for permission or being obvious?
What is necessary to build a networking infrastructure that can seamlessly follow devices and users moving between different contexts like their home and work, whilst avoiding any drop in quality of service?
How can ambient sensors that constantly watch and gather data from their environment be sustained, for example, with battery life or environmental resilience?
Alongside this active technical agenda, Computer Supported Cooperative Work (CSCW) and HCI researchers increasingly foregrounded the need to avoid neglecting the social messiness of the myriad contexts computing would now be used in. Reflecting on this point remains pertinent today, as developments in AI increasingly risk overlooking human interests again. One useful example from the time was designing ‘invisible interfaces’. In part, this required examining suitable materials, sensors, or where to place devices in a space. However, Peter Tolmie and others found designing for invisibility was not just a technical challenge. Understanding the social context of human interactions with a system was key, particularly the everyday routines of users’ daily lives and what makes these activities routine in the first place.Footnote 49 Lucy Suchman’s earlier work similarly centred the need to account for social practices of users in technology design, rather than assumptions of designers and categorical logics imposed by machines.Footnote 50 The research agendas in CSCW and HCI became central in establishing the importance of human centricity in Ubicomp system design. The fields provide (often qualitative) research methods necessary to develop vital insights around deployment contexts, system requirements, and evaluation with human interests at the forefront. For example, ethnomethodology and ethnographic approaches were invaluable because design ethnography provided understanding of the social order where technologies would be deployed, and this could inform system design.Footnote 51 The future-orientated nature of Ubicomp provided another way human needs could be forgotten. Scholars like Stuart Reeves and Paul Dourish and Genevieve Bell warn that user practices and concerns are often displaced or ignored in favour of ever-aspirational technologically driven futures that can never be achieved.Footnote 52
This provides a cautionary tale for contemporary IoT and AI trends in the latter chapters, where user interests are still secondary to technological solutionism.Footnote 53 For example, in smart home research examined in Chapter 4, Charlie Wilson and others found that the goals of efficiency, comfort, or convenience are often favoured over human interests and understanding the processes by which technologies come to have meaning and utility in homes. As they argue, ‘Use and meaning of technologies will be socially constructed and iteratively negotiated, rather than being the inevitable outcome of assumed functional benefits’.Footnote 54 This echoes earlier work by Sanna Leppënen and Marika Jokinen, who stated ‘inhabitants themselves make a home and little everyday practices make the known life go on … a smart home should not be smarter than its inhabitants’.Footnote 55 AI deployments similarly often disregard or actively work against human values, interests, users, and non-users who are subject to these systems, particularly with large-scale technology-driven projects. For example, Virginia Eubanks showed that access to public welfare based on automated decision-making disproportionally impacts poorer socio-economic groups;Footnote 56 Petra Molnar found that biometric and mobility AI security systems trialled on vulnerable refugees negatively impacted their human rights;Footnote 57 and Danielle Citron and Frank Pasquale observed automated credit scoring results in a range of intersectional harms.Footnote 58
Returning to the history of Ubicomp, despite decades of research and funding, scholars contested if it truly emerged in its planned form. Ramon Caceres and Adrian Friday posited it remained aspirational,Footnote 59 whilst for Yvonne Rogers its original goals shifted over time.Footnote 60 For Dourish and Bell, it did arrive but in a different guise, where Ubicomp emerged through real-world practice of appropriation and domestication of technologies into users’ daily lives.Footnote 61 As they state, ‘the lesson of the real world of ubicomp is that we will always be assembling heterogeneous technologies to achieve individual and collective effects, and they will almost always be messy’.Footnote 62 The current levels of adoption of mobile phones, tablets, IoT, and now AI, suggest that they were right.Footnote 63 The Ubicomp experience showed that understanding the social is a key part of technology design; a point that needs to be reiterated for current visions of AI-enabled IoT. Issues still arise because systems are still not built in a human-centred way, but this book extends this point further. Accounting for human needs also requires accounting for legal values, protections, rights, and responsibilities in Clever Computing design processes too.
1.2.3 IoT beyond RFID
Kevin Ashton coined the term the IoT in 1999 to describe an approach to tracking items within globally fragmented business supply chains using radio-frequency identification (RFID).Footnote 64 RFID can include battery-powered RFID tags that actively broadcast their ID to receivers or passive RFID tags that require a human to scan with a reader. RFID is now often combined with short-range near-field communications (NFC) and barcode like QR codes too. Critically, RFID provides unique digital IDs to physical things, making them documentable and traceable, where diverse applications have emerged over the years such as:
Monitoring the origin and quality of food through tracing packaging, for example, where fish is caught, packaged, transported, and sold (Figure 1.3).Footnote 65
Tracking high-value items like whisky on supermarket shelves to prevent shoplifting.
RFID wristbands at theme parks, such as Disneyland’s MagicBand+, to track visitors and support payments during their visit.Footnote 66
Developing intelligent fitting rooms like Ralph Lauren’s New York store, where RFID and a smart mirror would identify clothes and then tailor the lighting/ambience for the customer.Footnote 67
Tracking the histories and stories of objects for creative and cultural value, such as RFID storing and logging the battle stories of Warhammer 40K figurines.Footnote 68
RFID use case: food supply chain.
Despite its practical utility, Jordan Firth cautions that RFID created ‘infrastructures of identification’ for sorting, tracking, and surveillance of objects.Footnote 69 This extends to tracking humans too, through the digital proxy of RFID. For example, whilst contactless bankcards or mobile device apps like Apple Pay now dominate the market, originally Oyster RFID travel cards were the primary means for ticketing on the London Underground (with similar RFID cards for travel systems around the world).Footnote 70 This capability allowed both its parent organisation, Transport for London, and the police, to monitor passenger movements, positioning it as a significant data resource in criminal investigations.Footnote 71
The term IoT is no longer constrained to the limited technical capabilities of RFID. Instead, it is now synonymous with the physical sensing and data-gathering ‘tabs, pads, and boards’ of Weiser’s Ubicomp. The IoT exists in domestic settings with smart homes, in public and semi-public spaces with smart buildings and cities, and in workplaces too with industrial IoT and Industry 4.0. Current estimates state that there will be thirty-eight billion IoT devices online by 2030, and the sheer number of consumer products that have become networked is vast, from vacuum cleaners and children’s toys to thermostats and wearable devices.Footnote 72 Nevertheless, the IoT is a difficult term to define. Given its centrality to Clever Computing, I suggest the following working definition of IoT in Box 1.1.
The IoT is networked physical devices and sensors combining hardware and software to conduct data gathering and sharing within an ecosystem of human and non-human actors.
This consolidates elements of definitions from internet standards bodies, advocacy groups, and government agencies, and builds on my own fieldwork on defining the IoT.Footnote 73 The international Internet standards organisation, the Internet Engineering Task Force, defines the IoT as embedded devices with internet connectivity and an ability to communicate on a machine-to-machine basis without direct human control and oversight.Footnote 74 The international non-profit group, the Internet Society, focuses on the use of cloud computing for ongoing data gathering/storage, and miniaturisation of physical devices.Footnote 75 The former EU Article 29 Working Party (which preceded the EDPB) emphasises data sharing and remote control of devices by manufacturers.Footnote 76 In my expert interview study with thirteen lawyers and technologists on the nature of the IoT and regulatory challenges, I found neither group advanced a canonical vision that strictly demarcates what makes a system IoT or not (unlike with Ubicomp).Footnote 77 Instead, participants generally discussed the IoT as encompassing a wide variety of networked sensors and devices, where the smartphone is a key mediating control device. The lawyers’ foregrounded the importance of IoT applications domains like wearables. In contrast, the technologists highlighted the technical attributes of the IoT, such as the partiality of user interfaces, need for pervasive networking, and ubiquitous data collection. To develop a richer IoT definition that underpins Clever Computing, I focus on three key features (and associated risks), namely impacts of IoT sensing the physical world, ongoing control through the IoT, and the ecosystem of data gathering and sharing the IoT creates.
1.2.4 IoT All around Us: Interconnected Devices Sensing the Physical World
IoT devices are networked through techniques like short-range Bluetooth and IoT-specific networking protocols such as Zigbee or z-Wave. Devices can also be connected across cities and entire regions through Long-Range Wide Area Network (LoRaWAN), as exists in Edinburgh and Scotland’s Central Belt.Footnote 78 Such interconnectivity means that IoT devices are not sitting in isolation but can instead share data gathered from the physical world through sensors like accelerometers, cameras, motion sensors, and microphones. Many home IoT products, explored in more detail in Chapter 4, capitalise on this ability to share context-sensitive information, for example, with a front door lock being opened, then triggering a home CCTV camera to stop (or start) recording. A lack of interoperability has meant that cross-device communications have often been fragmented, although the landscape is shifting. For example, the Connectivity Standards Alliance Matter open-source standard has support of IoT manufacturers like Amazon, Alexa, Apple, LG, and Samsung. It creates an ecosystem with a common wireless protocol, Thread, that makes it easier for these devices to interact and break down proprietary standards.
Whilst the growth of networked connected devices allows manufacturers and deployers to use IoT devices in novel ways, it also surfaces socially problematic consequences. Apple AirTags highlight the need to account for human-centred, socio-technical, and legal considerations in both design and operation. Apple AirTags are coin-sized Bluetooth devices that users purchase to attach to their keys or luggage to keep track of them. The AirTags become part of a large peer-to-peer network of Apple phones, tablets, and laptops, which can help pinpoint where an AirTag, and any attached property, is. Whilst this ability to digitally search the physical world has convenience for those who often lose things, AirTags have also been appropriated for significant harm. Stalkers have secretly attached them to their victim’s rucksack or clothing. In a Welsh case, a court served an offender a restraining order and prison sentence for attaching an AirTag to their ex-partner’s car to follow and harass them.Footnote 79 Tragically, in the United States (US), there have been cases where stalkers have used information from AirTags to track and ultimately kill their victims.Footnote 80 Current-generation AirTags do take some privacy measures at the architectural level. This includes avoiding storing location data on tags, not sharing the identity of the device that discovered it, encrypting communications data, and providing phone notifications about unidentified AirTags.Footnote 81 A class action suit in the US argues that these measures are insufficient; for example, when phones are not on the Apple network, users will not be notified about unknown tags, unless they have a specific third-party app.Footnote 82
Accounting for human values in design, particularly privacy and safety, is central to addressing risks that can arise during use when devices are appropriated beyond the intentions of manufacturers. Finding mechanisms to surface and address risks effectively is necessary. For example, Diana Freed and others research on the relationship between intimate partner violence and technology suggests the need to conduct safety reviews on interface design to reduce, or degrade, usability of devices for abusers and to anticipate potential abuse, much like the process of security penetration testing.Footnote 83 I will return to this topic in Chapter 3.
1.2.5 IoT Bricking: Ongoing Control over Devices and Lack of User Agency
As with many digital products, the IoT is underpinned by an ongoing service relationship between manufacturers and users. Whilst once manufacturers had limited control after software was developed and deliveredFootnote 84 the shift towards more iterative, flexible software development methodologies, like agileFootnote 85 and now DevOps, provide more ongoing control.Footnote 86 Here, manufacturers can tweak user interfaces, adjust what data is harvested into their cloud servers, push over-the-air software updates, or remotely disable or enable functionality of onboard hardware.Footnote 87 As the development and operation of the IoT are more closely coupled, greater feedback on use is possible, allowing manufacturers continued abilities to oversee their software and data processing across the whole lifecycle of an IoT system.
Importantly, users often have limited agency to shape or change manufacturer decisions, which can lead to backlash. The Revolv IoT hub, which controlled different smart home devices, was remotely bricked after Nest acquired the company. Like the Moxie robot, the hub was rendered obsolete when support was removed, consigning an otherwise fully functioning device to e-waste.Footnote 88 In 2022, BMW offered customers a subscription to unlock and use their heated seats (which are already installed), a move they later dropped in 2023 due to criticism.Footnote 89 Whilst ‘planned obsolescence’ is a business decision to limit support periods or design lifespan for components, intellectual property laws also provide a route to IoT device redundancy through enforcing ongoing control. Technical protection measures and digital rights management have underpinned the denial of users ‘right to repair’ to fix, upgrade, or change their technologies, famously in the US with John Deere tractors.Footnote 90 An extreme example from 2023 shows how much power is retained where Polish train manufacturer Newag was alleged to have been using software that prevented repair of trains by unofficial repair shops. This meant entire trains were being ‘bricked’ and left out of service. In response, an ethical hacker collective, Dragon Sector, was hired by the independent repair shop to diagnose and fix the problem. Code on the train stopped it starting if a GPS tracker was detected in an unofficial repair location or unofficial components that lacked authentication codes were used. Manufacturer Newag threatened to sue the hacker collective and claimed their actions made the trains unsafe.Footnote 91
1.2.6 IoT as Invisible Witnesses: Impacts of an Ecosystem of Data Gathering and Sharing
IoT devices set up data processing infrastructures that an ecosystem of actors may want access to, including manufacturers, deployers, and third parties like insurers and law enforcement. The visibility of data IoT facilitates poses socio-technical and legal concerns, particularly around appropriate data protection and design controls to manage who can access and share the IoT data in the first place (an area considered at length in Chapter 4).Footnote 92 For now, the Ring connected and camera enabled doorbell and iRobot Roomba robotic vacuum cleaner provide examples of types of data gathered by IoT devices and how daily life can be made visible beyond the home.
An investigation by the US Federal Trade Commission in 2023 found that Ring had failings in controlling access to videos from their domestic camera devices. Externally, insufficient privacy and security measures allowed hackers to access the videos and accounts of 55,000 customers. It was not just informational breaches where hackers tormented vulnerable customers too. Internally, their own employees and contractors were also found to be accessing footage to train AI models without user consent and watching female users in their bedrooms and bathrooms.Footnote 93
The Roomba uses an ‘Imprint Smart Mapping’Footnote 94 system to generate detailed maps of a user’s house to help the device navigate different rooms and obstacles like furniture.Footnote 95 According to iRobot’s UK Privacy Policy, the resulting ‘floorplans, object detection, Wi-Fi heat maps, and images’ are stored in the cloud to be shared with others.Footnote 96 This data is valuable, and it was central to a deal where Amazon tried to buy iRobot Roomba for £1.1 billion in 2022,Footnote 97 a purchase that later fell through in 2024 in part due to competition concerns.Footnote 98 Showing the breadth of third-party access possible, the policy clarifies that data collected can be shared with subsidiaries of iRobot, service providers involved with payment and data analytics, any purchasers in a merger, and even law enforcement for detecting, preventing, or addressing crimes like stalking and fraud.Footnote 99
In a paper with Diana Miranda and Lena Podoletz, I examined how IoT devices in the home are becoming ‘invisible witnesses’ for everyday life, and they are increasingly being used in policing, posing risks to privacy from long term, domesticated forms of surveillance.Footnote 100 The Roomba would become yet another device with testimony to provide that can corroborate or challenge narratives around crimes that occur in the home. This is not hypothetical and has occurred in numerous cases in the US and UK, where data from smart watches,Footnote 101 doorbells,Footnote 102 pacemakers, or Apple health dataFootnote 103 has been instrumental in murder investigations. In one case in the US state of Arkansas, the murder of Victor Collins involved a combination of IoT evidence, including unexpectedly high-water usage logged by the smart meter in the middle of the night (to wash away blood from a crime scene), voice data from an Amazon speaker, and motion data from the alarm system.Footnote 104
Newer Roombas also include onboard cameras to support computer vision for object detection. This shift to using AI within IoT systems further broadens the ecosystem of actors who can surveil how users live and their intimate routines. In 2020, footage of Roomba test users sitting on the toilet famously surfaced on social media.Footnote 105 This raised questions about the data pipeline, where it transpired that workers hired by an AI gig economy platform to label the training data for AI classification had allegedly uploaded the shots online. This highlights how far data can travel beyond the physical context in which it was initially collected and also the exploitative prevalence of hiring poorly paid gig economy workers, often in the Majority World, to annotate data e.g. on platforms like Amazon’s Mechanical Turk.Footnote 106 I now examine more systematically how AI is increasingly being integrated into IoT systems, under the new term Clever Computing.
1.3 What Is Clever Computing?
My working definition of Clever Computing, provided in Box 1.2, foregrounds the closer coupling of AI and the IoT, where the materiality of IoT devices is being integrated with data-driven AI ecosystems, surfacing connected and emergent regulatory and design challenges. I present six ways AI is used with the IoT to (1) facilitate new modes of user interaction, (2) improve current user interactions, (3) support distributed data analysis, (4) make sense of and interpret data, (5) extend existing system functionality, and (6) enable greater autonomy. However, these are not exhaustive, and many examples throughout the book further show the increased use of AI in the IoT. But before examining these, I need to reflect on the nature of AI, specifically machine learning (ML), in Clever Computing.
Clever Computing is the convergence of interactive physical computing systems which digitally sense, observe, and actuate in the analogue world (IoT), with the inferential and data-driven infrastructures of AI.
Like the IoT, AI is not easy to ringfence, particularly given extensive ongoing hype around the term.Footnote 107 Leading AI experts Stewart J. Russell and Peter Norvig state that AI is ‘the study of [intelligent] agents that receive precepts from the environment and take action’.Footnote 108 AI has a long history, first emerging in 1956 at a workshop at Dartmouth University, USA, with researchers working on previously unaligned topics like neural nets, information theory, and cybernetics.Footnote 109 AI is often framed as creating systems that can complete tasks traditionally requiring human intelligence. This draws on foundational topics including logical reasoning, knowledge representation of the world through design of taxonomies, ontologies, and supporting machine perception of surroundings. As such, AI design encompasses many disciplines including psychology, philosophy, engineering, and mathematics. However, the socio-technical nature of living with and interacting with AI puts humanities, social sciences, and legal perspectives at the forefront of AI development too.Footnote 110
For Clever Computing, ML is a key AI technology underpinning voice and natural language processing, computer vision, recommender systems, and generative adversarial systems. Samuel originally defined machine learning as ‘a field of study that gives computers the ability to learn without being explicitly programmed’.Footnote 111 With Clever Computing, ML often involves task-focused work, such as sorting, classifying, filtering, or making inferences and decisions about data. Simply stated, a computational model is trained using input data, either through supervised, unsupervised, reinforcement, or combined learning approaches. A supervised learning model is provided with labelled training data (e.g. image A is a cat, image B is a dog) to explicitly help the model to differentiate inputs (Figure 1.4). An unsupervised learning approach uses algorithms that spot emergent patterns and relationships in unlabelled data. The model refines its assessment over time as it is fed more data by identifying variables and relationships between them; for example, a cluster of pixels in the bottom corner of an image may indicate a feature. Reinforcement learning rewards the model when it takes correct actions in the environment it is functioning in. Once trained to differentiate input data or spot patterns, the model can then make sense of new datapoints; for example, when presented with a new image, a trained model could state if it believes the image is a cat or dog, and with what degree of certainty (e.g. 64 per cent this is a chihuahua dog, not a blueberry muffin).
Example ML classification.
With unsupervised ML models, the complexity and multiple layers shaping decisions make it difficult to determine how variables have shaped an outcome. When using deep learning and convolutional neural networks, for example, the model is often treated as a ‘black box’, where inputs and outputs can be accounted for, but algorithmic processes of analysis are more difficult to trace and verify. There have been greater calls to understand how ML functions and gives rise to certain outputs, particularly when interacting with humans. So-called Explainable AI (XAI) aims to provide insights by tracing the factors shaping model decisions; for example, why a prediction about a face is classified as happy, sad, angry, or scared. One XAI technique, Shapley Additive Explanations (SHAP), assigns values and a score to different features in a model to better understand how they feed into a specific prediction.Footnote 112 XAI techniques can help developers with auditing, verifying, and validating that the model is functioning as intended, although it is easier with ‘good old fashioned’ rules-based AI where decision paths are easier to trace.Footnote 113 Generative AI uses transformer-based neural networks, trained on large corpuses of multimodal data to create new audio, text, code, and video based on user prompts such as requesting creation of short stories, poems, or artworks.Footnote 114 Widely used examples include large language models with chat-based user interfaces such as Google Gemini, OpenAI’s ChatGPT, Anthropic’s Claude, or Microsoft’s CoPilot. As discussed in more detail in Chapter 5, generative AI poses extensive legal and ethical issues.Footnote 115 For example, the provenance of training data is often questionable, where it has been scraped at scale from online sources without due consideration for copyright, bias, or privacy.Footnote 116 Before considering the Accountable Design approach in detail in Chapter 2, I will now outline different ways in which AI is being integrated into the IoT and the resulting regulatory concerns.
1.3.1 AI Is Providing New Interaction Techniques with Physical Clever Computing Systems
The ways users interact with IoT devices and sensors are changing through the integration of AI. As previously noted, the combination of voice analysis and natural language processing enables users to speak to devices like speakers, televisions, or in car entertainment. Voice data is gathered from microphones and fed into ML models to answer myriad user search queries, bringing sensing into different contexts of everyday life. Despite convenience, these systems have raised ethical and legal concerns such as gender-based harms of assigning female names like Alexa and default female voices to assistants,Footnote 117 racial harms such as not detecting black voices properly,Footnote 118 and wider privacy concerns around ensuring adequate control of often opaque data flows for voice data.Footnote 119
It is useful to dig deeper to consider data flows within a real-life system: Amazon Alexa. Like with the Roomba, examining the privacy policy shows the design choices taken alongside strategies to manage legal risk.Footnote 120 I focus on how the system manages cloud interactions through wake words and how it manages user identification and control over recordings (Figure 1.5).
Voice data flows in a smart speaker system.
The policy frames use of wake words as a ‘data minimisation’ approach, stressing Alexa is not always recording. By default, it only records when a wake word is detected for a few seconds before until the system determines the query has finished.Footnote 121 Under the EU Data Act 2023,Footnote 122 recorded and pre-wake word data should be accessible to users by default, without relying on data subject access rights in Article 15 GDPR. The Amazon policy clarifies that the user is interacting with a cloud-based service, and a recording of their data will be sent to the cloud for analysis by speech recognition tools, using natural language processing. Amazon is explicit that recordings are part of training data for supervised machine learning to improve the product. The user experience design helps demonstrate the temporality of that interaction through the use of colours. There is visual feedback that data will be sent to the cloud as the system light colour turns blue. Similarly, when the microphone is switched off, the light will turn red. As discussed later, the affordances of devices to communicate with users can shape their experience but also address legal obligations, such as consenting to data processing. William Seymour and others have explored if GDPR-compliant consent can be achieved through verbal interactions with voice assistants. They found challenges in communicating sufficient information to end users to be compliant, and data controllers may instead rely on another legal ground altogether for processing.Footnote 123
Voice recordings can be managed, reviewed, or deleted as users see fit. Deletion can be done automatically or after three to eighteen months, where associated voice transcripts will also be removed. Users can make a request in real time, right after they talk to the speaker.
Whilst this is what the privacy policy states, real-world practice has been different. With the Alexa Dot smart speaker targeted at children, parents could ask Amazon to delete voice recordings, transcripts, and geolocation data about their children. However, Amazon instead ignored these requests, retained the data, and even used it to refine their algorithms to better work with children, whose voices had unique features for speech recognition. As a result, they were charged in 2023 by the US Federal Trade Commission and Department of Justice and fined $25 million for breaching data protection rules.Footnote 124
1.3.2 Integration of AI Systems to ‘Improve’ User Interactions with a Clever Computing System
Since late 2023, both Amazon and Google have been integrating generative AI into Alexa and Assistant, to enable more natural, human-like conversations between users and IoT devices. With Alexa+, Amazon acknowledges that they have extensive sensing capabilities in homes through speakers and are now using features of those to improve conversation. Non-verbal cues are being integrated by using cameras, presence detection, and voice analysis. This also includes moving away from using wake words such as saying ‘Alexa’ to begin the query or using face detection to capture the start of discussions more naturalistically. Amazon are aiming to create more ‘empathic’ interactions, where Alexa will adapt depending on the conversation topic and how the user is speaking e.g. if they are hesitating/pausing more, Alexa will slow the pace of the responses.Footnote 125 With Assistant, Google has been integrating their LLM system, Gemini, to create a more personalised experience, to help manage daily search tasks, generate text or images, and increase integration with other Google services.Footnote 126 The promise is creating more contextually aware, personal, helpful user experiences. This augments existing recommender systems used for personalisation, an approach familiar to many through music recommendations on Spotify, TV show suggestions on Netflix, or online shopping. Here, a user’s past activity can be used to infer their preferences, while reinforcement learning or AI model training aims to help predict their future likes and dislikes. Integration of generative AI compounds existing data protection and security concerns around the IoT. For example, Meta plan to use conversations with their Meta AI Chatbots in targeting advertising.Footnote 127 On the security front, security researchers showed in 2025 that Gemini could be hacked to provide control of physical devices in the home, including the lights, boiler, and window shutters. By injecting prompts into Google Calendar invitations, which when summarised by Gemini would run the malicious script, hackers could then have control of the devices, creating real-world safety risks in smart homes.Footnote 128
1.3.3 AI Enabling Distributed Data Analysis
A shift in AI-enabled IoT has been an ability to utilise federated machine learning (Figure 1.6).Footnote 129 This aligns with the trend towards putting computational resources and data storage at the edge of the network, so-called edge computing.Footnote 130 As IoT devices can be more computationally well-resourced, data can now be processed locally, reducing the need to stream all data to the cloud. Instead, an AI model can be distributed across different IoT devices in the network and then fine-tuned on data sourced locally, sharing only limited information with the global model to improve it. Analysis of sensitive data from home devices or wearables can be conducted in situ and closer to the end user.Footnote 131 This helps address concerns around overreliance on cloud such as data latency or insufficient bandwidth, but more crucially can enable more privacy-preserving architectures. For example, by keeping data more proximate to users, avoiding third-country international data transfers outside the EU, it avoids many of the regulatory concerns around data being sent to US servers as seen with the Schrems decisions.Footnote 132 Edge-based approaches can support increased transparency around data processing and provide greater control for data subjects over their personal data, as discussed further in Chapter 3 with the ‘Databox’ architecture.Footnote 133
Federated machine learning.
1.3.4 AI to Make Sense of and Interpret Data
Computer vision is increasingly integrated into camera-based IoT systems. It is used to analyse footage to detect features or patterns in video, often recognising objects and faces. Such systems are considered at length in Chapters 3–5, so only mentioned briefly here. Example applications like smart home doorbells or locks, discussed in Chapter 4, use cloud-based analysis to verify and authenticate permitted home visitors and flag unexpected ones.Footnote 134 In contrast, facial recognition on mobile devices ordinarily involves more privacy-preserving storage of facial templates in the trusted execution environment on the device itself. Live facial recognition technology (FRT) by law enforcement is examined in Chapter 5. Object detection in airports or train stations is used to spot unattended luggage in real time, considered in Chapter 3.Footnote 135 Even animals cannot escape the gaze of Clever Computing. In 2025, Ring cameras added an AI feature to track down lost dogs in the local neighbourhood. If a pet is lost, users can upload a photo to the Ring community app and cameras in the area become part of a distributed ‘dog-veillance’ system, scanning for the missing pet and alerting the owners if it is spotted.Footnote 136 How long until the system is repurposed for evidence gathering for identifying dog fouling? Occasionally, AI also appears to be in use when it is not. Amazon’s Fresh stores used a ‘Just Walk Out’ system, which allegedly used AI to detect what items customers were buying without the need to go to a check-out. It transpired that AI was not used at all and 1,000 staff in India were remotely watching footage and labelling data.Footnote 137
1.3.5 AI Is Being Layered into Existing Clever Computing Systems to Extend Existing Functionality
Emotion recognition technology is often layered into the IoT. ERT analyses biometric and physiological data to infer intent or the emotional state of subjects with on- and off-body sensors.Footnote 138 ERT is being used in wearables like fitness bands and smart watches but also more interestingly, smart jewellery. For example, the Oura Ring uses sensors like infrared photoplethysmography (PPG)Footnote 139 to track physiological signals like blood oxygen levels, body temperature, and heart rate, alongside sleep patterns.Footnote 140 Similarly, the Leaf Urban is a leaf-shaped piece of jewellery made of wood and silver that tracks similar datapoints.Footnote 141 When combined with an app, both systems promise to support well-being, for example, by tracking stress and supporting strategies to increase resilience (by suggesting breaks) and even learning menstrual cycles to make inferences about fertility. Samsung’s smart ring, shares similar features and aims to link into wider health data and medical records, coupled with an LLM based AI well-being assistant to provide more personalised support.Footnote 142
Beyond wearables, affective computing is being layered into our two prior examples, with off-body sensing. Voice-based systems are analysing for timbre, pitch, and tone, again to infer an individual’s mood. Camera-based ERT analyses faces for the presence of facial action units such as upturned lips or furrowed brows that are cross referenced to basic emotional states to make inferences about how the subject is feeling. The science underpinning this approach is often challenged, for example, computer vision systems Microsoft AI and Face ++ were both found to associate negative emotion more frequently for black faces more than white faces.Footnote 143 Yet the (flawed) approach of trying to sense how someone is feeling is being rolled out by advertisers, police, security services, and recruiters, despite legal concerns. ERT and its regulatory challenges are discussed at length in Chapter 5.
1.3.6 Enable Greater Autonomy of Clever Computing Systems
Due to physical sensors and actuators, the IoT can directly respond to real-world inputs, increasingly autonomously. This could be as simple as connected light sensors that reach a certain level of lumens and then trigger actuators to open blinds around the house. However, not all IoT requires AI; for example, smart meters that only log and visualise hourly energy use data to an in-home display but conduct no prediction and recommendations around future use. Similarly, not all AI requires real-time input from physical sensors; for example, automated content moderation systems focused purely on identifying harmful online content on social media. However, with Clever Computing, the focus is on the convergence of physicality of the IoT coupled with inferential AI infrastructures.
As AI may support increased autonomy of IoT systems, more advanced Clever Computing systems drawing together multiple physical sensors and AI technologies migrate into the territory of robotics. Robotics research and development has traditionally involved more complicated, embodied machines interacting with their environment with a higher level of autonomy, completing a wider range of complex tasks, such as autonomous mobile machinery, as discussed in Chapter 5.Footnote 144 Autonomous vehicles utilise AI with onboard sensors, like LiDAR, to support navigation and make real-time assessments about collision hazards and how to respond. Yet, it can be difficult to strictly demarcate between existing technological labels. For example, is a Roomba vacuum cleaner a robot or an IoT device? What about a home security drone? Is a smart watch that infers emotions from collecting our heart rate and makes lifestyle suggestions AI or IoT? As I explored in work with Natalie Leesakul and Dominic Reedman Flint, domestic IoT devices are increasingly being augmented and adapting with the growth of home robotics.Footnote 145 Thus, as systems get more advanced and autonomous due to the complexity of sensors and actuators they are using, the boundaries between Clever Computing and robotics will blur further. The emergence of ‘agentic AI’ i.e. ‘autonomous systems designed to pursue complex goals with minimal human intervention’ is further confounding this blurring.Footnote 146 Agentic AI involves systems that can adapt, react and multi-task with greater independence, often over longer periods of time, not just assisting humans in tasks e.g. autonomous cybersecurity threat detection on networks.Footnote 147 Currently, Clever Computing provides a useful umbrella term to encapsulate the ways interactive, material IoT systems will continue to change through the addition of AI.
1.4 Conclusions
This chapter proposed the nascent concept of Clever Computing. Physical IoT sensing devices and ecosystems, which have been around for some time, are now changing through the integration of advanced data-driven AI techniques. AI is augmenting the IoT in many ways and will continue to do so. Currently, it is altering how users interact with devices, supports new data management architectures, layers new AI functionality into systems, and increases autonomy of decision-making by IoT devices. But AI-enabled IoT will continue to adapt and shift.
Clever Computing is a necessary new label that encapsulates the convergence of the data, software, and hardware aspects of the IoT, with AI infrastructures, as an analytical focus. It centres the importance of sensing the physical world and the associated socio-technical and legal impacts of materiality posed by data-driven AI. It foregrounds how AI further amplifies regulatory issues already present in the IoT, through increasing the complexity and uncertainty of how these systems interact in the world.
The chapter also looked backwards to contextualise these shifts within the history of precursors to Clever Computing, particularly Ubicomp. As the IoT and AI become more closely coupled, examining Ubicomp research and development showed how user requirements were often neglected in favour of techno-driven visions that promised better futures but did not deliver. These systems were often not integrated into users’ lives in the clean, unencumbered way that designers anticipated. Instead, the reality of technology deployment surfaced the messy, social complexities that design needs to contend with and which shape processes of domesticating technologies into everyday life. This shows why human-centred approaches that foreground user best interests, which extends to legal rights and responsibilities, are central to Accountable Design.
Present-day AI-enabled IoT often adopts a similar narrative of fixing or at least improving responses to complex societal problems through technology, whilst neglecting the complexity and lived experience of users, often leading to significant socio-technical and legal harms when released into the wild. In drawing inspiration from scholars who championed the need to understand the social in system design processes, I now propose Accountable Design to argue for accounting for the legal in design. This is necessary to move towards building more secure, trustworthy, safe, and privacy-preserving Clever Computing.
