This chapter provides an overview of user experience (UX), particularly focusing on the UX of human-centred machine learning (HCML). In this chapter we will focus on “less standard” measures that are particularly relevant in HCML; other measures such as accuracy are already covered very well in standard ML approaches. Central to our notion of HCML is that the set of people who are users of ML is much broader and also distinctive compared to other domains. For example, these “users” might be people who use the final system in their work or everyday lives, or people who help develop the HCML, or indeed regulators and assessors of these systems. In this chapter, we will outline what aspects to measure for assessing the UX of HCML systems and how to measure them. We conclude by looking at open research areas in measuring the UX of HCML.

Using examples from Bishop’s letters and diaries, as well as more recent theoretical accounts, the introduction explores various meanings of the word ‘style’ and their pertinence to Bishop’s work. The introduction highlights the predominance of biographical criticism in works on Bishop (and other twentieth-century writers) and instead argues for the importance of stylistic criticism. This chapter goes on to delineate two broader trends in contemporary literary criticism – historicist on the one hand, formalist on the other – and outlines their shortcomings in understanding the nuances of particular poems and literary works. The chapter then outlines this book’s focus on several aspects of style across Bishop’s entire oeuvre, including cliché, simile, allusion, and correctio. The introduction ends by arguing that aesthetic evaluation and judgement are central to the responsible and rigorous practice of literary criticism.

This chapter examines two dimensions of AI assurance for systems designed to assist unrepresented litigants: (1) reliability and (2) privacy and accountability. Reliability encompasses accuracy, calibration, robustness, and bias. Privacy and accountability involve balancing data needs with the protection of sensitive information and ensuring the responsible deployment of AI. These considerations take on heightened importance given unrepresented litigants’ unique vulnerabilities and legal contexts.

Reliability standards must be higher for unrepresented litigants, who cannot verify AI outputs, than for legal professionals, who can exercise oversight. Privacy and accountability challenges arise from AI systems’ need for vast training datasets containing sensitive legal information, creating tensions between the effectiveness of these systems and data protection rights under frameworks such as the General Data Protection Regulation (“GDPR”) and the EU AI Act.

Effective AI assurance requires interdisciplinary collaboration between technologists and legal stakeholders to develop appropriate methodologies for defining, measuring, and verifying both reliability and privacy protections in legal AI systems.

This chapter shows how Minerva authors championed the Press, taught readers how to read them and helped to shift the culture in proto-Victorian ways. It collects together the solutions that women authors proposed to the range of domestic, social and political issues they tackled, argues that their iterative imitations created a community of readers, as well as of writers, and evaluates Minerva Press fiction by the Aesthetics of Reuse.

This chapter addresses the evaluation of qualitative causal inferences, focusing on how to assess whether claims about causal continuity and discontinuity are credible. Building on the framework of bias-free causal graphs, it explains the iterative process of testing causal relationships through hypothetical interventions. Three types of claims are central: that a causal process could have produced the outcome (front-door continuity), could not have done so via a pre-treatment mechanism (back-door discontinuity), and would not have occurred through other unrelated causes (side-door discontinuity). The chapter develops a multilevel evaluation strategy. At the micro-level, it introduces three measures – theoretical plausibility, mechanistic credibility, and descriptive validity – for judging individual claims about hypothetical interventions between adjacent nodes. At the path level, it examines how these claims aggregate across front-, back-, and side-door paths to assess whether bias can be excluded. At the graph level, it considers the overall plausibility of substituting a bias-free graph for a bias-saturated one, outlining possible combinations of satisfied and unsatisfied criteria. Finally, the chapter proposes remedial measures for cases where bias cannot be eliminated. These include signing the bias, estimating relative magnitudes of effects, and discretizing outcomes. Even when systematic error remains, the method enhances transparency and strengthens the epistemic value of qualitative research.

The Conclusion urges us to consider practices that lead to becoming ‘valuable people’ as something that goes beyond overcoming stigma to changing the evaluations that define what is good. It brings the discussions about values together with a final example of how my interlocutors pursued valued inclusion, by embracing a biomedical model of personhood where people are judged on their minds rather than on their bodies. This draws attention to the wider relevance of questions of entitlement, distribution, and values: wherever my interlocutors went, discussions of values followed.

Assembling and evaluating warrant for your claims involves giving reasons why these claims might be true or false. Such reasons may be understood as consisting of conjunctions of factual and relevance claims which may be adduced in favour of the claims in question. These claims themselves may in turn be warranted by further such conjunctions, and so on through indefinite higher ‘levels’ of evidence. We show how to structure relevant claims into an evidence role-map. We set out how to apply this to indirect local evidence, referencing a situation-specific causal equation model of each causal step. By reference to our pluralistic account of causation, we identify eight categories of evidence for each step. We introduce a notation for this approach and illustrate it using the Barbados sugar-sweetened beverage tax example, describing its use in post-hoc evaluation and ex ante prediction.

Exploring the use of our methods has highlighted the importance of defining the boundaries of the system being studied and attending to features of underlying systems that afford causal processes. We note that our limited application of our methods cannot confirm that all the implementation actions were necessary, though our examination of L shows how this might be shown by broader study. We note that, while our methods may seem daunting, they’re no more detailed than statistical methods used in quantitative research and have practical advantages in supplying more detailed qualitative evaluations. They may also be used for ex ante appraisal. We stress the importance of understanding mechanisms in practical research and note that actual research involves back-and-forth between evidence and theories of change, even though we detail the former first in our evaluation. We note the value of such evaluations in combatting problems such as, relevantly, costly failed reforms.

As a further example of our approach, we offer a detailed post-hoc case study of the implementation of the Signs of Safety practice approach in child protection. This is chosen because it has been deployed in multiple places but evidence suggests it is often only partially implemented. Our case study is of a child protection agency M which evidence shows achieved a high standard of implementation. In this case study we had a rich evidence base to work with, both of background research on child protection implementations able to provide evidence from afar of systemic factors that might be causally important and of local evidence available from extensive contemporaneous documentation in M. Our method helps to illustrate and explain the range of changes needed to support Signs of Safety as the organisational practice approach in M.

Different sorts of evidence for a singular causal claim impact its overall warrant in different ways. We show how to assemble the evidence we have identified to provide an overall assessment of its strength of support/warrant for claims at issue, giving a rough guide to the significance of particular kinds of evidence within our schema and the identification of gaps in the evidence. This cannot be too formal a process (you shouldn’t pretend to more precision than is available) but we suggest the use of arguments to make sense of relations between pieces of evidence and causal claims, as per Fuller et al., and proper application of the ‘weakest link’ principle. We offer a catalogue of nodes to incorporate into such arguments based on our evidence categories.