This chapter challenges the assumption that placing a “human in the loop” is an adequate safeguard against the harms of AI. It introduces the concept of automation bias – a cognitive tendency for human decision-makers to over-rely on automated systems, often ignoring contradictory evidence or their own training. The chapter demonstrates how the human operator can become a mere “rubber stamp” for a flawed machine.

The risk is magnified with generative AI (GenAI), whose polished, authoritative-sounding outputs and ubiquitous availability deepen trust and erode critical thinking, leading to both errors of commission (following bad advice) and errors of omission (failing to monitor for errors). The chapter concludes that effective mitigation requires more than just human oversight; it demands intentional changes to system design, organizational culture, and human training to foster a healthy skepticism toward AI recommendations.

Chapter 14 explores the integration of CERIC with generative artificial intelligence (GenAI) tools, discussing both positive aspects of cognitive off-loading and reading/writing development and challenges in avoiding overreliance while ensuring comprehension and ethical practice. This chapter begins by defining GenAI and its relevance to academic research, then delves into how CERIC and GenAI interact, highlighting both benefits and limitations. It presents a case study comparing the outcomes of a GenAI (Humata.ai), and CERIC in identifying a paper’s central argument. This chapter further discusses the integration of GenAI into CERIC review exercises, examining the benefits of cognitive off-loading and the potential for enhancing learning while avoiding cheating. It also covers selecting and applying appropriate GenAI tools in CERIC reviews. Various case studies illustrate the practical application of CERIC and GenAI with the potential for improving critical reading skills.

The relatively new world of ESG indicators displays many similarities with the original markets for ratings and benchmarks, but it also has some distinguishing features. This chapter explores to what extent the regulatory strategies that were developed in ‘traditional’ financial law to support confidence in ratings and benchmarks can be exported to the ‘new’ world of ESG finance, and concludes that policymakers should be cautions when transposing rules. This is especially the case with ESG ratings. In this area, credit ratings are the immediate reference for ESG rating regulation also, because of the common label of ‘rating’, which is rather misleading, and of the anchoring effect this entails. First, the assessments underlying ESG ratings are often more subjective than those supporting traditional indicators, due to their multivariate nature. Second, the risk of regulatory failures connected to the authorisation and registration labels also seems higher in the ‘new’ world of sustainability. The chapter analyses the new ESG Ratings Regulation and the Benchamkr Regulation against this backdrop, and highlights the suboptimality of some policy choices.