This chapter examines technical methods for evaluating AI applications in legal contexts, and addresses the need for standards to assess AI system performance in an industry where quality standards for human work remain largely subjective. It identifies three primary evaluation approaches: categorical evaluation using precision, recall, and F1-score metrics; numerical evaluation of continuous predictions; and more complex evaluation methods of open-texture outputs.

The chapter demonstrates these methods through case studies, including technology-assisted review in e-discovery, legal research tools with varying accuracy rates and significant hallucination challenges, and access to justice applications like Beagle+. These applications show how iterative improvements can substantially increase accuracy through enhanced prompting and retrieval-augmented generation.

The analysis emphasizes that evaluation must extend beyond technical accuracy to include real-world utility, consumer protections, and security considerations. The chapter concludes that establishing robust evaluation frameworks is essential for responsible AI deployment in legal services, particularly given the legal profession’s current reliance on subjective quality assessments rather than empirical standards.