1. INTRODUCTION: LEGAL INFORMATION PROFESSIONALS AND THE NEED FOR ALGORITHMIC LITERACY
Given the prevalence of technological advancements in legal practice, legal information professionals and the information users they serve increasingly engage with new technologies that retrieve, suggest, predict, summarize, and stipulate the law on specific areas based on user inputs and queries processed by intelligent systems. The pervasiveness of artificial intelligence and its effects on the legal practice are the subjects of discussion from authors such as Swansburg (2021)Footnote 1, Legg and Bell (2020)Footnote 2, Liu et al. (2020)Footnote 3, Intahchomphoo et al. (2020)Footnote 4, Soares (2020)Footnote 5, Ridley (2019)Footnote 6, Johnson (2018)Footnote 7, Baker (2018a)Footnote 8, Alarie et al. (2018)Footnote 9, Ashley (2017)Footnote 10, Hilt (2017)Footnote 11, and Goodman (2016)Footnote 12. Legal information professionals, regardless of library or work environment, may, in time, require some competence in both advanced legal technologies and the legal issues related to the use of algorithms, legal data, and artificial intelligence tools in society. However, along with the potential for increased productivity and information accessibility within the legal field, artificially intelligent systems used in the delivery of legal research and information services have been noted as abstract, opaque, and oftentimes proprietary.
Swansburg (2021)Footnote 13 notes how advanced legal technologies are an enigma for most practitioners, as it is challenging to possess a clear sense of how the information, answers, and work product of artificially intelligent tools are created. Ethical issues emerging from decreased human oversight and increased reliance on these ‘black box’ technologies have been discussed by other authors such as Callister (2020)Footnote 14, Nayyer et al. (2020)Footnote 15, Wojcik (2020)Footnote 16, Yu and Ali (2019)Footnote 17, Bhattacharya et al. (2019)Footnote 18, Gardner (2019)Footnote 19, Henry (2019)Footnote 20, Ridley (2019)Footnote 21, Turner (2018)Footnote 22, Baker (2018a)Footnote 23, and Kennedy (2017)Footnote 24.
The literature reviewed suggests a need for algorithmic literacy and increased algorithmic awareness in the legal information community on account of the growing existence of advanced technologies and artificially intelligent systems used within the legal profession. Kitchin (2017)Footnote 25 discusses the widespread adoption of algorithms and intelligent systems in various disciplines and posits their increasing role in exercising power via information processing and automation. Koenig (2020)Footnote 26 similarly illustrates how algorithmic processes and the use of artificially intelligent systems is expanding. However, both note a lack of user understanding and clarity regarding how data is processed to make consequential decisions, thus prompting a need for users’ critical examinations, algorithmic engagement, and algorithmic literacy.
The definitions, characteristics, and requirements of algorithmic literacy may vary with sources. Clark et al. (2017)Footnote 27 define algorithmic literacy as a new competency aimed at finding transparency for the invisible logic embedded in software interactions. While this may be distinguished from computational thinking, Shute et al. (2017)Footnote 28 identify algorithms as part of computational thinking's six facets and suggest that parallels maybe drawn between learning conceptual thinking concepts and learning the data input and output structures associated with artificially intelligent systems.
Bakke (2020)Footnote 29 links algorithmic literacy with the concepts of digital and information literacy by noting how the field of computers and composition recognizes multiple types and layers of digital literacy, with the common thread being that digital literacy involves the ‘how’ of technology along with the ‘why.’ Algorithmic literacy is described as a component of information literacy instruction, and attention to algorithmic literacy entails evaluating individual information sources along with taking a step back and considering the processes used to generate those sources. Rainie and Anderson's (2017)Footnote 30 report containing seven major themes surrounding the algorithm era highlights societal changes and the growing need for ‘algorithmic literacy, accountability, and oversight’. Although algorithmic literacy is not given a fixed definition in this report, it is cited as going ‘beyond basic digital literacy’, involving accountability processes, oversight, and transparency.
Leander and Burriss (2020)Footnote 31 claim that, while artificial intelligence is a prevalent topic in education and in public discourse, critical literacy theories have not sufficiently accounted for how artificial intelligence has changed what it means to be ‘critically literate.’ They further state that, in order to think about the relationships between artificial intelligence and education, the field needs not only more rigorous engagement with changing technologies, but also new ways of conceiving digital literacies apart from those found in representational paradigms. Horton (2017)Footnote 32 states that the teaching of digital literacy should be a part of the teaching of general legal skills. Although Johnson (2018)Footnote 33 does not explicitly discuss the concept of algorithmic literacy, the standard to which artificially intelligent systems are to be examined is emphasized. Given the growth of information technology and artificially intelligent systems, Gardner (2019)Footnote 34 reasons it is imperative that libraries play a central role in teaching patrons how to appraise, interrogate, and analyze the roles algorithms play in structuring information retrieval and use and that greater work can be done in the area of understanding algorithm-assisted systems and evaluative bias.
The need for algorithmic literacy opens the discussion regarding the roles of legal information professionals in legal education, both in the academic and professional contexts. Wang (2019)Footnote 35 and Galloway (2017)Footnote 36 intertwine the legal information profession with the practice of law by stating that if the legal profession must adapt to technological changes, so must legal education. Binsfeld (2019)Footnote 37 discusses how information literacy opportunities, specifically for new barristers, extend beyond the academic sphere and exist within transitional and professional environments. For legal information users, Galloway states that of particular interest is developing skills in the critical and appropriate use of technologies in the context of the law, rather than mastering technology simply as a tool in isolation or as a gateway to discipline-specific knowledge and skills.
Law library staff and legal information professionals engage in teaching and learning initiatives on research technology, information retrieval, and information evaluation strategies aimed at legal information users in varying degrees. As such, understanding and evaluating the use of algorithms and artificially intelligent systems will likely be of continued importance to legal information practitioners, notably those without complementary computer science or data science backgrounds. This article postulates that legal information practitioners working with information users reliant on advanced technologies are obliged to foster the critical evaluation of information and increased algorithmic literacy.
2. ALGORITHMIC LITERACY IN THE CONTEXT OF INFORMATION, DIGITAL, AND COMPUTER LITERACY
In recognizing the need for greater algorithmic awareness, we encounter the question of whether existing information literacy, digital literacy, and computer literacy frameworks are able to ground algorithmic literacy. This article examines some theoretical grounds for algorithmic literacy within existing frameworks. Algorithmic literacy is initially explored as an extension of information, digital, and computer literacy, and existing theories related to enhancing these literacies are examined as theoretical frameworks for algorithmic literacy.
Hoechsmann and DeWaard (2015)Footnote 38 state that digital literacy requires a combination of technological capacities, intellectual competencies, and ethical/behavioral components and involves both participatory and social engagements and responsibilities. Bawden (2009)Footnote 39 examines various definitions of digital literacy from diverse sources, allowing the definition to encompass critical thinking, judgment, critical assessment, metacognition, and ethical considerations surrounding the use of digital sources. UNESCO's Global Framework of Reference on Digital Literacy Skills (2018)Footnote 40 defines digital literacy as ‘the ability to access, manage, understand, integrate, communicate, evaluate and create information safely and appropriately through digital technologies’ and is inclusive of competences that are variously referred to as computer literacy, ICT literacy, information literacy, and media literacy.
Building on pre-existing definitions of digital literacy, Koenig (2020)Footnote 41 defines algorithmic literacy as the ‘merging of technology and literacy research’. In her study, Koenig states that digital literacy requires a critical view from information consumers regarding what they are consuming and extends digital literacy scholarship to consider algorithmic literacy. Concurring with Kalantzis et al. (2010)Footnote 42, who state that this entails considering the broader affordances of the new digital communications technology for the production of different modes of meaning, Koenig asserts that the next phase in technological literacy is to incorporate the role of algorithms and algorithmically-run platforms. Wang (2019)Footnote 43, in referencing the then-President of the Law Council of Australia, posits that algorithmic literacy must constitute three elements:
1) a foundational understanding of the operations and language of predictive coding;
2) a conceptual understanding of computational analysis and machine ‘learning’; and
3) a principled understanding of the rules, assumptions, and existence of possible biases in programming.
In a CILIP report on the impact of artificial intelligence, machine learning, automation, and robotics used in the information profession, Cox (2021)Footnote 44 identifies algorithmic literacy as an extension of information literacy and places data literacy alongside this to facilitate critical interactions with intelligent systems. Similar to other computer literacies, algorithmic literacy focuses on the role that technology plays within a society. Speaking from an academic practitioner's purview, Wang (2019)Footnote 45 states that, in responding to changes prompted by the growth of artificially intelligent systems, legal educators must not lose sight of the fundamental mission of higher education, which consists of cultivating knowledge and analytical skills, that can be of value beyond the workplace, and encouraging wide-ranging intellectual enquiry.
As the connections between users, data, algorithms, and legal work outputs increase, information practitioners' concerns over the opaqueness of some of these intelligent systems become more valid. Information practitioners may be called to understand the effects of these connections to better instruct information users. Assisted by existing literature, some legal information professionals may postulate that algorithmic literacy in the legal field may be approached as an extension of information, digital, or computer literacy and that theoretical and conceptual frameworks applied to increase these literacies may also be extended towards increasing algorithmic literacy.
2(a). ACRL Framework for Information Literacy in Higher Education
Gardner (2019)Footnote 46 posits that theoretical guidance for algorithmic literacy may be rooted in the Association for College and Research Libraries (ACRL)Footnote 47 Framework for Information Literacy in Higher Education. The ACRL Framework provides six threshold concepts central to information literacy. Because the ACRL Framework may guide an information literacy program, the learning outcomes for learning initiatives on algorithmic awareness and algorithmic literacy may be aligned with these threshold concepts. Neither the critical analysis of algorithms nor concepts such as algorithmic bias are explicitly discussed within the ACRL Framework. However, evaluating information and reflecting on its creation are covered within the ACRL Framework, and content on understanding and assessing algorithms aimed at building algorithmic literacy may align with an information and digital literacy program grounded on this.
The concept authority is constructed as contextual may be interpreted to encompass the understanding of algorithm-based and artificially intelligent technologies so that legal information professionals may reflect upon their added value and credibility. Information professionals who develop information literate abilities develop awareness of the importance of assessing content with a skeptical stance, along with a self-awareness of their own biases. This self-awareness may extend to the critical assessment of legal research and information technologies used in the field. Similarly, the concept information creation as a process requires legal information professionals to seek out information products that indicate their underlying creation and computational processes, fostering technological transparency. The ACRL Framework's concepts also provide the grounds for requiring legal information professionals to see themselves as contributors to the development of legal information technology and to ongoing scholarship and debates. Meeting the framework's threshold concepts may require legal information professionals to take the lead in designing research strategies and inquiry processes; follow ethical and legal guidelines in gathering and using information; and assess the value of the outputs of various artificially intelligent technologies.
Despite its broad application, the ACRL Framework has received some criticism for possessing a gap and failing to encompass algorithmic literacy. Clark et al. (2017)Footnote 48, in their grant proposal for the Institute of Museum and Library Services of the same year, link algorithmic awareness to information and digital literacy and claim that, while the ACRL Framework considers how authority is constructed, instructional programs based on this framework have not caught up with how algorithms in intelligent software construct information user experiences.
2(b). SCONUL Seven Pillars of Information Literacy
The SCONUL (2011)Footnote 49 Seven Pillars of Information Literacy model integrates seven lenses defining core skills and competencies (abilities) and attitudes and behaviours (understanding) foundational to information literacy development. While these lenses are founded on an information literacy landscape, the same lenses may be used to guide the development of algorithmic literacy in the legal information services field within the context of information and digital literacy. The application of the SCONUL Seven Pillars model to the use of legal data and artificially intelligent systems in providing legal research and information services highlights metacognitive practices such as:
• identifying scopes and limitations of artificially intelligent systems;
• meeting the standards of conduct for academic and research integrity with information sourced and generated from artificially intelligent systems; and
• the user-centred and purposeful implementation of artificially intelligent systems within an organization.
When applied, the SCONUL Seven Pillars model provides grounds for requiring legal information practitioners to keep systematic records pertaining to the usage of and subscription to artificially intelligent systems and to inquire about how personal and organizational data inputs may be used to refine, train, or improve proprietary algorithms. Extending beyond the ACRL Framework which may be applied to focus on the use of artificially intelligent systems, the SCONUL Seven Pillars model includes lenses (such as manage and presentation) which require legal information professionals to account for the communication, technology recommendation, adoption, information handling, and privacy aspects of artificially intelligent systems within their organizations and wider professional communities. While each pillar may be extended to foundational cognitive abilities and understandings associated with algorithmic literacy, the depth and breadth of an individual's abilities and understandings shall largely be dependent on their technological competence and knowledge extending to the computer, information technology, and data science fields.
3. PROFESSIONAL COMPETENCY STANDARDS SUPPORTING ALGORITHMIC LITERACY
Professional association reports, standards, and competency principles may contribute to the need for an algorithmic literacy framework in the legal information field. These sometimes reflect the duty of technological competence required by various jurisdictional law societies for law students and members of the bar who are core users of legal information. The Legal Education and Training Group's (2013)Footnote 50 report on legal training in England and Wales cites new ways of learning and flexibility as two of the key messages, alongside ongoing competence, on account of technology continuing to reshape the way the law is taught and practiced. Associations such as the British and Irish Association of Law Libraries (BIALL) and the American Association of Law Libraries (AALL) have developed information literacy and legal research competency principles and standards for evaluating legal information literacy and research in practice. These professional principles and standards serve as touchstones for information literacy and research skills at the core of legal information practice in certain jurisdictions. Each of the standards and principles discussed may serve as foundational grounds for incorporating algorithmic literacy in the development and delivery of legal information services.
The BIALL Legal Information Literacy Statement (2012)Footnote 51 requires legal information practitioners to demonstrate the ability to undertake systematic and comprehensive legal research. This includes the ability to determine the most appropriate legal resources for an information query, taking into consideration all relevant and accessible technological sources. Legal information practitioners are also required to refresh and update their legal research knowledge as part of their career development. This requirement encompasses new media and technologies as well as changing structures of legal information and service delivery tools. In recent years, legal information practitioners such as Mishkin (2017) have called for a timely review of the BIALL Legal Information Literacy Statement in order to establish the extent of its use and assess its impact and relevance based on changing information practitioner needs. These changing needs may be interpreted as inclusive of the growing adoption of legal technology and intelligent systems and the resulting need for increased algorithmic literacy in the profession.
The AALL Competencies of Law Librarianship (2010)Footnote 52 include the determination and evaluation of the technology needs of users along with the provision of effective training to meet those needs. While the competencies do not explicitly address reflective metacognition surrounding the changing legal landscape on account of new technologies, the adherence to the Ethical Principles of the American Association of Law Libraries and the supporting of the shared values of librarianship along with the needs of the legal profession may be expounded to include meeting new legal technologies with new or modified digital or algorithmic literacies. The AALL Principles and Standards for Legal Research Competency (2020)Footnote 53 state that a successful legal researcher is one who critically evaluates information, distinguishes between ethical and unethical uses of information, and understands the legal issues associated with its discovery, use, and application. Competencies in this area include accurately communicating privacy, confidentiality, security, diligence, and ethical issues related to research and practice, including the benefits and risks associated with relevant technology. Baker (2018b)Footnote 54 notes how principle three, on the ethical evaluation of information, and principle five, on distinguishing between the ethical and unethical uses of information and understanding the legal issues associated with the discovery, use, and application of information, may relate directly to the competent and ethical use of algorithms in law. The AALL Principles also require legal information professionals to affirmatively undertake training on research platforms and practices as new iterations, tools, and technologies are introduced.
Depending on the extent of their interpretation, both the BIALL Legal Information Literacy Statement and the AALL Competencies of Law Librarianship, when read alongside the AALL Principles and Standards for Legal Research Competency, may be invoked to address the impacts of new legal research technology and maintain the balance between the optimisation of research and information services productivity using these tools and the cognition of the functional, social, and ethical aspects of intelligent systems.
Goltz and Dondoli (2019)Footnote 55 indicate that artificially intelligent systems are assistive tools meant to enhance the research and information retrieval and synthesis processes; they are not intended to be directive or definitive of the processes in themselves. It is recommended that the use of intelligent systems to conduct research must be done with the stewardship of legal practitioners or legal information professionals. For the time being, both the BIALL Legal Information Literacy Statement and the AALL Competencies of Law Librarianship and Principles and Standards for Legal Research Competency may be viewed as sufficiently encompassing the merging of increased technological competence with the metacognitive practices associated with advanced and evolving legal technologies. Nevertheless, legal information professionals may still benefit from the support of an algorithmic literacy framework as they engage in the design and delivery of effective and adaptive algorithmic literacy initiatives.
4. ADOPTIONS OF EXISTING COMPUTER LITERACY THEORY
Some authors explore and consider the adoption of computer literacy theories to address the need for algorithmic literacy, and this may be due to the scope and nature of the subject matter. Kaplan (2016)Footnote 56 defines artificial intelligence as analytical computing alongside the continuing advance of automation. Kennedy (2019)Footnote 57, writing from the perspective of research libraries, defines artificial intelligence as the theory and development of computer systems able to perform tasks normally requiring human intelligence. If existing digital literacy frameworks are to be considered as grounds for algorithmic literacy initiatives centred on artificially intelligent systems, consideration may also be given to existing computer literacy theories to support these initiatives.
Two authors jointly contribute to the following visual representation of concepts relevant to the building of algorithmic literacy. The first, Selber (2004)Footnote 58, states that information users need to be effective users of technology, informed questioners of technology, and reflective producers of technology. Selber approaches multiliteracies as part of a larger context and breaks down that context by engaging in research, reflection, and critical analysis. He encourages the practical and theoretical implications of rethinking technological literacy and re-imagines computer literacy by dividing it into three categories: functional literacy, critical literacy, and rhetorical literacy. These three distinct, yet overlapping, categories of literacy may be applied to an algorithmic literacy context (Figure 1 and Table 1). Selber (2004)Footnote 59 argues that, while functional literacy is important, information users must also understand how to critique the politics of technology and make informed rhetorical decisions about it. Adopting Selber's computer literacy categories as a guide for developing algorithmic literacy initiatives may assist legal information professionals and information users with synthesizing functional and critical literacy practices, engaging in metacognitive reflection and open dialogue regarding algorithmic literacy, and facilitating algorithmic literacy alongside technological proficiency in other artificial intelligence system users.
A Diagram of Selber's (2004) Three Categories of Computer Literacy, Adopted and Applied within the Context of Algorithmic Literacy.
An Adoption and Application of Selber's (2004) Three Categories of Computer Literacy within the Context of Algorithmic Literacy.
Koenig (2020)Footnote 60, the second author who discusses and applies Selber's work, states that algorithmic literacy practices must exist within a framework where the artificially intelligent technologies are better understood using various lenses. In her study, Koenig develops and applies the following three approaches to algorithmic awareness to assess the technological literacy practices within an undergraduate academic setting: basic approach, critical approach, and rhetorical approach (Figure 2 and Table 2). Although Koenig does not present the three approaches to algorithmic awareness as a levelled hierarchy, with the study participants and information users having presented strengths in various approaches, the adoption and modelling of these concepts in this article attempts a hierarchical illustration reflecting Bloom's (1956)Footnote 61 Cognitive Taxonomy and Krathwohl's (2002)Footnote 62 Revised Cognitive Taxonomy. This is on account of the seemingly progressive levels information users in this study were recorded to reflect as they transitioned from observations to perceptions, critical inputs, and metacognitive reflections illustrating the immediate personal and larger social impacts of artificially intelligent systems.
An Adoption and Diagram of Koenig's (2020) Three Levels of Algorithmic Awareness.
An Adoption of Koenig's (2020) Three Levels of Algorithmic Awareness.
The approaches proposed by Koenig (2020)Footnote 63 and Selber (2004)Footnote 64, when modified to ground algorithmic literacy initiatives, may be holistic and wide-ranging. The approaches offered by Koenig may be engaged contextually depending on the research, information, or digital literacy activities of legal information professionals. Both Selber and Koenig advocate seemingly fulsome literacies and levels of algorithmic awareness, with an expressed necessity towards the critical and rhetorical categories and levels. In a wider context, these approaches align with Galloway (2017)Footnote 65 who states that assuring these technologically focused literacies requires ensuring that users are competent in the professional, social, cultural, and personal aspects associated with and affected by artificially intelligent systems. Galloway suggests that these competencies represent the essential skills, knowledge, and attitudes for a society steeped in and mediated by technological advancement.
5. BUILDING ALGORITHMIC LITERACY WITHIN THE LEGAL INFORMATION PROFESSION
Wang (2019)Footnote 66 states that technologies in legal education present educators with three sets of considerations: (1) the adoption and adaptation of technologies to teaching and learning; (2) the study and research of disruptions and other impacts of technologies in society to assist in formulating legal responses to them; and (3) the preparation of future legal practitioners. Although these considerations are directed towards legal practitioners’ education, the same considerations and their interrelations may be adopted for the education of and initiatives delivered by legal information professionals.
In considering the work of Binsfeld (2019)Footnote 67 and Wang (2019)Footnote 68, it may be said that fulfilling the duty of technological competency is not about teaching information users to be technology experts. Instead, it focuses on authentic and reflective practice and teaching information users to understand the principles underlying the technologies within a holistic framework, while also equipping information users with various practical skills, including technological competency. By examining some of the issues associated with the use of artificially intelligent systems in legal research and information services, it is possible to understand the role that legal information practitioners may have towards building algorithmic literacy, algorithmic awareness, and explainable artificial intelligence. Normalizing the importance of characteristics such as trust, accountability, and explainability when examining intelligent systems and advanced technologies assists with bolstering the evaluative, transparency, and accessibility components of legal information services.
5(a). ‘Explainable Artificial Intelligence’ in Relation to Algorithmic Literacy
The responsible incorporation and adoption of artificially intelligent systems may be assisted by meeting the opacity of such systems with initiatives aimed at explainability and user inclusivity. Ridley (2019)Footnote 69, citing de Mul and van den Berg (2011)Footnote 70, acknowledges that the danger of reliance on artificially intelligent systems is not so much in the increased delegation of cognitive tasks to these systems, but in information professionals and information users distancing themselves from, and not knowing about, the nature, precise mechanisms, and repercussions of that delegation. Adding to this, Henry (2019)Footnote 71 states that instilling policies that require accountability, mandating not just access to the algorithms themselves and the processes followed when using the data but an accessible explanation of the extent to which the data used, is a key aspect of future governance and regulatory frameworks that foster ethically responsible behaviors in the use of intelligent systems.
Ridley (2019)Footnote 72 explores the concept of explainable artificial intelligence, which is broadly defined as a diverse set of strategies, techniques, and processes that render artificially intelligent systems interpretable and accountable. Trust and accountability are deemed the two pillars of explainable artificial intelligence, and Ridley emphasizes user-centred explainability as an essential requirement for a technology defined by its opacity. He outlines strategies, techniques, and processes utilizable by research libraries to shape the development, deployment, and use of artificially intelligent systems in a manner consistent with the values of scholarship and librarianship. In relation to this, Turner (2018)Footnote 73 cites Whitenack (2016)Footnote 74 who suggests three general capabilities required for transparency in artificially intelligent systems: data provenance; reproducibility; and data versioning.
Explainable artificial intelligence may possess various manifestations. Verheij (2020)Footnote 75 states that the integration of knowledge and data can be addressed by how programming rules and inferences are connected along with how they influence one another. Only then can explainability requirements and responsibilities of proprietary intelligent systems be properly addressed. In furthering the explainability of artificially intelligent systems, Turner (2018)Footnote 76 discusses the process of semantic association. This process aims to build upon explanation techniques to provide a narrative for individualized decisions in order to teach artificially intelligent systems semantic associations with their decision-making processes. Artificially intelligent systems may be taught to perform a primary cognitive task followed by a secondary task of associating computational or decision-making events with words. This approach, which is also referred to as AI rationalization, aims to generate explanations for autonomous system behaviour as if a human had performed the behaviour and is being asked to explain their actions. From a technical viewpoint, this process may involve natural language explanation in the form of labelled actions; however, there may likely exist incongruities with relating all computational and algorithmic operations with semantic associations given the complex nature of artificially intelligent systems and machine learning.
The concept of explainable artificial intelligence aligns with Wang's (2019)Footnote 77 proposition of developing basic understandings of operational principles of various technologies so as to avoid situations in which legal practitioners and legal information professionals must deal with issues that they do not fully understand due to rapid technological development or entrenching the mismatch between skills taught and skills needed in practice. Explainable artificial intelligence may also be assistive in grounding the development of computational thinking skills of legal information practitioners.
5(b). ‘Training the Trainers’: Algorithmic Literacy for Legal Information Professionals
Stevenson and Beatson (2020)Footnote 78 note that, if using artificial intelligence tools in practice affords practitioners significant advantages, or eventually becomes a prerequisite in legal practice, it would be advisable to provide training and other educational resources in the field prior to the imposition of new standards of professional conduct. One goal of algorithmic literacy in the legal information field is to assist legal information professionals and the information users they serve towards becoming both skillful practitioners and critical independent thinkers. The critical evaluation of legal technology, along with its applications and limitations, is something which legal information professionals both within and outside the academic sector may respond to.
Remarking on the ongoing relevance of developments in legal technology to legal information professionals, Wiggins (2019)Footnote 79 notes how many burgeoning technologies and artificial intelligent systems are underpinned by the principles that librarians specialize in, such as Boolean logic, hierarchies, information architecture, analyzing user interfaces, and facilitating user experiences. Smith (2016)Footnote 80 likewise notes how many artificial intelligence technologies reflect the skills embedded within the legal knowledge and information management communities. The principles of knowledge management along with the fundamental practices surrounding the collection, organisation, use, learning, and improvement of knowledge bases may be said to exist within the core of training intelligent systems to meet the needs of legal information users. Just as librarians’ roles may involve evaluating technologies and providing feedback as part of procurement processes, the traditional skills that underpin librarians’ professional training will require pivoting to help develop new algorithmic literacy initiatives.
In his literature review and empirical study, Mishkin (2017)Footnote 81 examines the question of how law librarians working in both academic and non-academic sectors are able to deliver legal research training most effectively. The research results in six recommendations, two of which may be extracted to form a dual-pronged approach for algorithmic literacy training in legal environments: first, a greater collaboration between academic and non-academic law librarians in designing learning objectives, curricula, and supporting learning activities geared towards legal technology and increasing algorithmic literacy; and second, training law librarians themselves on how to deliver legal technology, information, and algorithmic literacy training more effectively. Mishkin posits that technology use and training must provide a clear connection with the learning objectives, educational goals, and professional goals of legal practitioners. This supports the need for user-centric recommendations surrounding algorithmic literacy training.
Kitchin (2017)Footnote 82 similarly delves into critically thinking about and researching algorithms from the perspective of teaching and academic institutions. Kitchin formulates four key arguments related to the study and research of algorithms and discusses six methodological approaches for optimizing the study and research of algorithms. Kitchin's work shows several methodological approaches to studying algorithms that do not depend heavily on library and information professionals possessing pre-existing backgrounds in information technology or computer science. Teaching and learning initiatives grounded on approaches such as reviewing outcomes of reverse engineering tests, interviewing designers or conducting ethnography studies of coding teams, unpacking the socio-technical assemblages of algorithms, and examining how algorithms work in the world, along with elements such as human computer interaction and information seeking behaviour, all facilitate increasing computer literacy and algorithmic awareness in the use of artificially intelligent systems in legal information and research contexts. These humanist explorations of artificially intelligent systems align with Ridley (2019)Footnote 83, who states that some approaches of explainable artificial intelligence may involve studies on broad social and political policies surrounding regulation, legislation, or practice, without necessarily requiring highly technical expertise.
These methodological approaches may be used as supplementary guidance by legal information professionals or as theoretical foundations for initiatives to build on their own algorithmic awareness in addition to Koenig's (2020)Footnote 84 three approaches to algorithmic awareness and Selber's (2004)Footnote 85 computer literacy categories. Interventions grounded on of Koenig's rhetorical approach and Selber's rhetorical literacy may be supplemented by the work of other researchers who examine the greater social impacts of autonomous artificial intelligence, such as Liu's (2018)Footnote 86 three power structure levels of artificial intelligence.
With respect to their applications in the legal field, Koenig's Three Levels of Algorithmic Awareness and Selber's Three Categories of Computer Literacy may also be explored in relation to other approaches to learning and incorporating technology, such as Jones’ (2016)Footnote 87 expansive learning approach to legal research. This approach focuses on the activity-centered context of both the individual and social aspects of human behavior, the design-oriented nature of human problem-solving, the role of tacit knowledge and unwritten rules in daily activity and practice, and a cultural-historical approach to learning and development. These elements of the expansive learning approach may all hold relevance to the exploration of algorithmic literacy and the growing role of artificially intelligent systems in legal research and information services.
5(c). Legal Research Instruction: Promoting Algorithmic Literacy in the Legal Research Process
5(c). (i). Algorithmic Literacy Opportunities in Academic and Non-Academic Settings
Legal research instruction undertaken by legal information professionals may largely be directed towards new barristers and solicitors, law students, new users of law libraries, and members of the public. Ridley (2019)Footnote 88 discusses how libraries have been active proponents of enhancing literacy in various formats, from traditional reading and writing to more recent digital literacy involving evolving technological tools. Both Kennedy (2019)Footnote 89 and Janoski-Haehlen (2019)Footnote 90 discuss the importance of introducing legal technologies into the law school curriculum and initiatives spearheaded by research libraries at a time when technological innovations are occurring at a rapid pace. With much underway in the field of artificial intelligence, there exists a need for research libraries to act, beginning with the clarification of artificial intelligence scopes, limitations, ethics, policies, principles, and practices. While both Ridley and Janoski-Haehlen write from academic perspectives, Mishkin (2017)Footnote 91 notes the greater co-ordination and cooperation required between the academic and non-academic legal sectors to ensure that law students, as information users, are being instilled with the legal research skills they need to practice with algorithmic competency. The initiatives discussed are largely from academic instructional contexts but may be modified to organizational or professional settings.
Galloway (2017)Footnote 92 suggests that for a whole curriculum or “immersive approach” to digital literacy in the legal field, teaching and learning initiatives in digital literacy, and by extension algorithmic literacy, may be considered in the broader context of the law. Although Wang (2019)Footnote 93 questions the idealism and practicality of this approach, along with the presumptions on the digital competence and algorithmic literacy of legal educators required to achieve it, Wang does acknowledge that literacy initiatives must include an understanding of technological development alongside the underlying principles of technology, so that skills and understandings from the latter may reinforce learning of the former.
On data literacy, Henry (2019)Footnote 94 states that engaging librarians in computer and data science courses may assist with teaching learners about important concepts such as validating information, understanding data provenance, finding appropriate information resources, vetting data to use in research and experiments, and other issues related to privacy and ethical uses of data. Binsfeld (2019)Footnote 95 further asserts that legal research training for incoming law students and new associates within the organization or law firm setting is a prime opportunity for legal information professionals to assist in bridging the gap between the academic or educational environment and professional practice. While algorithmic literacy may be seen as a subset of digital literacy or computational thinking, Ridley (2019)Footnote 96 posits that algorithmic literacy possesses unique characteristics and applications that warrant specific attention. Just as information literacy provides users with skills and perspectives to assess resources, algorithmic literacy is a strategy rooted in explainability which allows users to navigate and utilize algorithmic tools and services.
Wang (2019)Footnote 97 states that it is integral that information users not only receive intellectual cultivation but also acquire sophisticated skills that are transferrable and adaptable in the age of technology. Emphasizing the elements and understandings, Wang highlights how traditional skills such as critical and analytical thinking and problem solving, and values of legal practice such as ethical propriety and social responsibility, may still underlie algorithmic literacy, along with newer skills, such as emotional intelligence, digital literacy, data literacy, teamwork, and collaboration. Clark et al. (2017)Footnote 98, in their grant proposal for the Institute of Museum and Library Services, provide the broad frameworks for a rubric for algorithmic awareness and building new competencies surrounding the use of artificially intelligent systems. One key objective discussed was to find and encourage transparency for the invisible logic embedded in users’ software interactions. This may likewise be extended to the data used in machine learning initiatives. Success in this objective would entail the library community finding new teaching methods to make this logic visible for both information professionals and information users.
5(c) (ii). Legal Information Professionals and Legal Research Training
Burchfield (2021)Footnote 99 underscores the role of legal information professionals in teaching students to evaluate the accuracy and authority of the legal information they use, and Margolis and Murray (2012)Footnote 100 note how academic legal research programs are pressed to adapt to both technological advancement and the developments being made in electronic legal research tools. As legal writing programs made the pedagogical shift to the process-oriented or functional approach, so too did legal research instruction. Legal research instructors continue to balance teaching bibliographic research with computer-assisted legal research, and adding to this is the arrival of artificial intelligence. This may present an opportunity for legal information practitioners to demonstrate value by integrating algorithmic literacy into advanced legal research training and to prepare future legal practitioners for the increasing number of technological tools in this area.
Mishkin (2017)Footnote 101 highlights how law librarians are required to rapidly adapt to new technologies and how training has become an increasingly important role for law librarians. Mishkin also states that that the quality of legal research training will be improved if law librarians themselves are provided with training on how to deliver it most effectively. Henry (2019)Footnote 102 states that teaching library users about good research practices early on - for example, documenting results, questioning data sources, and archiving software and data so that the expected system behavior can be replicated by others - is useful information that academic librarians may already be teaching as part of their curriculum.
In amalgamating traditional legal training with computer literacy, Tucker and Chapman (2019)Footnote 103 discuss credit-bearing academic initiatives within law schools aimed at increasing student cognition surrounding legal analytics. These curriculum-embedded courses aim to make computational thinking and working with legal analytics software approachable for students without a strong STEM education while simultaneously challenging students coming from a STEM background. Tucker and Chapman posit that with an emphasis on computational thinking and legal analytics, students are able to practice the legal tradition of problem solving with an expanded set of tools permitting greater innovation.
Examples of replicable experimentation which may involve students and existing legal research tools include Mart's (2017)Footnote 104 and Callister's (2020)Footnote 105 study on algorithms and natural language processing capabilities. These illustrate instances wherein legal information professionals working in an academic context sought to enhance algorithmic literacy and natural language processing search techniques by experimenting with common legal research databases available in law libraries. Callister's techniques for teaching search inquiry processes were prompted by empirical investigations on the opacity of search engines, the differences in legal database search results, the effects of varying search queries, and the recognition of some biases present in search algorithms. Callister's study, in part, demonstrates Kitchin's (2017)Footnote 106 reverse engineering methodological approach, which takes into account the proprietary or ‘black box’ nature of some artificially intelligent systems. Callister concludes that the adeptness of natural language processing is uneven among various legal information vendors and that what is received in search results from such systems varies widely depending on a host of unknown variables. This supports Cox's (2021)Footnote 107 six recommendations speaking to algorithmic literacy and the critical evaluation of research results. Both the Mart and Callister studies highlight varying algorithmic capabilities along with the importance of using a critical comparative approach when using multiple search tools to achieve more comprehensive results. Users ought to be wary of how natural language processing may introduce some uncertainty to legal research databases and resulting legal research outputs and how complacent reliance on either bias or inadequate datasets may impact information seeking behaviours.
Bakke (2020)Footnote 108 highlights algorithmic literacy as a component of information literacy instruction and draws attention to how algorithmic literacy entails requiring information users to evaluate individual sources and consider the processes associated with those sources. One advantage of basing algorithmic literacy initiatives on Kitchin's (2017)Footnote 109 reverse engineering methodological approach is that this approach may be engaged in by those with various levels of degrees of functional literacy. The comparative examination of the same input data in different output systems or legal research databases may not require extensive technical expertise as learners would still be able to record and observe differences in outputs generated by different systems as well as reflect on the quality of these outputs in relation to the subject or topic of research.
In addition to the planning and delivery of legal research and information training sessions directed towards building algorithmic awareness, algorithmic literacy, and explainable artificial intelligence, Woellhaf (2017)Footnote 110 discusses the importance of continuously promoting learning initiatives in manners that resonate with patrons. This is particularly true when information users such as lawyers and law students may be short on time or utilitarian in their approach to the use of legal information. When engaging with ‘millennial’ legal information users, Vickery (2018)Footnote 111 highlights how encouraging participation in external information, legal clinics, analytics labs, and research workshops held outside of synchronous teaching and learning environments may prompt continued cognizance and examination of artificially intelligent systems as a form of exercising information, computer, and data literacy.
5(c) (iii). Lesson Planning, Learning Outcomes, and Reflection in Academic Contexts: Applications of Existing Frameworks and Learning Practices
Burchfield (2021)Footnote 112 notes that teaching legal technology shall require continuous evaluation and adaptation of instructional objectives and methods as various technological tools, and the rules surrounding their use, evolve. Azyndar (2015–16)Footnote 113, in citing Talley (2014)Footnote 114, advocates for the use of the backward design framework in legal research pedagogy. The backward design framework entails three sequential steps:
(1) identifying the desired outcome in terms of academic standards;
(2) determining how to measure student achievement of that outcome; and
(3) crafting instruction and activities in support of academic standards identified in the first step.
Given the seemingly general (conceptual) to specific (technical) direction of building understanding surrounding the use, evaluation, ethics, and limitations of advanced legal technologies, the application of this framework to algorithmic literacy initiatives may be worth considering. In providing examples of how academic law libraries have engaged in curricular innovations which may have been aligned with desired outcomes and academic standards, Burchfield cites law schools which have introduced courses with summative assessments requiring the creation of new technologies (or apps) to solve legal problems. Burchfield also cites schools which have introduced courses specific to the intersection of law and technology and the introduction and evaluation of new technologies and their applications to legal practice.
Gardner (2019)Footnote 115 provides some reflective notes on teaching algorithmic awareness for a credit-bearing information literacy course geared towards undergraduates. The course's significant component on algorithmic bias provides some aspects which may be adopted in a public-serving, private, or academic law library context. The course developed by Gardner focused on frameworks for evaluating other information sources, analyzing personal information seeking and use behaviors, privacy, and information privilege. The course's learning outcomes were grounded by the ACRL Framework for Information Literacy in Higher Education, and the course's aim was to teach students from all majors to appraise, interrogate, and analyze the roles algorithms play in structuring information seeking and use. Gardner's use of the ACRL Framework in teaching algorithmic bias in a university credit-bearing course illustrates how the ACRL Framework may be extended to ground algorithmic literacy initiatives in law schools and how introductory algorithmic literacy learning outcomes may still be grounded on these long-standing framework concepts.
In her study, Koenig (2020)Footnote 116 concludes that a reflective journaling practice over a number of weeks may assist in building and evolving the basic, critical, and rhetorical levels of algorithmic awareness. Koenig uses the methodology of journaling to encourage self-exploration and reflection on technological literacies and algorithmic engagement. Koenig states that the first step to defining algorithmic literacy practices is to understand (1) how information users engage with algorithmic platforms; and (2) what awareness they have of their own practices. The study findings illustrate that the reflective journaling intervention prompted a number of information users to engage in second-level cognitive thinking building upon critical and rhetorical literacy. This metacognitive examination aligns with a deeper approach to learning characterized by intrinsic self-motivation, constructive integration, and the incorporation of personal experiences.
Koenig's (2020)Footnote 117 classifications of algorithmic literacy and algorithmic awareness may be extended to law students, law practitioners, and legal information professionals. While practical challenges exist surrounding the implementation of a reflective journaling practice in the legal profession, this practice may be worth attempting to examine legal technology and artificially intelligent systems in academic environments via credit or non-credit bearing courses. The enduring value of Koenig's study is its examination of how algorithmic literacy prompts users to recognize and act upon their own agency when interacting with artificially intelligent systems. This places into practice the observations Margolis and Murray (2012)Footnote 118 make several years prior to the rapid increase of artificial intelligence in law. They state that by reframing the goal of legal research instruction to that of increasing legal information literacy, academic institutions would be able to leverage the research skills students possess and instill in students the skills that are ‘transferable to the legal research tools of tomorrow’. One wonders whether rapid technological advancements have now created a need for legal algorithmic literacy and legal data literacy in addition to legal information literacy.
With respect to metacognitive reflections as learning interventions, Zahid (2017)Footnote 119 demonstrates the importance of engaging in active discussions with information users and commending thought-provoking metacognitive reflections. These may facilitate intellectual bonds that encourage information users to engage with learning materials and their own independent algorithmic literacy initiatives. These interactions support humanist approaches to learning, which, as Tangney (2014)Footnote 120 states, center on personal growth, consciousness raising, and empowerment. Metacognitive reflections also facilitate the experiential aspect of adult leaning, as discussed by Gerdy (2001)Footnote 121, which is enhanced by a mutual exploration and inquiry between information professionals and information users. These reflections may be encouraged in informal, conversational, and sporadic contexts, as permitted by legal information professional and information user interactions. As legal information professionals come to terms with technology-driven changes in the legal practice, there is an increasing need to discuss not only specific legal research and information tools, but also a deeper understanding of intelligent systems such that skills can fluidly be transferred as the technology continues to evolve.
5(d). Vendor Liaising: Assessing and Evaluating Artificially Intelligent Tools for the Law Library
Legal information professionals may be required to interact with legal vendors, developers, and publishers regarding the evaluation, assessment, and adoption of artificially intelligent systems and the integration of these new legal technologies into existing processes. Grossman and Cormack (2021)Footnote 122 note that, at present, practitioners take it upon themselves to identify reasonable tools, procedures, and validation protocols based on best available information augmented by their own vetting and evaluation efforts. This is commonly facilitated by dialogue and open exchanges with legal vendors and publishers. Grossman and Morrison (2019)Footnote 123 provide practical examples of critical inquiry for the evaluation of advanced legal technologies which may guide legal information professionals as they investigate the capabilities of intelligent systems, the use of assisted or unassisted learning or training techniques, the scope and trustworthiness of data, the underlying algorithms and assumptions of the systems, and the tools for the interpretation and explanation of system activities.
Wiggins (2019)Footnote 124 posits that providing development feedback for new technologies will likely require collaboration with a greater diversity of vendors in the legal sector. Developers may require procedural insight, such as how a lawyer or information user is likely to carry out their research and meet their own information needs, and this facilitates a space in which legal information professionals may be well placed to assist. In writing from the perspective of a legal publisher and vendor, Becker (2019)Footnote 125 cites the dual benefits associated with collaboration and states that legal information professionals may consider engaging with and providing inputs to vendors with respect to developing user-centred interfaces; urging transparency regarding data-handling practices; requiring honest assessments of content and functionality trade-offs in developing data solutions; and advocating for product features that focus on everyday use cases. Additionally, legal information professional must continue to remain cognizant of issues surrounding the capture and subsequent use of user-generated inputs and data and how these may be utilized for machine learning or training processes.
Ridley's (2019)Footnote 126 concept of explainable artificial intelligence may serve as a theoretical foundation and guide for legal information professionals as they liaise with vendors and evaluate the understood capabilities and potential uses of artificially intelligent systems for their organizations. A modification of Ridley's four elements or question themes contributing to explainable artificial intelligence may be found in Figure 3. Similar guides may facilitate collaboration and dialogue between legal information professionals and vendors during inquiry processes involving the adoption, application, and integration of artificially intelligent systems within libraries and other organizations. The modification of Ridley's elements may be used conversely as an explanation framework for vendors in crafting responses to the queries of legal information professionals.
An Application of Ridley's (2019) Four Elements (Question Themes) Contributing to Explainable Artificial Intelligence: A Vendor Interview Guide for Legal Information Professionals.
Ridley's (2019)Footnote 127 elements or questions contributing to explainable artificial intelligence may assist with addressing the challenges cited by Kitchin (2017)Footnote 128 in his arguments and methodological approaches promoting the critical examination of algorithms. The use of similar theoretical guides and applied models may assist librarians and legal information practitioners possessing limited technical, data science, and computer programming knowledge with the development of an organization-specific explainable artificial intelligence guideline, toolkit or rubric for use in acquisition processes and in interacting with artificially intelligent system vendors and developers. The extent to which each vendor representative may respond to each of the questions may prompt areas of further discussion or investigation in alignment with the inquiry and competency principles associated with the legal information profession.
5(e). Serving Library Patrons: Communicating and Explaining AI-assisted Legal Reference and Research Services
Swansburg (2021)Footnote 129 notes how the line between artificial intelligence acting as an assistant and performing what is arguably legal work is not always clear, and how legal information professionals may effectively and ethically assist in supervising the work conducted by intelligent systems remains an open challenge. With respect to professional competencies and legal services, Wang (2019)Footnote 130 poses the following question which may be extended to legal information professionals: Have practitioners discharged their duty of competence if they have rendered their services with the assistance of artificially intelligent technologies without a comprehensive understanding of how such technologies work?
Swansburg (2021)Footnote 131 further notes the dual nature of competency, noting that many professional associations may require legal practitioners to make use of available technologies as a matter of competence, ethics, and professionalism. Relating to the goals of algorithmic literacy, one aspect warranting examination is what a ‘basic’ (or presumably foundational) understanding entails. Turner (2018)Footnote 132 notes four characteristics which serve as key evaluative components and requirements of artificially intelligent systems. These are identification, explanation, bias, and limitations of use (Table 3). These evaluative components may serve as conceptual guidelines or informational speaking points for brief teaching and learning opportunities with legal information users.
Source: Gardner (2019).
Internalizing Turner's characteristics in addition to cultivating algorithmic literacy using Koenig's (2020)Footnote 133 Three Levels of Algorithmic Awareness and Selber's (2004)Footnote 134 Three Categories of Computer Literacy applied within an algorithmic literacy context may support legal information professionals in communicating and explaining key evaluative components of artificially intelligent systems and repercussions of their use. Legal information professionals may use these models to expound on their roles in legal research, reference, and information services and build a basic understanding of these systems from the lens of current and potential areas for increased human involvement.
An Illustration of Turner's (2018) Laws and Key Evaluative Components: Building a Conceptual Understanding of Algorithmic Literacy.
Given the nature of information users such as lawyers and law students, many of whom are under cognitive and time pressures, Yu and Ali (2019)Footnote 135 provide a condensed list of three comparative best practices for legal researchers and users of artificially intelligent systems which may be implemented via informal, experimental interactions or as algorithmic literacy activity suggestions. These are: (1) using multiple artificially intelligent systems; (2) trying different inputs with single or multiple artificially intelligent systems; and (3) active human monitoring and engagement. These three comparative best practices may be perceived as informal applications of three of Kitchin's (2017)Footnote 136 six methodological approaches for thinking critically about algorithms, namely: reverse engineering; unpacking the full socio-technical assemblage of algorithms; and examining how algorithms do work in the world (or do work in context). These three comparative best practices are implementable in time-limited user interactions and executing either of the best practices may help establish interest in future algorithmic literacy engagements between legal information professionals and information users.
When working with information users that come and go in quick succession, immersive, interactive, and conducive environments must be made to provide opportunities for education. Koenig (2020)Footnote 137 states that human beings are powerful pattern-recognizers and these patterns are realized through experiential learning and repeated use of digital environments. Yu and Ali’s (2019)Footnote 138 three comparative best practices may be adopted as guidelines for exchange wherein information users are invited to have informal conversations regarding the observation and analysis of outcomes. Gramming et al. (2019)Footnote 139 provide contextual guidance by introducing the concept of nano-learning in law firms and enumerating ten methodological takeaways for those seeking to create a suite of learning opportunities in private firm settings.
As digital environments featuring artificially intelligent systems continue to evolve to meet the needs of information users, legal information professionals may take advantage of opportunities to prompt metacognitive reflection as these changes happen. Opportunities to engage in introductory discussions involving artificially intelligent systems or applications like natural language processing or machine learning with library patrons or information users may exist in informal, conversational contexts as with opportunities for determining further interest in more formal and planned learning initiatives.
4. CONCLUSION
Concluding thoughts for this literature review may be drawn from a number of authors examined. Aman (2019)Footnote 140 asserts that one way in which legal information professionals may continue to add value to their organisations is to humanise the technology transforming the legal information sector. As Coleman (2017)Footnote 141 states, law librarians and legal information professionals may serve as co-creators and facilitators of intelligent information systems. In an ideal context, these systems and their developers would respect and evaluate the various sources of data, recognizing any underlying biases or flaws; encourage human engagement via experimentation and critical inquiry; foster imagination and the possibilities associated with socially-responsible and transparent technological advancements; and support human learning and knowledge creation. Araszkiewicz and Rodríguez-Doncel (2019)Footnote 142 acknowledge that the influence and prevalence of advanced technologies in the legal information profession generates interests from various stakeholders with respect to the responsibility, explainability, trustworthiness, and transparency associated with these emerging intelligent systems. Ridley (2019)Footnote 143 further surmises that libraries engaged in research have a unique and important opportunity to shape the development, deployment, and use of intelligent systems in a manner consistent with the values of scholarship and librarianship. Stevenson and Beatson (2021)Footnote 144 posit that although the future of algorithms and artificially intelligent tools in the legal information field raises fundamental uncertainties, those who build literacies surrounding the use of such tools will likely be better positioned to react and adapt. These interests support the continued research into the development of teaching and learning opportunities aimed at increasing legal information, data and algorithmic literacy for both legal information professionals and the information users served.
As algorithms and artificially intelligent systems continue to influence and direct the interactions between users and legal technologies, it becomes imperative that the algorithmic literacy of legal information professionals and information users must shift towards more critical and rhetorical levels of awareness and understanding alongside foundational and technical competence. The literature supports that some initiatives and interventions involving metacognitive components may be grounded in existing digital and computer literacy theories, while other sources posit that algorithmic literacy and the concept of explainable artificial intelligence may serve as standalone enterprises, requiring new algorithmic literacy frameworks which take into account critical awareness and increased user engagement alongside the evolving, opaque, proprietary, and, at times, inaccessible nature of artificial intelligence. Regardless of whether learning initiatives are grounded in existing information, digital, and computer literacy frameworks, or if new algorithmic literacy frameworks and models are developed to account for an amalgamation of literacies leading to ethical, responsible, and algorithmically-aware technology users, both existing literacy frameworks and new frameworks applied towards artificially intelligent systems and new legal technologies must promote higher orders of thinking. Legal information professionals may be required to shift their roles towards higher value judgments and reasoning, and initiatives supporting this must be inclusive of both academic and professional contexts to facilitate continued learning and deeper levels of cognition.
