HAVE YOU HEARD ABOUT CHATGPT?

When AI exploded into the public consciousness in late 2022 with the launch of ChatGPT, it initially felt like just another tool to explore – something worth a quick look at, but nothing too seismic. However, it quickly became apparent that this was not just another tool; this was the genie out of the bottle, and our world has never been the same since.

Over the past few years, we’ve seen an influx of tools flooding the market, all purporting to transform the way we work. For knowledge teams, the challenge has been keeping up with these changes, understanding what’s new and determining where to focus our time, effort, and ultimately money.

The legal knowledge sector has never been static, but the pace of change has been unparalleled in recent years. Take legal research as an example. Historically, this domain was dominated by a few key players – Lexis, Thomson Reuters and vLex – who refined their offerings incrementally with expanded content, additional features or even new interfaces. But then these providers introduced new AI-enhanced platforms: Lexis released Lexis+ AI, Thomson Reuters developed Westlaw Edge with CoCounsel and Search & Summarize for Practical Law, and vLex launched Vincent AI.

These resources promise to revolutionise the legal research process, but they also take time to assess, to understand their relative strengths and weaknesses. But just when we feel we’ve understood these innovations, the market shifts again. Lexis launches Protégé, Thomson Reuters consolidates its products under CoCounsel, and vLex forms strategic partnerships to expand its knowledge content. And these are just the established players within one area of legal practice.

The question, then, becomes: how do we stay on top of everything?

UNDERSTANDING AI: WHAT IT IS AND WHERE IT EXCELS

Before diving into the process of assessing AI-powered tools, it’s worth pausing to consider what AI actually is – and crucially, what it is not. The Oxford English Dictionary defines AI as: “The capacity of computers or other machines to exhibit or simulate intelligent behaviour … software used to perform tasks or produce output previously thought to require human intelligence, esp. by using machine learning to extrapolate from large collections of data.”Footnote ¹

This definition highlights two key aspects of AI. First, AI does not ‘think’ as humans do; it simulates intelligent behaviour by predicting outcomes, based on patterns in the data it has been trained on. Second, its primary strength lies in its ability to process vast and complex quantities of information, enabling it to perform tasks with speed and accuracy.

AI is therefore not infallible. It doesn’t know everything, and it can’t do everything. It is a tool, an exceptionally powerful one, but a tool nonetheless. Its value lies in the different forms that it comes in and its ability to address specific challenges. The most commonly referenced use of AI is generation of content, but AI is much more than that. AI can detect anomalies in large datasets, summarise extensive volumes of text, or categorise and extract information from a host of documents. The range of potential use cases is extensive and impacts every element of our businesses, as the graphic below displays:Footnote ²

The challenge for all knowledge teams when assessing and adopting AI tools then is to recognise that AI is not a ‘silver bullet’ but a tailored solution to our existing business problems. Teams must identify the pain points, inefficiencies and opportunities within their business processes, in order to consider how AI-powered resources can provide a targeted solution to those specific challenges.

BUILDING THE FOUNDATIONS FOR AI SUCCESS

To fully maximise AI’s potential, businesses need a framework that provides end users with the confidence and skills to harness the full potential of AI-powered tools. Three pillars underpin successful adoption: having a clear AI policy, fostering a culture of practice, and investing in comprehensive training.

ESTABLISH AN AI POLICY

The foundation of any AI resource rollout should be a clear and visible framework governing its use. Yet, almost half of businesses in the professional services sector still lack a policy governing their use of AI-backed technology.Footnote ³

A robust AI policy clearly sets out limitations, such as prohibiting the input of client-confidential information into open-market tools. It also contextualises AI as a natural extension of existing technologies, demystifying it and fostering confidence in its use. Without a framework, knowledge teams will struggle to secure effective engagement when testing emerging technologies, due to a lack of clarity regarding the technology’s place within the business.

PROMPTING MAKES PERFECT

Even without a formal policy, organisations can build familiarity with AI by encouraging employees to use tools like ChatGPT in their personal lives. Everyday tasks such as drafting shopping lists or personalised gym plans may seem trivial but provide valuable insights into how different prompts influence results.

Understanding the impact of different prompts and the ways they can maximise output is vital to the success of using AI. In 2024, Addleshaw Goddard’s Innovation & Legal Technology team published The RAG Report: Large Language Models in Legal Due Diligence, which included an exploration of the effectiveness of different prompts for this specific use case.Footnote ⁴ Whilst the findings focused on a niche area, they have wider implications when thinking about AI as a whole. For example, the study found that in this instance asking follow-up prompts improved the accuracy of the response by an average of 9.2%, and that using emotive prompting – for example encouraging AI to do its best or applying pressure or an incentive — also improved the quality of response.

These findings underscore the importance of treating prompt crafting as a skill in its own right. Much like the precision required in Boolean searches, small adjustments to prompts can lead to vastly different outcomes. Encouraging employees to experiment with prompts and share their findings is crucial for developing best practices and improving the overall quality of AI outputs.

THE IMPORTANCE OF TRAINING

Due to the importance of understanding how different prompts can impact the quality of a response when using AI, training becomes critical to successful implementation. Whilst considering training before resource identification, assessment and implementation, may seem an illogical approach, if people within the business who will be testing the products in trials don’t have the skills to do so effectively, the quality of the feedback from the tests will be limited. If users are not equipped with the skills to use the resources effectively, they will try and force them to do tasks they’re not designed for and then risk becoming frustrated at the failed results.

Despite its importance, training remains under-prioritised. In 2025 Lexis reported that only 15% of businesses provide advanced AI training, while 28% offer no training at all.Footnote ⁵ Similarly, Thomson Reuters found that only 20% of staff at professional services firms had had AI based training.Footnote ⁶ Running centralised training can help contextualise AI developments and skills, by providing it in a context that is relatable to the day-to-day activities of individuals within the business.

This is something that falls within the Knowledge Management (KM) bracket – as an industry, KM teams have years of experience and appreciation for the importance of language in different searches, as well as critical skills in sharing knowledge and best practice across teams in our businesses. By centralising training, we can empower employees to expand their skillset and support them to fully engage in the process of assessing and implementing AI-powered solutions to their day-to-day business challenges.

UTILISING POPULARITY

The urgency of developing AI skillsets is underscored by one simple, salient fact. People want to use AI. Historically, the legal industry has been resistant to new technology and hesitant to disrupt established workflows or invest time in learning unfamiliar tools. However, AI has brought with it an unprecedented enthusiasm for integrating technology into legal practice.

Recent data underscores this change. A 2025 Law360 survey found that 62% of lawyers were already using ChatGPT in some aspect of their work.Footnote ⁷ While there are valid concerns about the use of generalised platforms such as ChatGPT over more secure, specialist tools, the statistic is telling: the appetite for AI is undeniable. This sentiment is echoed by clients, with 71% of in-house counsel expecting their law firms to use AI.Footnote ⁸

The message is simple, especially for those in private practice – clients want us to use AI. Of course they want us to use it responsibly, in the right way and at the right time, but the desire to maximise these resources is clear.

KNOW YOUR AI: A FRAMEWORK FOR NAVIGATING AI ADOPTION

Given this increasing demand for AI powered technology, and an increasing array of tools on the market, the challenge for knowledge teams is how to identify, test and choose the resource that’s right for your individual business needs. To assist in navigating this challenge, I propose a four-step solution – KNOW your AI:

• • Keep on top of evolution

• • Navigate beyond the hype

• • Optimise stakeholder engagement

• • Weight your scoring criteria

Keep on Top of Evolution

The first challenge when choosing AI-powered tools goes back to the beginning of this article – how do you keep up with an ever-changing landscape? Businesses have to adopt proactive strategies to monitor developments and identify tools that align with their needs. Start by leveraging your professional network – peers in the industry can offer valuable insights into emerging trends and tools, but remember that each firm’s requirements and AI maturity are unique, so avoid using them as a template for your own business.

Similarly, make use of resources you already have at your disposal – press aggregators like Manzama or Vable allow you to track updates from vendors and key publications such as Artificial Lawyer and Legal Tech Monitor. Consider subscribing to third-party resources like Jinfo or Outsell, which summarise industry events and product launches, saving you time while keeping you informed.

When you’ve identified the areas to focus on, find the detailed reports that have done the initial analysis for you. Whilst the analysis done in these reports shouldn’t be treated as absolute (particularly after the issues with the 2024 Stanford report) other comparisons still offer a useful initial insight into the relative strengths and weaknesses of resources.Footnote ⁹ Footnote ¹⁰

Navigate Beyond the Hype

The change in the art of the possible over the last few years has made it easy to feel overwhelmed when assessing tools – to become fixated on how vendors are pushing technological boundaries and lose sight of our objective. AI is not a magic tool that can solve an indefinite number of issues and therefore our aim is not dissimilar to purchasing more conventional resources: to solve a specific pain point or create an opportunity.

Start by identifying the problems – processes that are overly time-consuming or resource-intensive – and work backwards to identify the specific issues within those wider problems and then find tools to address these issues. By maintaining a clear focus on the wider problems and your objectives, you can establish parameters for evaluating tools and avoid being distracted by capabilities that are not directly relevant to your needs.

Optimise Stakeholder Engagement

The buzz surrounding AI often generates significant interest across firms, making stakeholder engagement both easier and more complex. Many people will want to participate in trials, but selecting the right people to test the right tools (and managing their involvement effectively) can be tricky – especially when vendors are placing stricter limits on trial participant numbers.

When considering trial participants, carefully select those who you are confident will commit to testing tools thoroughly, and involve senior stakeholders to nominate their team members, creating accountability and ensuring junior participants remain engaged. It’s also critical to consider specialist knowledge when putting together a trialist group, being clear with everyone within the trial of your expectations and how they can contribute to that knowledge.

For example, when evaluating a legal research AI tool, consider splitting the trial into three distinct groups:

• • Knowledge: Made up of Research/Library professionals and Knowledge Lawyers/Professional Support Lawyers whose role is to assess the end product for accuracy and reliability.

• • Innovation: Made up of IT/Technology professionals whose role is to assess the effectiveness of the technology including integration with existing systems and how liable it is to hallucinate.

• • End users: Those who will be using the system on a day-to-day basis whose role is to assess how the product works in practice for them.

The last user group is critical, yet it’s the easiest to lose sight of. We’ve seen plenty of examples of resources that are impressive technologically but fall short of the rigorous content standards that we require for legal research, but we’ve also seen numerous examples of tools where the technology does not stand out, but the end solution offered to the end user is useful enough in practice to set it apart as the resource to invest in.

Weight Your Scoring Criteria

Once the issues have been identified, the resources have been found and the trial participants have been selected, the final challenge is how to make meaningful comparisons between tools. Trials often run at different times, so it is essential to establish a consistent and structured approach to evaluation and maintaining detailed records.

Where possible, use the same group of participants and evaluation criteria across similar trials but tailor the length of trials to the complexity of the tool – longer trials may be necessary for in-depth tools, while shorter trials can create engagement for simpler solutions. Try to vary the way you gather feedback, using a mix of surveys, one-to-one chats and in-depth group discussions.

Using a centralised system to track the key metrics related to the purpose, unique features, and performance of each tool is crucial to make comparisons easier – something as simple as a spreadsheet will allow you to base your final assessment on the key parameters identified at the beginning of the assessment.

PRACTICAL STEPS AHEAD OF INVESTMENT

Whilst the above KNOW your AI framework sets a template for how to identify, test and assess AI tools, the rapid development of these resources and the costs associated with them mean that purchasing these technologies is not currently an option for all businesses. However, the eventual implementation of these technologies is inevitable. So, whilst businesses may not be in a position to purchase now, they need to consider practical steps for their eventual rollout.

• • Consider your business needs: As mentioned throughout this article, AI tools are not one-size-fits-all. They excel at solving specific problems but won’t work miracles across the board. Spend time identifying the pain points in your business and the opportunities where AI could add tangible value and provide useful solutions. By focusing on specific challenges, you can better align your investment with tools designed to address those issues.

• • Be honest with yourself: When assessing these needs, be honest about your business’ AI maturity. Resources are only as good as the information they are utilising, so if the quality of your data is poor or existing systems that would need integrating with are not mature enough, then it may not yet be the right time to invest. AI doesn’t need to take you from A to Z immediately; sometimes finding a resource that gets you from A to B is the right move.

• • Challenge your existing vendors: Almost all of our current vendors are considering the impact of AI on their business model, so now is the time to engage with them and challenge them about their next steps. Implementing AI doesn’t have to involve identifying new disrupters in the market, it can also be about working with existing, trusted suppliers to improve the quality of their offering through AI enhancements.

• • Involve key stakeholders early: Once you do identify a resource it’s important to ensure that you utilise your knowledge of your businesses and manage internal stakeholders. Many of the companies releasing AI tools are start-ups that may lack the information security infrastructure to meet our business requirements. Even if the tools are developed by more established organisations, there remain questions around the use of data inputted into the tools which will need to be addressed early by information security, risk and IT departments.

• • Don’t be tempted by long agreements: And finally, when you do come to purchase a resource, the temptation can be to seek longer term agreements to manage the associated costs. Whilst this can sometimes be more practical, especially when integrating with existing resources, the AI market is evolving so rapidly that what works today may not be the best solution in three years’ time, so where possible increased flexibility is preferable.

STRATEGIC ADOPTION OVER HYPE

The proliferation of AI tools offers immense potential, but successful adoption requires a deliberate, solution-oriented strategy. Prioritising a solution rather than succumbing to the allure of AI for its own sake is key – AI is not a one-size-fits-all solution or a magic fix for every business challenge. Its value lies in addressing specific pain points and creating targeted opportunities. By taking the time to identify your firm’s unique needs, upskill your teams and implement a structured framework for assessing and testing tools, you can ensure that your investment delivers meaningful results.

AI’s prevalence will only accelerate in the coming years. Whilst businesses all have different AI maturities and are at different stages of their AI journey, the key is not to chase what’s new and enticing but to implement the right tool for your business at the right time.