Could your firm already be in breach of its professional obligations — without anyone realising it? That is not a hypothetical question. In a judgment handed down in November 2025, the Upper Tribunal issued explicit guidance on the use of AI tools in legal practice, with direct relevance to every regulated law firm in England and Wales.
In UK v Secretary of State for the Home Department (AI hallucinations; supervision; Hamid) [2026] UKUT 81 (IAC), the Tribunal observed:
“Uploading confidential documents into an open-source AI tool, such as ChatGPT, is to place this information on the internet in the public domain, and thus to breach client confidentiality and waive legal privilege, and any such conduct might itself warrant referral to the SRA and should, in any event, be referred to the Information Commissioner's Office.”
This observation was made in a Hamid professional standards hearing rather than as a binding determination of privilege law, but it carries significant authority as the view of a three-judge Upper Tribunal panel. It also reflects an established English common law principle: voluntary disclosure of privileged material to a third party risks waiving that privilege. Solicitors who upload client documents to consumer AI tools are therefore exposed on two fronts — regulatory referral risk as articulated by the tribunal, and privilege waiver risk under common law.
This blog post forms part of our two-part mini-series, Using AI Safely in Legal Practice. In this first part we explore three key principles for evaluating AI providers. Part 2 addresses AI hallucinations and the supervision liability that now sits squarely with partners and fee-earner supervisors.
ChatGPT, Google Gemini, Claude, Copilot and the Frontier AI Providers
Many firms will already be familiar with the major AI tools on the market and a significant number have already adopted them for everyday work. However, the Tribunal's observation requires some important context before you take its words at face value.
The Tribunal described ChatGPT as an “open-source AI tool”. This is technically incorrect. When software is described as open-source, it specifically means that the original source code is made available to anyone to inspect, modify and distribute. ChatGPT, Google Gemini, Claude and all of the other frontier AI providers are commercial applications — their source code is private and not available to the public.
That said, the Tribunal used the term loosely and its legal conclusion stands. What matters is not whether a tool is open-source in the technical sense, but how a particular AI provider uses the data you send to it and receive back from it. That is the correct question for any regulated firm to ask — and the answer varies significantly between providers, tiers and contracts.
What Are the Specific Privacy Concerns with AI Tools?
There are several ways in which AI providers and the application providers that embed AI into their products can utilise the data you share with them. Understanding these categories is essential to making an informed choice.
Training the AI Model on Your Data
The first and most significant risk is that providers may use your inputs to train and improve their AI models. This process can happen through automated means or with human review and either route can result in the contents of your interaction being exposed.
There is a further risk that is less commonly understood: data used for model training can, in some circumstances, be reproduced in responses to other users. This means that privileged client information entered into an AI tool could, in principle, surface in a response delivered to a completely unrelated third party. The data is not simply stored — it may be retransmitted.
Principle One — Select an AI provider that does not use your data to either train their models or improve their service.
Abuse Monitoring and Human Review
Data retention for training is only part of the picture. AI providers also monitor their services for misuse, which is a legitimate and necessary function — but one that has significant implications for client confidentiality.
Depending on the provider, your interactions will typically be stored for up to 30 days. If a particular interaction triggers an abuse threshold, it may then be subject to human review, once again creating a route by which client data could be exposed.
This risk became concrete in 2025 through litigation in the United States. In The New York Times Company v. Microsoft Corporation et al., Case No. 1:23-cv-11195 (S.D.N.Y.), US Magistrate Judge Ona T. Wang issued an order on 13 May 2025 requiring OpenAI to preserve and segregate all ChatGPT output log data that would otherwise have been deleted — including chats deleted by users and data from “temporary chat” mode. This order applied to users of ChatGPT Free, Plus, Pro and Teams.
The case escalated further in November 2025, when the court ordered OpenAI to produce 20 million de-identified ChatGPT chat logs to the plaintiffs. It is worth noting that ChatGPT Enterprise and API customers with a Zero Data Retention (ZDR) agreement were outside the scope of both orders — a distinction that directly illustrates the importance of contracting correctly, not just choosing the right brand.
The practical implication for solicitors was stark: even pressing “delete” did not help. When a user deleted a chat, it disappeared from their account view but remained on OpenAI's servers under legal hold. The delete function had, in effect, become cosmetic. OpenAI confirms that the obligation to retain all new consumer data under the preservation order ended on 26 September 2025, and has since returned to standard deletion practices.
Beyond the confidentiality risk, this scenario introduces significant data protection concerns. Firms may find themselves unable to honour data retention obligations, in conflict with their own Privacy Notices and Data Processing Agreements and potentially exposed to clients whose personal information has been retained and processed outside the scope of what was disclosed to them.
Principle Two — Select AI and application providers that have transparent abuse monitoring and retention policies — in particular, providers that do not retain messages sent by end users or the responses returned from the AI model. Note: this principle applies both to AI providers directly (such as OpenAI) and to application providers that embed third party AI models into their solutions.
Geographic Data Residency
A third area of risk concerns where your data is processed. This is particularly important when handling personally identifiable information subject to the UK Data Protection Act 2018 and UK GDPR.
The capabilities of different providers vary considerably. Some can guarantee regional deployments; some can isolate processing to a specific country. Selecting a provider without considering residency requirements can mean you need to update your Privacy Notices, Data Processing Agreements and in some cases notify clients about changes in how and where their data is processed — an outcome that is both time-consuming and reputationally sensitive.
There is also a practical tension worth acknowledging: the most capable and accurate AI models are typically released in the United States or Europe first and may not be available with UK-only data residency at launch. Firms therefore need to make a conscious decision about how to balance model capability against data residency requirements, rather than simply defaulting to whichever tool is most familiar.
Principle Three — Select a provider that can meet your specific data residency requirements and ensure that your contractual documentation reflects those requirements.
The Chancery Approach to AI Privacy
Chancery AI has been built with these principles at its foundation. We recognise that UK law firms cannot adopt AI tools on trust alone — they need contractual assurance. Our platform is designed so that firms can adopt it with confidence, knowing that the three principles above are not aspirations but contractual commitments embedded in how we select and work with our AI service providers.
We only work with AI providers who guarantee that your data is not used to train or improve their models, that messages sent to and returned from the AI are not retained — including for abuse monitoring purposes — and that firms have flexibility to choose where their data resides. If you would like to understand how this works in practice, we are happy to walk through the detail with you.
Summary
There are of course many other factors to consider when evaluating the security of any AI supplier and no single checklist will cover every scenario. However, by applying these three principles as a starting point, law firms can make more informed decisions and work towards ensuring they uphold client confidentiality and maintain legal privilege when adopting AI.
Part 2 of this series addresses a separate but equally significant risk: AI hallucinations, the new statement of truth requirements introduced by the Upper Tribunal and why the Tribunal has held that supervisors carry greater regulatory exposure than the fee earner who produced the error.
Free Download: AI Provider Evaluation Checklist
A practical checklist for UK law firms evaluating AI providers — covering model training, data retention, abuse monitoring and geographic data residency. Work through it with your prospective provider before signing any agreement.