The short answer
- Sending customer data to an AI model is processing under UK GDPR, so you need a lawful basis, transparency, data minimisation and appropriate security.
- An AI provider you call through an API is usually your processor, which means you need an Article 28 contract and must know where it stores data and for how long.
- The ICO’s guidance on AI and data protection says most uses of AI with personal data are likely to need a data protection impact assessment.
Using AI with customer data is processing personal data, so UK GDPR and the Data Protection Act 2018 apply in full. In practice that means a lawful basis for the use, a privacy notice that explains it, sending the model no more data than it needs, a written contract with the AI provider, and usually a data protection impact assessment before you go live. Personal data is any information relating to an identified or identifiable living person, and that includes what a model writes about them.
This is a practical summary for people building or buying software. Take legal advice for anything with real risk.
Choosing a lawful basis for AI processing
Every use of personal data needs a lawful basis under Article 6 of UK GDPR. For AI features in business software, two bases come up most often.
- Contract. The processing is necessary to deliver a service the customer has asked for. An AI feature that is part of the product the customer signed up to, such as a tool that drafts quotes from their own enquiries, may fit here.
- Legitimate interests. You have a genuine interest, the processing is necessary for it, and it doesn’t override the person’s interests and rights. This needs a written legitimate interests assessment, kept on file.
Consent is possible but awkward for AI, because it must be freely given, specific, informed and as easy to withdraw as to give.
Special category data, such as health information, needs an additional condition under Article 9. Children’s data needs extra care, and online services likely to be accessed by children must also meet the ICO’s Age Appropriate Design Code, which we cover in the Children’s code for edtech.
Telling people what the AI does
Transparency means people know their data is used by AI, for what purpose and who receives it. Your privacy notice should name the purpose in plain words, such as “we use an AI model to summarise your support requests so an agent can reply faster”. It should name the categories of recipient, including the AI provider, and say where data goes and how long it is kept.
Where AI output affects someone directly, say so on the screen where it happens as well as in the policy.
People keep their rights over data processed by AI. They can ask for a copy of their personal data, including AI outputs about them, and you normally have one month to respond. Build the feature so those outputs can be found and exported.
Data minimisation and the AI prompt
Data minimisation means using only personal data that is adequate, relevant and limited to what the purpose needs. With AI, the place to apply it is the prompt. Developers often send a whole customer record because it is convenient. A model summarising a complaint usually needs the complaint text and the order details. It doesn’t need the customer’s date of birth, phone number or payment history.
- Choose the fields each task needs in code, and send only those.
- Replace names and identifiers with placeholders where the task allows, and restore them afterwards.
- Strip signatures, forwarded chains and attachments that aren’t relevant to the task.
- Never paste whole exports or databases into a prompt to save time.
AI providers as processors and sub-processors
When you send data to an AI provider through its API to do a job for you, the provider is usually your processor. You are the controller, which means you decide why and how the data is used, and you stay responsible for it. Article 28 requires a written contract with every processor. Most providers offer a data processing addendum to business and API customers. Read it and keep a copy.
The provider will also use sub-processors, typically the cloud companies that host its models. The contract should list them, or say where the list is published, and explain how you will hear about changes. Ask these questions before you sign.
| Question to ask the provider | Why it matters under UK GDPR |
|---|---|
| Is there a data processing addendum that meets Article 28? | A written processor contract is a legal requirement |
| Is our data used to train or improve your models? | If so, the provider is using it for its own purposes, which a processor arrangement doesn’t cover |
| How long are inputs and outputs kept, and why? | Retention must be limited and explained in your privacy notice |
| In which countries is data processed and stored? | Transfers outside the UK need a valid transfer mechanism |
| Who are your sub-processors, and how are changes notified? | You need to know everyone who handles the data |
| What security controls and certifications do you have? | Article 32 requires security appropriate to the risk |
International transfers and retention by AI providers
Many AI models run in data centres outside the UK. Sending personal data abroad is a restricted transfer under UK GDPR and needs a lawful route. The usual routes are UK adequacy regulations, which cover the EEA and some other countries, the UK extension to the EU-US Data Privacy Framework for US companies certified under it, and the ICO’s International Data Transfer Agreement or the UK Addendum to the EU standard contractual clauses. The contractual routes also need a documented risk check. Some providers offer processing in a UK or European region, which simplifies this.
Retention is the other question. Many providers keep inputs and outputs for a period, often to detect misuse, and some offer shorter or zero retention for business customers on request. Whatever the answer, record it in your records of processing and reflect it in your privacy notice. Your own logs of AI calls count too, so give them a retention period and delete on schedule. Logging and monitoring are covered in how to test an AI feature.
When you need a DPIA
A data protection impact assessment, or DPIA, is a documented assessment of the risks a type of processing poses to people and of how you will reduce them. Article 35 makes a DPIA mandatory where processing is likely to result in a high risk to people’s rights and freedoms. The ICO’s guidance on AI and data protection says that most uses of AI with personal data are likely to meet that test, and the ICO lists the use of innovative technology, including AI, among the factors that point to a DPIA.
Do it before you build, while the design can still change. Describe the flow of data from the screen to the provider and back, the lawful basis, what could go wrong and the controls that will prevent it. Typical risks are wrong outputs about a person, data leaking through logs, and outputs that treat one group worse than another. If a high risk remains after your controls, you must consult the ICO before going ahead.
Automated decisions and Article 22
Article 22 of UK GDPR has long restricted decisions based solely on automated processing that have a legal or similarly significant effect on someone, such as refusing credit online or rejecting a job application with no person involved. “Solely” means no meaningful human involvement. A person who approves every output without looking doesn’t count.
The Data (Use and Access) Act 2025 replaces Article 22 with a new set of rules. In broad terms, it allows more solely automated significant decisions as long as safeguards are in place, and keeps tighter limits where special category data is involved. The safeguards include telling people about such decisions, letting them make representations, and giving them a way to obtain human intervention and contest the outcome. Check which rules apply on the date you go live, read the ICO’s current guidance and take legal advice before automating any decision of this kind.
Most AI features in business software avoid the question by design. The model drafts, sorts or suggests, and a person decides. That is good practice in any case, as we explain in designing AI features for when the answer is wrong.
Questions people ask
Is it legal to use AI on customer data in the UK?
Yes, if you meet UK GDPR as you would for any other processing, with a lawful basis, a clear privacy notice, data minimisation, a processor contract, a lawful route for transfers abroad and appropriate security. Take legal advice where the risks are high.
Can staff paste customer emails into an AI chatbot?
Only if the tool is covered by a proper processor contract and your privacy notice and lawful basis cover the use. Consumer chat apps on personal accounts usually aren’t covered. Give staff an approved tool and a short written rule about what may be pasted into it.
Do we need consent to use AI on customer data?
Not necessarily. Consent is one of several lawful bases, and contract or legitimate interests often fit AI features in business software better. Whichever you choose, record why it applies and explain the use in your privacy notice.
Does UK GDPR apply to anonymised data in prompts?
Truly anonymised data, where nobody can be identified by any means reasonably likely to be used, is outside UK GDPR. Swapping names for placeholders is usually pseudonymisation, and pseudonymised data is still personal data. Treat it as personal data unless you are confident it can’t be linked back to a person.