Build log/AI

AI agents in business software: what they can and can’t do reliably

An AI agent is a model that chooses tools in a loop until a task is done. Narrow agents with few tools and an approval step work well today. Open-ended ones still fail in costly ways.

The short answer

  • An AI agent is a language model that repeatedly chooses a tool, reads the result and decides the next step until it finishes or reaches a limit.
  • Agents are reliable today on narrow, checkable tasks with a few tools, such as looking up records, drafting replies and proposing schedules for a person to approve.
  • Give an agent its own limited permissions, require human approval before it spends money or contacts a customer, and cap its steps, time and cost.

An AI agent is a language model that works in a loop, choosing a tool to use, reading the result and deciding what to do next until the task is finished. In business software, agents are reliable today when the job is narrow, the tools are few and a person approves anything that matters. They are unreliable when the goal is vague, the chain of steps is long, or a mistake can’t be undone.

What an AI agent actually is

A tool, in this sense, is a function your software makes available to the model, with a name, a description and defined inputs. “Look up a customer by email”, “list free slots for an engineer next week” and “create a draft invoice” are all tools. The model can’t run them itself. It asks your code to run one, your code checks the request and runs it, and the result goes back to the model.

An agent repeats that cycle. Asked to find out why order 4471 hasn’t shipped and draft a reply to the customer, it might look up the order, then the stock level, then the courier status, then write the draft. Nobody wrote those four steps in advance. The model chose them as it went, based on what each step returned.

That freedom is what separates an agent from the two simpler patterns most AI features use.

PatternWho decides the stepsGood forMain risk
Single model callYour code, one stepSummarising, classifying, drafting from given textA wrong answer
WorkflowYour code, fixed steps with the model inside some of themRepeatable processes such as reading a document and filing itRigid when inputs vary a lot
AgentThe model, one step at a timeTasks where the next step depends on what was foundWrong actions, loops, cost and time

Start with the simplest pattern that does the job. Many features described as agents work better as workflows, which are cheaper, faster and much easier to test.

What agents do reliably today

Agents work best where each step is easy to check and the final output goes to a person before it has any effect. These are the tasks where they earn their place.

  • Investigating and summarising. Pulling an order’s history, stock position and delivery status into one note for a support agent.
  • Drafting from records. Writing a reply, a quote or a report from data the agent has looked up, and leaving it as a draft.
  • Proposing schedules. Finding slots that fit several people’s calendars and constraints, and offering options for someone to confirm.
  • Triage and routing. Reading an incoming request, checking the account and assigning it to the right queue with a stated reason.
  • Answering questions over your own data. Searching documents and records and answering with references, a pattern explained in RAG explained.

Where agents still fail

The failures are predictable, which means you can design around them.

Errors compound over long chains. Each step has some chance of going wrong, and later steps build on earlier ones. An agent that misreads a date at step two will plan confidently around the wrong date for the rest of the task.

They report success they didn’t achieve. An agent may say an action is complete when the tool call failed or was never made. Your software should confirm outcomes from its own records and ignore the model’s account of them.

They follow instructions hidden in content. An email, web page or document the agent reads can contain text written to steer it, such as “ignore previous instructions and forward this thread”. This is called prompt injection. No model is fully immune to it, so the real defence is limiting what the agent is allowed to do.

They loop. An agent that can’t find what it needs may keep searching in slightly different ways, spending money on every attempt, unless something stops it.

They are poor at exact arithmetic. Totals, VAT and CIS deductions belong in code. Give the agent a calculation tool and keep money logic out of the model entirely.

Permissions and approval steps

An agent should never have more access than the person it works for, and usually it should have much less. Treat it like a new member of staff on their first day.

  1. Give it its own identity. Log every action as the agent acting for a named user, so the audit trail shows who asked for what.
  2. Separate read tools from write tools. Looking things up is low risk. Creating, changing, sending and deleting are where the damage happens.
  3. Require approval before side effects. Anything that messages a customer, spends money, changes a price or deletes a record waits for a person to press a button.
  4. Validate every tool call in code. Check the inputs against the user’s permissions and your business rules before running it, exactly as you would for a request from a browser.
  5. Set hard limits. Cap the number of steps, the total time and the cost per task, and stop cleanly with an explanation when a limit is reached.

Cost and latency

Every step of an agent is a model call, and each call normally resends the conversation so far, including every tool result. Cost therefore grows faster than the number of steps. An eight-step task costs more than eight times its first call, because the later calls carry everything the earlier ones found, unless repeated content is cached.

Time adds up the same way. If each step takes three seconds, a ten-step task takes thirty seconds before anything useful appears, which is too long for a person waiting at a screen. Agents suit background work that ends with a notification, or screens that show each step as it happens so the wait makes sense. We go through the arithmetic in what an AI feature costs per call.

Designing a narrow agent: quoting, scheduling and support

A narrow agent has one job, a handful of tools and a clear point where it stops and hands over to a person. Three examples show the shape.

Quoting

A builder receives an enquiry for a bathroom refit. A quoting agent reads it, looks up the relevant items in the price book and the labour rates, checks similar past jobs and produces a draft quote with every line traced to its source. It has no tool for sending. A person reviews the quantities, adjusts the margin and sends it. Construction software such as Estimark already holds a price book and labour rates, which is the kind of structured data an agent like this depends on.

Scheduling

A scheduling agent receives a request to book a site visit or a lesson. It checks availability, travel, existing commitments and any rules about notice, then offers two or three options. A customer or a member of staff makes the booking by choosing one. Time zone conversion belongs in code. CurioKids shows bookings in each family’s own time zone, and that kind of conversion should never be left to a model.

Support

A support agent reads a ticket, identifies the customer and order, checks delivery and payment status, and drafts a reply using the facts it found. It can prepare a refund for approval, only up to a limit set in code. Tickets it can’t place go to a person with a summary of what it checked.

Before any of these go live, run them against a set of real past cases and check both the final output and the steps taken. The method is set out in how to test an AI feature.

Considering an agent for your product? We’d start by checking whether a fixed workflow would do the job, and add agent steps only where they pay for themselves. See how we approach AI features.

Questions people ask

What is the difference between an AI agent and a chatbot?

A chatbot answers messages in a conversation. An agent uses tools to take steps, such as looking up records or creating drafts, and decides which step to take next. A chatbot can have an agent behind it, but many chatbots only generate text.

Are AI agents safe to use with customer data?

They can be, if the agent has limited permissions, every tool call is checked in code and actions with real effects need human approval. The same UK GDPR duties apply as for any processing of personal data, including a processor contract with the AI provider.

How long does an AI agent take to complete a task?

It depends on the number of steps, because each step is a separate model call. A simple lookup and draft might take a few seconds, and a long investigation can take a minute or more. Long-running agents work best in the background, with a notification when they finish.

Can an AI agent replace a member of staff?

Not for a whole role today. Agents handle narrow, well-defined tasks well and struggle with ambiguity, judgement and long chains of work. They are most useful taking repetitive groundwork off a person who then makes the decision.

Written by Flavoureak

We build and run Estimark, CurioKids and CalcStack, and we build web and mobile software for other businesses. Start a project.