Build log/AI

What an AI feature costs per call, and why we work it out first

A model call has a price you can calculate before you build anything. Here is the sum, where the money goes, and three ways to spend less.

The short answer

  • Every model call costs (input tokens × input rate) + (output tokens × output rate).
  • The shape of the feature matters more than the rate: resending context and looping agents multiply cost.
  • Send less, route simple work to smaller models, and cache what repeats.

The sum

Language models are billed by the token. A token is a piece of a word; in English text a token averages about four characters, so a thousand words is roughly 1,300 tokens. Every provider publishes two rates: one for the tokens you send, and a higher one for the tokens the model writes back.

cost per call = (input tokens × input rate) + (output tokens × output rate)

That is all there is to it, which means the cost of a feature can be worked out on paper before anyone writes a prompt.

A worked example

Take a feature that reads a 30-page tender and drafts a priced schedule. Thirty pages of dense text is around 15,000 words, or about 20,000 input tokens. The draft schedule it writes back might be 1,500 tokens. At example rates of £2 per million input tokens and £10 per million output tokens:

TokensRateCost
Input20,000£2 per million£0.040
Output1,500£10 per million£0.015
Per tender£0.055

At 400 tenders a month that is £22. If it saves an estimator two hours on each one, the model is the cheapest part of the feature. The rates here are examples; check your provider's current price list, because they change often.

Where the money actually goes

The rate is rarely the problem. The shape of the feature is.

  • Resending context. A chat assistant sends the whole conversation on every turn. By the twentieth message each call carries the nineteen before it, so the cost of a conversation grows much faster than its length.
  • Retrieval that over-fetches. Pulling ten documents into every call to be safe can multiply input tokens tenfold for no better answer.
  • Loops. An agent that plans, calls a tool, reads the result and plans again makes several model calls for one request. Each step is cheap. The loop is not.
  • Retries. If an output fails validation and is generated again, you pay twice.

Three ways we bring it down

Send less

Find the three pages of a tender that hold the bill of quantities and send those, not the whole document. Good retrieval is a cost control as much as a quality one.

Route by difficulty

Sorting an email into one of six categories doesn't need the largest model. A small, fast model handles routine work, and the large one is called only for drafting and reasoning.

Cache what repeats

Most providers discount input that repeats exactly between calls, such as a long set of instructions or a reference document. Put the fixed part of a prompt first and the changing part last.

The cost of being wrong

The last line in the sum is the one nobody writes down: what happens when the answer is wrong. A drafted quote that a person checks costs a few minutes. A drafted quote that goes straight to a customer costs whatever the mistake costs. We design the review screen at the same time as the prompt, because it decides whether the feature saves money at all.

Try it with your own numbers. The AI guide has a calculator for volumes and rates.
Written by Flavoureak

We build and run Estimark, CurioKids and CalcStack, and we build web and mobile software for other businesses. Start a project.