The short answer
- Every model call costs (input tokens × input rate) + (output tokens × output rate).
- The shape of the feature matters more than the rate: resending context and looping agents multiply cost.
- Send less, route simple work to smaller models, and cache what repeats.
The sum
Language models are billed by the token. A token is a piece of a word; in English text a token averages about four characters, so a thousand words is roughly 1,300 tokens. Every provider publishes two rates: one for the tokens you send, and a higher one for the tokens the model writes back.
That is all there is to it, which means the cost of a feature can be worked out on paper before anyone writes a prompt.
A worked example
Take a feature that reads a 30-page tender and drafts a priced schedule. Thirty pages of dense text is around 15,000 words, or about 20,000 input tokens. The draft schedule it writes back might be 1,500 tokens. At example rates of £2 per million input tokens and £10 per million output tokens:
| Tokens | Rate | Cost | |
|---|---|---|---|
| Input | 20,000 | £2 per million | £0.040 |
| Output | 1,500 | £10 per million | £0.015 |
| Per tender | £0.055 |
At 400 tenders a month that is £22. If it saves an estimator two hours on each one, the model is the cheapest part of the feature. The rates here are examples; check your provider's current price list, because they change often.
Where the money actually goes
The rate is rarely the problem. The shape of the feature is.
- Resending context. A chat assistant sends the whole conversation on every turn. By the twentieth message each call carries the nineteen before it, so the cost of a conversation grows much faster than its length.
- Retrieval that over-fetches. Pulling ten documents into every call to be safe can multiply input tokens tenfold for no better answer.
- Loops. An agent that plans, calls a tool, reads the result and plans again makes several model calls for one request. Each step is cheap. The loop is not.
- Retries. If an output fails validation and is generated again, you pay twice.
Three ways we bring it down
Send less
Find the three pages of a tender that hold the bill of quantities and send those, not the whole document. Good retrieval is a cost control as much as a quality one.
Route by difficulty
Sorting an email into one of six categories doesn't need the largest model. A small, fast model handles routine work, and the large one is called only for drafting and reasoning.
Cache what repeats
Most providers discount input that repeats exactly between calls, such as a long set of instructions or a reference document. Put the fixed part of a prompt first and the changing part last.
The cost of being wrong
The last line in the sum is the one nobody writes down: what happens when the answer is wrong. A drafted quote that a person checks costs a few minutes. A drafted quote that goes straight to a customer costs whatever the mistake costs. We design the review screen at the same time as the prompt, because it decides whether the feature saves money at all.