Agent FinOps: the cost per transaction
Quick answer:
An agent does not have a monthly cost: it has a cost per task. And the question that decides whether the project survives is whether that figure sits below what resolving the task is worth.
The bill arrives at month end, aggregated, hiding precisely what you need to know. An organisation can be paying eight cents for one resolved query and four euros for another without knowing it, because both land on the same line of the statement.
Why the aggregate bill misleads
With a data platform, spend is relatively stable: compute per hour, storage per volume. With agents it is not. The same agent, faced with two similar requests, may resolve one in three steps and another in twenty, because the second did not find the evidence first time and retried.
The result is a long-tailed distribution: most tasks are cheap and a handful eat the budget. Look only at the average and that tail never appears.
It is the same mistake we analysed with compute overspend on data platforms: looking at the total instead of the unit.
Where the spend actually comes from
Four sources, in order of how surprising they are:
- Retries. An agent that does not validate its evidence repeats the cycle. Each loop pays for the full context again.
- Accumulated context. Every step drags along what came before. In long tasks, the same text gets paid for over and over.
- The model chosen. Using the large model to classify an intent is paying surgeon rates for a plaster.
- Loops. The extreme case of the first: the agent does not converge and keeps going until something stops it. With no cap, nothing does.
Four levers that work
Model routing
Not every decision an agent makes needs the same model. Classifying, extracting fields or deciding whether a search is needed are tasks where a small model performs just as well at a fraction of the cost. Reserving the large model for the step that genuinely reasons typically cuts spend in half without touching quality.
Context caching
When several tasks share the same reference material, that context is cached rather than resent. It is the lever with the best effort-to-saving ratio in systems with a lot of stable documentation.
A cap on steps and spend per task
A hard limit per task: exceed it and the task escalates to a person. This is not service degradation, it is the difference between a four-euro incident and a four-hundred-euro one.
Context pruning
Dragging the whole conversation into every step is convenient and expensive. Passing only what the step needs takes design work, but it is what makes cost grow linearly rather than quadratically.
The threshold that decides whether a case survives
Here is the uncomfortable part. For every use case you have to write down two figures: what it costs to resolve with the agent and what it cost to resolve before. If the first does not fall below the second with margin, the case is not viable, however interesting it is technically.
Writing those two figures at the start saves difficult conversations at the end. And it rules out in an afternoon cases that would have consumed a quarter.
Who watches this
Cost per transaction belongs on the same dashboard as the agent's other indicators, not in a monthly finance report. Kept separate, it gets looked at too late. Sitting next to the human intervention rate and steps per task, it tells a complete story: an agent starting to go in circles shows up in spend before it shows up in errors, as we saw when discussing agent observability.
How Galde can help
We instrument cost per transaction from the first prototype, not at the end, and we agree the viability threshold of each use case with the client before building it. It is part of the same standard we apply when taking AI with Palantir AIP to production.
Conclusion
The discussion about the cost of generative AI usually stops at the price per million tokens, which is the least useful figure of all. What decides an agent's viability is how much each complete task costs, retries and context included, compared with what that task is worth.
Measuring it is easy. What is hard is accepting what it says when the answer is bad.
Frequently asked questions
What is cost per transaction for an AI agent?
It is what resolving a complete task costs, including every reasoning step, tool call, resent context and retry. It is the unit that determines whether a use case is viable, unlike the price per token.
What is the main source of overspend?
Retries and accumulated context. An agent that does not validate its evidence repeats the cycle paying for the full context each time, and in long tasks that drag multiplies spend without adding quality.
What is model routing?
Using different models for different steps: a small one to classify or extract, a large one only for reasoning that requires it. It typically halves spend without affecting the quality of the result.
What cap should be set?
A limit on steps and spend per task which, once exceeded, ends the task and escalates it to a person. Without that cap, a looping agent consumes until somebody notices, which is usually late.
