Raise first-pass completion
Improve task framing, structured outputs, validation and model fit where failures create repeated work.
Guide / AI unit economics
Measure LLM cost per completed task
Token rates tell you what a provider charges for model usage. They do not tell you what your product pays to finish useful work. Cost per completed task includes the entire route from the initial request to an acceptable outcome, including failure, retries, fallbacks, tools and human review.
The key distinction
Cost per attempt is not cost per outcome.
Suppose two models receive the same task. Model A has a lower token rate but often returns an invalid structure, misses required information or needs a stronger fallback. Model B has a higher rate but completes the task correctly on the first attempt. Comparing only the first call can make Model A appear cheaper. Comparing the cost of accepted completed tasks may reverse the decision.
This matters for chat, extraction, coding, classification, search, RAG and agents. Every system has a definition of useful completion, whether it is an answer accepted by a user, a valid record written to a database, a support case resolved, a pull request produced or a business decision approved. That definition should become the denominator of the cost metric.
What belongs in the numerator
Count every material resource required to finish the task.
Primary model callsInput, output, cached input and any provider-specific charges associated with the main route.
Retries and repairRepeated attempts after timeouts, malformed output, validation failures or low-confidence results.
Fallback and escalationCalls to alternative or stronger models when the primary route cannot complete the task.
Tools and retrievalSearch, vector retrieval, databases, browsers, OCR, code execution and third-party services used by the workflow.
Guardrails and evaluationPolicy checks, moderation, quality evaluators, structured-output validators and verification models.
Cloud infrastructureQueues, functions, gateways, storage, networking and observability where they are material to the task.
Human interventionReview, correction, approval or exception handling multiplied by the probability and duration of intervention.
Failed workResources consumed by tasks that never reach the accepted completion definition must remain in the numerator.
The denominator
Define completion so product, engineering and finance agree.
A completion definition must be observable and tied to business value. “The model returned a response” is usually too weak. A document extraction task may require all mandatory fields to pass validation. A coding agent may require tests to pass and a reviewer to accept the change. A support assistant may require the issue to remain resolved without reopening. A sales workflow may require a qualified record rather than a generated paragraph.
The definition should also distinguish partial success. Some workflows produce a useful draft that still needs human completion. That outcome can be measured separately with its own cost and labour requirement. Avoid forcing every route into one binary metric when the business genuinely values several levels of completion.
Once the denominator is stable, the team can compare models, prompts and architectures on a common basis. It can also detect a dangerous form of optimization: reducing token spend while lowering completion quality enough that total cost per accepted outcome rises.
A practical formula
Use weighted route cost divided by accepted completions.
Start by mapping every meaningful route: first-pass success, retry success, fallback success, human-assisted completion and failure. Estimate or measure the cost of each route and its probability. The expected cost per attempt is the probability-weighted sum of those routes.
Then divide total expected cost by the probability of accepted completion. If human-assisted outcomes count as completed, include them in the denominator and include labour in the numerator. If they do not meet the product requirement, leave them outside the denominator.
For actual measurement, aggregate costs and completions over a period that contains enough volume to be representative. Segment the results by task type, model route, customer tier or difficulty so a changing workload mix does not create misleading conclusions.
Worked comparison
Why the cheapest rate can lose.
Route ALower first-call rate, more invalid outputs, two repair attempts on some tasks and frequent escalation.
Route BHigher first-call rate, stronger first-pass completion and fewer human corrections.
Token-only viewRoute A appears cheaper because the first attempt uses a lower-priced model.
Completed-task viewAll calls, failures and labour are counted; Route B may produce the lower accepted-outcome cost.
DecisionChoose the route that meets quality, latency and risk requirements at the best completed-task economics.
Uncertainty
Cost per completed task is also a distribution.
Some tasks complete on the first call while others require long outputs, several retrieval steps or human review. A single average can conceal that variability. Model task difficulty, output length, route choice, retries and intervention as distributions where the tail is material.
P50 describes a typical modelled outcome. P90 supports a conservative budget case under the declared assumptions. For a product with millions of tasks, even a small tail probability may create significant monthly exposure. Report the expected unit cost, percentile unit costs and the monthly distribution after demand is applied.
Keep estimate maturity visible. Early route probabilities based on a test set are not equivalent to measured production behaviour. Store the original basis, compare it with actuals and update future forecasts without rewriting history.
Instrumentation
Capture enough evidence to calculate the metric correctly.
Task identifierAssign one stable identifier across all model calls, tools, retries, fallbacks and human actions related to the business task.
Route and attemptRecord model, provider, prompt version, route, attempt number and reason for retry or escalation.
QuantitiesCapture input, output, cached tokens, tool use, latency and metered infrastructure.
OutcomeRecord completion status, validation result, quality score, human intervention and final acceptance.
Business contextInclude feature, team, environment, task type and customer segment without logging sensitive content unnecessarily.
Rate versionPreserve the dated rate basis used to calculate cost so the result can be reconstructed after prices change.
Optimization
Improve the route, not only the token count.
Match difficulty to capability
Keep routine work on efficient routes and escalate only tasks that need stronger capability.
Reduce repeated input
Retrieve only relevant information, cache stable prefixes and summarize state between turns.
Fix recurring failure classes
Do not treat retries as a permanent operating assumption when the cause can be removed.
Target review by risk
Use confidence, impact and exception rules to focus human attention where it changes outcomes.
Budget per task
Set a unit-cost envelope and multiply by forecast task demand to create a transparent monthly budget.
Governance
Use unit economics before merge.
A pull request can change the completed-task economics by switching models, increasing maximum output, adding tools, changing routing or permitting more retries. Those code signals can be combined with declared demand and measured completion rates to estimate a monthly delta before deployment.
The approval should show both unit and monthly effects. A small increase per task can become material at scale, while an apparently large unit cost may be acceptable for a rare high-value workflow. The reviewer needs the task definition, value context, assumptions, estimate class, P90 monthly delta and largest controllable driver.
FAQ
Cost-per-task questions.
Is cost per completed task the same as cost per request?
No. One business task may require several requests, and some requests may fail. Cost per completed task aggregates the full route and divides by accepted outcomes.
Should human review be included?
Include it when review is required to produce the accepted outcome. Estimate both the probability of review and the labour time per reviewed task.
How do I compare models fairly?
Use the same task set, completion definition, quality threshold and route rules. Compare total completed-task cost, latency and quality rather than isolated token rates.
What if tasks have very different difficulty?
Segment by task type or difficulty and report the workload mix. A blended average can move simply because the mix changed.
Can Class1 estimate this before production?
Class1 can price code-visible changes and declared assumptions before merge. Actual completion rates then calibrate future estimates. Explore the worked estimator.
Next step
Replace token-price comparisons with outcome economics.
Define one completed task, instrument its routes and calculate the full cost of accepted outcomes. That metric gives engineering, finance and product a common basis for model selection and budget approval.