Know who created the spend
Tag calls with team, product, feature, environment, route and business task so shared infrastructure does not erase accountability.
Guide / Operating model
LLM FinOps for engineering, finance and product
LLM FinOps is the operating discipline for making AI cost visible, attributable, forecastable and governable without separating cost from product value. It gives engineering, finance and product one shared language for deciding what an AI feature should cost before and after deployment.
Definition
LLM FinOps is not merely a dashboard of token spend.
A spend dashboard answers what has already been billed. LLM FinOps goes further. It connects architecture to demand, demand to unit economics, unit economics to product value, and every forecast to actual outcomes. The discipline asks who owns the cost, which software change created it, how uncertain the forecast was, whether the feature is economically justified and which engineering control should be applied when the cost moves outside the approved range.
The operating model is collaborative. Engineering understands callsites, context, retries, routing and tool behaviour. Product understands user demand and the value of a completed task. Finance understands budgets, variance and accountability. None of those functions can produce a defensible AI forecast alone. LLM FinOps creates the review process, evidence and feedback loop that joins them.
Why AI is different
Traditional cloud allocation is necessary but insufficient.
Architecture changes spend immediatelyA model swap, larger output ceiling, extra tool schema or retry policy can change recurring cost in one pull request.
Usage is probabilisticOutput length, route choice, retries, fallback frequency and agent turns vary from task to task, creating a distribution rather than one fixed unit cost.
Provider rates are only one layerToken prices matter, but completed-task cost also includes failure, rework, tools, cloud services and human review.
Demand can be product-drivenA successful feature may increase usage rapidly, while background agents can create demand without a user explicitly pressing a button.
Quality and cost interactThe cheapest model per token may be expensive per useful outcome if it retries, fails, escalates or creates more human work.
Cost ownership is ambiguousShared models, central gateways and multi-product agents can make allocation difficult unless identifiers are designed into the workflow.
The lifecycle
Govern cost before, during and after deployment.
Plan. Define the business task, expected demand, target quality, acceptable latency and cost envelope. Treat these as connected product requirements.
Estimate. Convert the proposed architecture into calls, tokens, tools, retries, routes and probability distributions. Report expected cost and conservative percentiles.
Approve. Assign a budget owner and review the positive monthly P90 delta while the architecture is still negotiable.
Operate. Attribute usage to teams, products, environments, features and task types. Monitor cost, quality, latency and completion together.
Reconcile. Compare the approved estimate with actual usage and invoices. Explain variance by quantity, rate, structure and behaviour.
Calibrate. Update future assumptions using measured distributions while preserving the historical estimate that was originally approved.
Core practices
Seven capabilities make LLM FinOps operational.
Measure useful outcomes
Track cost per completed task, successful user action, document processed or decision supported rather than cost per token alone.
Model uncertainty
Separate architecture quantities, demand growth, provider rates and operational tails, then report P50, P90 and P95.
Put review where change happens
Use advisory comments first, then apply a blocking gate when a positive P90 delta exceeds an owned budget.
Attack the largest driver
Prioritize model fit, context, output, retries, fallbacks and tools according to their contribution to the tail.
Explain estimate versus actual
Distinguish changes in volume, token mix, route frequency, provider price and implementation behaviour.
Connect cost to product outcomes
A higher-cost workflow may be justified when it materially improves completion, conversion, quality or labour savings.
Allocation design
Cost attribution begins in the application, not in month-end reporting.
Provider invoices often identify an account, project or API key but not the business reason for a call. A useful allocation model adds application-level identifiers before traffic reaches the provider. Typical dimensions include team, service, feature, environment, customer segment, workflow, task type, experiment and model route. The identifiers should be stable enough for finance and detailed enough for engineering without exposing sensitive customer content.
Allocation should also preserve shared costs. Gateways, vector databases, observability and human review may support several features. Choose a transparent rule such as request volume, token volume, completed tasks or reserved capacity. Do not hide shared costs simply because direct allocation is difficult. Record the rule and change it only when a better causal driver exists.
Forecasting
Separate quantity, rate, structure and escalation.
Quantity includes users, tasks, calls, tokens, tools, retries and route frequencies. Rate includes provider pricing and metered infrastructure. Structure describes how the workflow is assembled: which models are used, what context is repeated and where fallbacks occur. Escalation describes how rates, demand and architecture may change over time. Keeping those categories separate makes variance explainable and prevents a single growth percentage from concealing several unrelated assumptions.
For a new feature, use scenarios and distributions rather than one point estimate. The estimate can include a base adoption case, a growth case and a tail case, but the assumptions must remain visible. P90 is useful as a budget case because it captures more uncertainty than the average without pretending to be a maximum. Estimate maturity should be reported separately so a new Class 5 screening estimate is not mistaken for a definitive forecast.
Operating cadence
A lightweight rhythm prevents cost governance from becoming bureaucracy.
Per pull requestReview material architecture changes, positive P90 deltas, model swaps, output ceilings, retries, fallbacks and new agent routes.
WeeklyInspect unusual cost per task, route shifts, retry spikes, context growth and new unallocated spend.
MonthlyReconcile estimates with actuals, explain variance, update forecasts and review budget ownership.
QuarterlyReview model portfolio, vendor exposure, unit economics, policy thresholds and the maturity of the estimation basis.
Maturity model
Move from visibility to calibrated governance.
Stage 1: visibility. Teams can see total spend and basic provider usage. Allocation and business context remain weak.
Stage 2: attribution. Calls are tagged to teams, features and environments. Owners can explain where the money went.
Stage 3: unit economics. Cost is connected to completed tasks, quality and product outcomes. Optimization becomes economically meaningful.
Stage 4: pre-merge forecasting. Material code changes receive cost estimates before deployment. Review includes uncertainty and budget ownership.
Stage 5: calibrated policy. Estimates are reconciled with actuals, modelled distributions improve and trusted P90 gates can block unapproved recurring spend.
FAQ
LLM FinOps questions.
Is LLM FinOps only for large companies?
No. Small teams benefit from simple ownership, cost per completed task and pre-merge review before architecture debt grows. The process should match the scale of the spend.
Who should own LLM FinOps?
Ownership is shared. Engineering owns technical quantities and controls, product owns demand and value, and finance owns budgets and variance discipline. One accountable programme lead should coordinate the system.
What metric should executives see?
Show monthly cost, P90 forecast, cost per completed task, business value, variance to plan and the largest controllable driver. Avoid presenting token volume without outcome context.
When should a gate become blocking?
After the team trusts the basis, has a declared budget owner and understands the exception process. Advisory reporting should come first.
How does Class1 fit?
Class1 moves estimation and approval into the pull request, then supports the estimate-to-actual calibration loop. See the PR cost gate and methodology.
Next step
Start with one workflow and one owner.
Choose a material AI feature, define its completed-task unit, map its cost drivers and assign a monthly P90 budget owner. That creates a practical foundation for LLM FinOps without requiring a large transformation programme.