LLM API Cost Guide: Token Pricing, Inference vs Training & Unit Economics (2026)

Why Token-Based Pricing Changes FinOps Math

Traditional cloud FinOps optimizes provisioned capacity: an instance runs whether it is busy or idle, so the lever is matching size to load. LLM API spend has no idle state to eliminate — every token billed corresponds to a token actually processed. The optimization question shifts from "is this resource sized correctly?" to "does this request need this many tokens, from this model, without a cache hit?"

The Token-Based Pricing Model

Every major LLM API provider prices on the same basic structure: a rate per million input tokens, a separate (usually lower) rate per million output tokens, and a discounted rate for tokens served from a prompt cache. Providers publish official pricing pages that are the only reliable source for current rates, because list prices change with each new model release.

// prices below checked July 19, 2026 against provider pricing pages — treat as illustration of the pricing model, not a current-rate guarantee

The structural pattern that matters for FinOps planning is consistent across providers even as the exact numbers change: input and output tokens are priced separately (output is typically several times more expensive per token), cached tokens cost a fraction of fresh tokens, and asynchronous batch processing is discounted roughly in half. Build your cost model around that structure, not around today's specific dollar figures.

Inference vs Training Cost Structures

Managed API spend is entirely inference — there is no training cost, because the provider trained the model and amortizes that cost into the per-token price. This is the key structural difference from self-hosted models: a team using OpenAI or Anthropic APIs has no training line item to manage at all, while a team fine-tuning or training its own model incurs GPU-hour training costs (see GPU Cost Optimization) on top of, or instead of, inference costs. Fine-tuning a managed model sits in between: providers typically charge a separate per-token or per-hour training fee for the fine-tuning job itself, then bill inference on the fine-tuned model at a different (often higher) per-token rate than the base model.

Prompt Caching and Batch Processing Reduce Effective Cost

Two discount mechanisms are available on essentially every managed LLM API and require no infrastructure change to use — only application-layer changes:

• Prompt caching. Repeated identical prompt prefixes (system prompts, RAG context, few-shot examples) are cached server-side and billed at a fraction of standard input pricing on subsequent calls — per Anthropic's published pricing, cache reads are billed at 0.1× the base input rate. Cache writes carry a premium (1.25×–2× depending on cache duration), so caching pays off only once the same prefix is reused at least once.
• Batch processing. Both OpenAI and Anthropic offer an asynchronous batch API at roughly 50% off standard synchronous pricing, appropriate for any workload that does not require an immediate response — nightly summarization jobs, bulk classification, offline evaluation runs.

Both mechanisms are documented in provider pricing pages linked above and should be re-verified before being used in a budget, since rates and multipliers are subject to change with new model releases.

Unit Economics: Cost Per Request

The FinOps-relevant metric for LLM API spend is rarely total monthly token spend on its own — it is cost per unit of business value: cost per user query, cost per document processed, cost per completed agent task. Calculating this requires attributing token spend to the feature or product area that generated it, the same application-layer instrumentation problem covered in Unit Economics for Cloud. Once cost-per-request is visible, it becomes a metric to review alongside conversion, latency, and quality — making model-selection and prompt-length tradeoffs explicit rather than invisible.