How to Reduce LLM Costs: Caching, Batching, Routing and Fewer Tokens

Four levers cut most LLM API bills, and each provider's own price list sizes them. Prompt caching bills a repeated prompt prefix at 2.5% to 10% of the input price. The batch APIs at OpenAI, Anthropic, and Google charge half price for work that can wait up to 24 hours. Small models such as GPT-6 Luna and Claude Haiku 5.5 cost a twentieth of GPT-6.1 Sol and Claude Sonnet 5.5. And output costs 5 to 8.3 times as much as input, so a token you cut from the answer saves more than a token you cut from the prompt. In the worked example below, caching, batch, and a small model first take a $5,850 month to $1,404.

Every price below comes from the provider's own page, checked October 29, 2026, and our LLM API pricing guide has the full tables.

What is the cheapest way to run an LLM API?

Run the smallest model that passes your evals, cache everything that repeats, send everything that can wait to batch, and cap the output.

Caching and batch keep the same model on the same prompt, so start there. Routing and output caps change what users get: test them against your evals first.

How does prompt caching work at OpenAI, Anthropic, and Google?

All three cache the start of a prompt. A request hits the cache only when its first tokens match an earlier request's exactly, so put what never changes first (instructions, tool definitions, reference documents) and what changes last (the user's question, timestamps, account details).

OpenAI caches without any setting, and prompt_cache_key is optional on GPT-5.6 and later. Read usage.input_tokens_details.cached_tokens on each response to see how much of the prompt hit. Its batch price list carries cached input too, at $0.05 per million on GPT-6.1 Sol.

Anthropic needs cache_control. A 5-minute write pays for itself after one read, a 1-hour write after two. The cache follows the order tools, system, messages, and a change at one level breaks it and every level after: edit one tool definition and you lose the whole cache. Prompts under the minimum run uncached with no error, so check cache_creation_input_tokens and cache_read_input_tokens; if both read 0, the request missed.

Gemini caches on its own, and Google's advice matches the others: "Try putting large and common contents at the beginning of your prompt." Gemini 4 Argon and later models drop explicit caching, so the implicit cache is the one you tune for.

When should you use a batch API?

Whenever nobody waits for the answer. All three providers charge half price for batch requests:

• OpenAI returns within 24 hours and takes up to 50,000 requests or 200 MB per batch, on rate limits of its own, for /v1/responses, /v1/chat/completions, /v1/embeddings, and more. OpenAI Batch API.
• Anthropic finishes most batches within an hour, expires them at 24 hours, doesn't bill expired requests, and takes up to 100,000 requests or 256 MB. Batch and caching discounts stack, but cache hits inside a batch are best effort, 30% to 98% by Anthropic's count. Claude batch processing.
• Gemini targets 24 hours, accepts input files up to 2 GB, and supports context caching in batch. Gemini Batch API.

Flex is the middle option: a synchronous call at batch prices. OpenAI's service_tier: "flex" bills at Batch API rates, answers slower, and can return 429 Resource Unavailable, which it doesn't charge for. Gemini's Flex tier costs 50% of standard, targets 1 to 15 minutes, and returns a 503 or a 429 when capacity runs out, without moving the request to the standard tier, so your code retries or falls back. OpenAI flex processing · Gemini Flex inference.

Should you send easy requests to a smaller model?

Yes, once your evals show the small model holds up. Anthropic's pricing page says: "Choose Haiku for simple tasks, Sonnet for most production workloads, and Opus for the most complex reasoning." OpenAI recommends GPT-6 Luna "for cost-sensitive, high-volume workloads." OpenAI models. GPT-6 Luna, and Claude Haiku 5.5 on prompts up to 100,000 tokens, cost $0.10 in and $0.50 out, a twentieth of GPT-6.1 Sol and Claude Sonnet 5.5 and a hundredth of GPT-6 Astra and Claude Fable 5.1. Gemini 3.5 Flash-Lite costs $0.30 and $2.50, against $2 and $10 for Gemini 4 Argon through November 30, 2026, and $4 and $20 from December 1.

• A router picks the model before the call, from rules you write: task type, prompt length, the customer's plan. Rules cost no tokens, but someone has to keep them in step with the traffic.
• A cascade sends every request to the small model first and escalates the ones that fail a check, such as a JSON schema or a grader. Escalated requests pay twice, and with the small model at a twentieth of the price, the cascade wins while fewer than 95% escalate.

Amazon Bedrock sells routing as a feature: Intelligent Prompt Routing chooses between models of one family for $1 per 1,000 requests. At 300,000 requests a month that's $300, more than the $88.50 Claude Haiku 5.5 costs for the same traffic in the example below. Amazon Bedrock pricing.

How much can caching, batching, and routing save?

Take a company with two workloads on Claude Sonnet 5.5. A support assistant answers 10,000 questions a day: each request sends a 4,500-token system prompt plus 500 tokens from the customer and gets a 400-token answer. A nightly job summarizes 5,000 tickets a day: 1,500 tokens of fixed instructions plus a 2,500-token ticket in, 300 tokens out. Over 30 days, that's 450,000 requests.

Support assistant: 300,000 requests a month
  input 1,350M cached prefix + 150M new tokens, output 120M tokens
  start      1,500M x $2 + 120M x $10                      = $4,200.00
  cached     1,350M x $0.10 + 150M x $2 + 120M x $10       = $1,635.00
  Haiku      1,350M x $0.01 + 150M x $0.10 + 120M x $0.50  =    $88.50
  escalated  30% x $1,635.00 on Sonnet 5.5                 =   $490.50
  cascade    $88.50 + $490.50                              =   $579.00

Nightly summaries: 150,000 requests a month
  input 225M fixed prefix + 375M ticket tokens, output 45M tokens
  start      600M x $2 + 45M x $10                         = $1,650.00
  cached     225M x $0.10 + 375M x $2 + 45M x $10          = $1,222.50
  batch      600M x $1 + 45M x $5                          =   $825.00

Caching alone cuts the month by 51.2%, because the fixed prefix is 90% of the assistant's input and 37.5% of the job's. Batch takes the job from $1,222.50 to $825, and the Haiku-first cascade takes the assistant from $1,635 to $579. Per request, the month falls from 1.3 cents to 0.312 cents.

We made three assumptions. The caches stay warm, so the few cache writes are left out. The 30% escalation rate is ours; yours will come from your own traffic. And the batch row counts no cache hits, since Anthropic calls them best effort. If every prefix hit, the job would cost $611.25.

How do you cut output tokens?

Once the prefix is cached, output dominates the bill: in the example it's $1,200 of the assistant's $1,635. Cutting the average answer from 400 to 300 tokens saves 30 million output tokens a month, $300 on Claude Sonnet 5.5.

• Cap the length. OpenAI's max_output_tokens limits visible and reasoning tokens together, and a request that hits it can bill input and reasoning with no visible answer; OpenAI suggests reserving at least 25,000 tokens when you start with reasoning models. Anthropic's max_tokens caps thinking plus response text. OpenAI reasoning.
• Lower the reasoning effort, since all three providers bill reasoning as output. Anthropic's output_config.effort runs from low to max and defaults to medium on Opus 5.5 and Haiku 5.5 and to high elsewhere; OpenAI's reasoning.effort and Gemini's thinking_level do the same job. Claude effort · Gemini thinking.
• Ask for a label or a short JSON object instead of prose. OpenAI's Structured Outputs (text.format with json_schema and strict: true) and Anthropic's output_config.format hold the model to your schema; the first request with a new schema takes longer while it compiles. OpenAI Structured Outputs · Claude structured outputs.

How do you trim context and RAG chunks?

You pay for every input token, including the ones you sent last turn. OpenAI says that with previous_response_id, "all previous input tokens for responses in the chain are billed as input tokens." A 20-turn chat pays for its first turn 20 times, unless those turns hit the cache. OpenAI conversation state.

• Compact long conversations. OpenAI compacts through context_management or /responses/compact. Anthropic's context editing clears the oldest tool results once a prompt passes 100,000 input tokens by default; clearing breaks the cached prefix, so set clear_at_least high enough to be worth it. Claude context editing.
• Retrieve fewer chunks. OpenAI's vector stores cut files into 800-token chunks with a 400-token overlap by default, and a search returns 10 results unless you set max_num_results. Ten chunks add 8,000 input tokens to every question, $4,800 a month on 300,000 questions at $2 per million, and they change with each question, so they sit outside the cached prefix. Five chunks halve it. OpenAI retrieval.
• Keep tool definitions short and fixed. They count as input on every call.
• Count before you send, with OpenAI's POST /v1/responses/input_tokens, Anthropic's free /v1/messages/count_tokens, or Gemini's countTokens.

Do Bedrock and Google's Agent Platform cost less than the direct APIs?

No. On their global endpoints they charge the providers' own list prices, and regional and multi-region endpoints cost 10% more. Pick them for where you want the bill and the data.

• The direct APIs charge for location too: OpenAI adds 10% for regional processing on models released on or after March 5, 2026, and Anthropic's inference_geo: "us" multiplies every token price by 1.1.
• Bedrock lists no batch price for the Claude 5.5 models. Anthropic's own API and Agent Platform both list Claude Sonnet 5.5 batch at $1 / $5.
• Both bill through your AWS or Google Cloud account, so the spend lands on the same bill and under the same budgets as the rest of your cloud.
• Bedrock subscribes your account to Claude, and to the other models it sells through AWS Marketplace, on their first call. In Brazil, that's a second invoice: AWS SBL bills AWS services in reais, and AWS, Inc. invoices Marketplace purchases in US dollars. Amazon's own models don't go through Marketplace. Bedrock model access · AWS in Brazil.

How do you measure LLM spend per team, feature, and key?

Give every team, and every feature you want to price, its own billing unit, and read cost per unit every day. A 30% drop in the total doesn't tell you which feature got cheaper.

• OpenAI: a project per team and a key per feature. The Costs API groups daily cost by project_id or api_key_id, and the Usage API groups tokens by model, batch, and service tier, so you see how much traffic already runs at batch rates.
• Anthropic: a workspace per team, read through the Cost API by workspace. Amazon Bedrock: since April 9, 2026, a Cost and Usage Report 2.0 export with caller identity names the IAM user or role behind each call.

Track cost per request next to the total. In the example it fell from 1.3 cents to 0.312 cents.

Our guide How to Track OpenAI, Anthropic, and Bedrock Costs by Team has the setup and the API calls for each provider, and our glossary defines AI cost allocation.

What guardrails stop a runaway LLM bill?

A hard limit per team at each provider, plus caps in your own code. Our glossary covers cost guardrails in general.

• OpenAI: hard limits per organization and per project. A project over its limit gets a 429 error with the project_spend_limit_exceeded code, and OpenAI says "spend alerts do not enforce a cap." OpenAI spend limits.
• Anthropic: a monthly spend limit on each workspace except the Default Workspace, which takes none. Claude workspaces.
• Google Cloud: a monthly spend cap budget on one project and one service, such as the Gemini API or Agent Platform, pauses new usage at 100%. Enforcement isn't instant and overage bills as normal, so set the cap a little under your real limit. Spend cap budgets.
• AWS: a budget on the team's tag or account, plus Cost Anomaly Detection, both set up in our AWS cost guide.
• Your code: a token cap on every call, a ceiling on agent steps per task, and a rate limit per end user.

Common mistakes when cutting LLM costs

• A timestamp or the user's name at the top of the system prompt, which changes the prefix and misses the cache on every request.
• Tool definitions rebuilt per request, in a new order or with new wording. On Anthropic, that breaks the whole cache.
• A tight output cap on a reasoning model, which can bill input and reasoning tokens with no visible answer.
• Routing on price without evals. Past 95% escalation, a Haiku-first cascade costs more than Sonnet alone.
• A regional endpoint by default, at 10% more, with no data rule behind it.

Where does LevelFour fit?

LevelFour reads your OpenAI and Anthropic bills and the Bedrock lines on your AWS bill next to the rest of your cloud and SaaS spend: bills by model and key, usage by person and team. Virtual tags allocate them without retagging, so team=Payments can claim an AWS account, an OpenAI project, and an Anthropic workspace. Budgets and anomaly alerts post to Slack, Microsoft Teams, Jira, or any webhook. Cloud Savings ships the savings it finds on your cloud bill, applied for you or as a pull request, and Savings Audit confirms each one on the bill. AI spend comes with every plan, from $30 a month. See what LevelFour can save you in 5 minutes, with read-only access.

Frequently asked questions