How to Reduce LLM API Costs by Up to 90%: A Practical Playbook

LLM bills grow in silence. No single request is expensive, nothing errors, and then finance asks why the API line tripled. The good news: most teams can cut 30–60% without touching product behavior, because the biggest levers are billing mechanics, not model quality trade-offs. Here are the seven that matter, ranked by payoff-per-effort — with real numbers where we have them.

1. Measure before you optimize (one afternoon)

You cannot fix what you can't see, and provider dashboards show spend, not waste. The two numbers that drive everything below: your cache hit rate (what share of input tokens were billed at the cached price) and your wasted spend (tokens that should have been cached but weren't). A metering proxy shows both per key and per model with a base-URL swap; even an observe-only mode that changes nothing is enough to find the leaks.

2. Fix prompt caching — the biggest lever (a day)

Every major provider bills repeated prompt prefixes at ~10% of list price. Since agents and chatbots resend system prompts, tool schemas and history on every call, 60–90% of input tokens are usually re-reads. Yet real hit rates disappoint, for reasons that never throw errors: caches expire after ~5 idle minutes, one unstable character (a timestamp!) breaks prefix matching, and on Anthropic nothing happens at all until someone sets cache_control breakpoints. The fix checklist:

• Put stable content first (system prompt, tools), variable content last.
• Remove timestamps, request IDs and randomized examples from the prefix.
• On Anthropic, set breakpoints; on GPT-5.6+, add an explicit breakpoint and a stable prompt_cache_key (SDK defaults measured 0% cross-request hits).
• If your traffic has idle gaps longer than the TTL, warm the cache through them — when the ping cost is below the re-write cost — or move to the 1-hour TTL.

Measured on 10,000+ real billed calls (method and raw logs public): 67% net savings on sparse support traffic, 89% on a shared-prefix batch. This is what Caching.ai automates end-to-end; the mechanics are in our prompt-caching guide if you'd rather build it.

3. Right-size the model (a day, ongoing)

The price gap between a frontier model and its smaller sibling is typically 5–20x, and a large share of production requests — classification, extraction, routing, summarization — don't need the frontier model. Route by task: keep the expensive model for the hard 20%, send the mechanical 80% to a small model, and A/B the quality delta instead of assuming it. This multiplies with caching: cheap model × cached prefix compounds.

4. Trim what you send and cap what you get back (days)

• Prune dead prompt weight: obsolete instructions, redundant few-shots, tools the model never calls. Every token rides on every request forever.
• Summarize or window long conversation history instead of resending all of it.
• Set max_tokens honestly — output tokens cost 3–5x input tokens on most models, and unbounded answers are pure downside.
• In RAG, rerank and send 3 tight chunks instead of 10 loose ones.

5. Use batch APIs for anything async (hours)

Anthropic and OpenAI both run batch tiers at ~50% off with results within 24 hours. Nightly classification jobs, embeddings backfills, evals, report generation — anything that doesn't need an interactive answer has no business paying the real-time price. Batches with a shared prefix also cache superbly (our 300-call batch benchmark: 89% cheaper than SDK defaults).

6. Add a response cache for identical repeats (a day)

Distinct from prompt caching: a response cache returns a stored answer without calling the model at all — 100% savings on exact repeats, with a staleness risk to manage. Worth it when many users ask literally the same thing (FAQ bots, shared dashboards, test suites). Gateways like Helicone, Portkey, LiteLLM or Cloudflare AI Gateway ship this. How the two caches differ and combine: prompt caching vs semantic caching.

7. Put guardrails on spend (an afternoon)

Budgets and alerts per key, team and feature; anomaly alerts for the 3 a.m. retry loop that burns a week's budget; and a monthly review of cost per feature. Boring, and the only reason the other six stay fixed.

The order that works

Measure (1) → caching (2) → right-size (3) captures most of the win in the first week, without touching product behavior. Then trim (4), batch (5), response-cache (6) and guard (7). If you want steps 1, 2 and 7 done for you: Caching.ai is a one-line proxy that meters everything, fixes the cache, and charges 20% of what it verifiably saves — nothing if it saves nothing.