Quickstart

Caching.ai is a drop-in proxy for Anthropic, OpenAI, Gemini, and Grok APIs. Integration is a base-URL swap — your code, SDK, and request shapes stay exactly the same.

1. Create a key
Sign up, open Console → API Keys, and register your own Anthropic / OpenAI / Gemini / Grok keys once for your account — they're encrypted with AES-256-GCM and used only to forward your requests. Then create a Caching.ai key (ck_…); every key you create uses your account's provider keys automatically. Prefer a zero-touch trial? Create the key in observe-only mode — full analytics, no request modification — and turn on optimizations later.

2. Swap the base URL

Point your SDK at the proxy and use your Caching.ai key:

# environment variables — works with every Anthropic SDK
export ANTHROPIC_BASE_URL="https://proxy.caching.ai"
export ANTHROPIC_API_KEY="ck_your_key_here"

# python
import anthropic
client = anthropic.Anthropic(
    base_url="https://proxy.caching.ai",
    api_key="ck_your_key_here",
)

// typescript
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic({
  baseURL: "https://proxy.caching.ai",
  apiKey: "ck_your_key_here",
});

Also on OpenAI, Gemini & Grok
With an OpenAI, Gemini, or Grok key registered to your account, just point those SDKs at the same proxy. OpenAI caches stable 1,024+ token prefixes automatically and Gemini bills context caches at a fraction of the input price — we meter it all, price the savings, and warn you when an unstable prefix is silently disabling the cache. Grok (xAI) speaks the OpenAI wire format: send it to the same endpoint and grok-* model names route automatically, and we inject a stable x-grok-conv-id cache-routing header to lift hit rates — never touching one you send yourself.
```
# openai — same base URL swap
from openai import OpenAI
client = OpenAI(base_url="https://proxy.caching.ai/v1", api_key="ck_your_key_here")
```
```
# gemini (google-genai)
from google import genai
client = genai.Client(
    api_key="ck_your_key_here",
    http_options={"base_url": "https://proxy.caching.ai"},
)
```
```
# grok (xAI) — OpenAI-compatible, routed by the grok-* model name
from openai import OpenAI
client = OpenAI(base_url="https://proxy.caching.ai/v1", api_key="ck_your_key_here")
client.chat.completions.create(model="grok-4.5", messages=[...])
```
For streaming OpenAI requests we add stream_options.include_usage when it's missing so usage is still metered — your stream content is untouched. /v1/completions and /v1/embeddings pass through with the same metering.

Connect any tool — copy-paste recipes

Every popular AI tool can point at the proxy with one base-URL change. Pick yours below; the ck_ key goes wherever the provider key used to go.

Fastest: let your agent set it up

Paste this into Claude Code, Codex, or any coding agent (replace the key). It follows our setup guide with backups, diffs, and a final verification call.

Set up the caching.ai proxy for my AI tools.
1) Fetch https://caching.ai/agent-setup.md and follow it exactly, including its safety rules.
2) My caching.ai key: ck_your_key_here
3) Detect which supported tools I use, confirm the list with me, back up every config you touch, apply the changes, then run the verification step and show me the result.

Claude Code

Anthropic

Officially supported via settings or env vars. First interactive run asks you to approve the custom key once. Then add the Cache Keeper plugin (below) and your cache stays warm between sessions automatically.

# ~/.claude/settings.json
{
  "env": {
    "ANTHROPIC_BASE_URL": "https://proxy.caching.ai",
    "ANTHROPIC_AUTH_TOKEN": "ck_your_key_here"
  }
}
# or just: export ANTHROPIC_BASE_URL="https://proxy.caching.ai"
#          export ANTHROPIC_AUTH_TOKEN="ck_your_key_here"

# recommended: the Cache Keeper plugin (auto warm-hold — see below)
/plugin marketplace add caching-ai/caching.ai
/plugin install cache@caching-ai

OpenAI Codex CLI

OpenAI

Codex speaks the Responses API — the proxy supports it natively, so this just works.

# ~/.codex/config.toml
model_provider = "caching"

[model_providers.caching]
name = "Caching.ai proxy"
base_url = "https://proxy.caching.ai/v1"
env_key = "CACHING_API_KEY"     # export CACHING_API_KEY=ck_your_key_here
wire_api = "responses"

Cline / Roo Code (VS Code)

Anthropic · OpenAI

Both extensions have an official custom-base-URL field per provider.

# Settings → API Provider
#   Anthropic       → check "Use custom base URL" → https://proxy.caching.ai
#   OpenAI Compatible → Base URL https://proxy.caching.ai/v1
# API Key: ck_your_key_here

Continue (VS Code / JetBrains)

Anthropic · OpenAI

# ~/.continue/config.yaml
models:
  - name: claude-via-caching
    provider: anthropic
    model: claude-sonnet-4-5
    apiBase: https://proxy.caching.ai
    apiKey: ck_your_key_here
  - name: gpt-via-caching
    provider: openai
    model: gpt-4o
    apiBase: https://proxy.caching.ai/v1
    apiKey: ck_your_key_here

Aider

OpenAI · Anthropic

OpenAI-path models need the openai/ prefix. For Anthropic, pass the ROOT url — litellm appends /v1/messages by itself.

# OpenAI-path models
export OPENAI_API_BASE=https://proxy.caching.ai/v1
export OPENAI_API_KEY=ck_your_key_here
aider --model openai/gpt-4o

# Anthropic-path models (root URL — litellm appends /v1/messages)
aider --anthropic-api-key ck_your_key_here \
      --set-env ANTHROPIC_API_BASE=https://proxy.caching.ai \
      --model claude-sonnet-4-5

Gemini CLI

Gemini

Applies in gemini-api-key auth mode (not Google login). Restart the CLI after changing.

# ~/.gemini/.env
GEMINI_API_KEY=ck_your_key_here
GOOGLE_GEMINI_BASE_URL=https://proxy.caching.ai

LangChain

Anthropic · OpenAI

# python
from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic
llm = ChatOpenAI(model="gpt-4o", base_url="https://proxy.caching.ai/v1", api_key="ck_your_key_here")
claude = ChatAnthropic(model="claude-sonnet-4-5", base_url="https://proxy.caching.ai", api_key="ck_your_key_here")

// typescript
const llm = new ChatOpenAI({ apiKey: "ck_...", configuration: { baseURL: "https://proxy.caching.ai/v1" } });
const claude = new ChatAnthropic({ apiKey: "ck_...", anthropicApiUrl: "https://proxy.caching.ai" });

LlamaIndex

Anthropic · OpenAI

OpenAILike avoids strict model-name validation; the Anthropic class takes base_url directly.

from llama_index.llms.openai_like import OpenAILike
from llama_index.llms.anthropic import Anthropic
llm = OpenAILike(model="gpt-4o", api_base="https://proxy.caching.ai/v1",
                 api_key="ck_your_key_here", is_chat_model=True)
claude = Anthropic(model="claude-sonnet-4-5", base_url="https://proxy.caching.ai",
                   api_key="ck_your_key_here")

Vercel AI SDK

Anthropic · OpenAI · Gemini

Note the differing paths: /v1 for OpenAI and Anthropic providers, /v1beta for Google.

import { createOpenAI } from "@ai-sdk/openai";
import { createAnthropic } from "@ai-sdk/anthropic";
import { createGoogle } from "@ai-sdk/google";

const openai = createOpenAI({ baseURL: "https://proxy.caching.ai/v1", apiKey: "ck_..." });
const anthropic = createAnthropic({ baseURL: "https://proxy.caching.ai/v1", apiKey: "ck_..." });
const google = createGoogle({ baseURL: "https://proxy.caching.ai/v1beta", apiKey: "ck_..." });

OpenAI Agents SDK

OpenAI

Route the agent through a chat-completions client; disable tracing uploads (they would need a real sk- key).

from openai import AsyncOpenAI
from agents import Agent, OpenAIChatCompletionsModel, set_tracing_disabled

set_tracing_disabled(True)  # tracing uploads would need a real sk- key
client = AsyncOpenAI(api_key="ck_your_key_here", base_url="https://proxy.caching.ai/v1")
agent = Agent(name="Helper",
              model=OpenAIChatCompletionsModel(model="gpt-4o", openai_client=client))

Cursor

OpenAI

Works, with limits set by Cursor itself: the base-URL override is a UI-only feature, requests route via Cursor servers, and Tab autocomplete stays on their models.

# Cursor Settings → Models → OpenAI API Key: ck_your_key_here
#   → "Override OpenAI Base URL": https://proxy.caching.ai/v1

Windsurf currently has no custom base-URL option in its BYOK settings, so it cannot route through any proxy. The proxy accepts all three auth styles (Authorization: Bearer, x-api-key, x-goog-api-key) — whichever your tool sends.

3. Watch the dashboard
Every request now reports its cache hit rate, estimated savings, and estimated waste to your dashboard — plus end-to-end latency percentiles (P50/P95), a weekday-by-hour traffic heatmap, a cache health grade, projected monthly savings, and a raw CSV export, over a 7 / 30 / 90-day window. Streaming (SSE) is passed through without buffering; usage is read from the stream as it flies by.
What the proxy does
- Analytics — Records usage tokens, latency, and status per request — prompt and response bodies are never stored. The dashboard adds latency percentiles, a traffic heatmap, a cache health grade, savings projections, and CSV export.
- Auto cache_control — If a request has no cache breakpoints at all, we add them to the last system block and last tool — with the cache TTL you pick per key (5 minutes, or 1 hour at a 2× write premium). Requests that already use caching are never modified.
- Cache-breaker detection — If your system prompt or tool list hashes differently on every request, we flag it on the dashboard with the likely cause.
- Cache Warmer (opt-in) — Re-warms your prefix with max_tokens: 1 pings while re-use is still economical — 62.5 minutes on the 5m TTL, about 20 hours on Anthropic's 1h TTL — within a daily budget you control. Anthropic-only, by measurement: other providers hold their caches upstream, so pings there only burn budget. Stepping away? Type cai:hold 2h (or just “keep my cache warm for 2 hours”) as a chat message — the proxy answers it itself, nothing reaches the AI, it pre-warms the conversation you asked from (so the cache is live the moment you ask) and holds warming for that window; long holds are served as a single 1-hour cache write instead of a ping stream — whichever is cheaper for the window you asked for.
- Provider cache tuning — Per key: Anthropic cache TTL (5m/1h), GPT-5.6+ cache restore (the new models only match at breakpoints, so naive shared prefixes get 0% cross-request hits — we inject an explicit breakpoint and a stable prompt_cache_key to bring them back), and stable conversation routing via x-grok-conv-id on Grok. Gemini 2.5+ caches implicitly with no knobs; we meter it, and explicit Gemini caches are on the roadmap. On the hosted cloud, auto-tune mode learns each key's call pattern and picks these settings for you.
- Weekly savings report — One email per week, only for weeks you actually had traffic: dollars saved, dollars still leaking, and the one prompt fix that would help most.
- Budget alert — If a key's warming spend reaches its daily budget, warming rests until midnight UTC and you get one email about it — never more than one per day.
Hold the cache with a chat message
Stepping away mid-session? Ask the proxy in plain words. Send one short message through any SDK — the proxy intercepts it, replies instantly, and never forwards it to the AI, so it costs zero model tokens. It also pre-warms the very conversation you asked from, so the cache is already live when you walk away:
```
# a chat message, through any SDK — the proxy answers it, the AI never sees it
"keep my cache warm for 2 hours"
"캐시 2시간 지켜줘" · "キャッシュを2時間保温して" · "帮我保温缓存 2 小时"
"mantén mi caché caliente 2 horas" · "halte meinen Cache 2 Stunden warm"
"держи кэш тёплым 2 часа" · "giữ cache nóng trong 2 giờ"
cai:warm 45m   # explicit command — works anywhere, any language

# → 🔥 Pre-warmed this conversation right now (18,204 tokens cached) and
#   holding it warm for 2 hours.   (answered at the proxy, 0 model tokens)
```
- Duration: 2 h by default, from 5 minutes up to 12 hours — “30 minutes”, “3 hours”, and cai:warm 45m all work
- Understood in 16 languages — Korean, English, Japanese, Chinese, Spanish, Portuguese, French, German, Italian, Russian, Turkish, Vietnamese, Indonesian, Hindi, Thai and Arabic — and answered in the one you used, starting with 🔥
- Pre-warms on the spot: the prefix of the request carrying the command is written to the provider once, and the reply quotes the token count the provider reports cached — never an estimate. Nothing saved yet? No problem: the command itself is the first warm request
- A hold of 90 minutes or more is pre-warmed as a single 1-hour cache write instead of a ping stream — cheaper for that window
- Available on every path: Anthropic Messages, OpenAI chat & responses (Codex), Gemini — and Grok via the chat path. Warming itself is Anthropic-only by measurement, and on the other paths the reply says so
- The Cache Warmer must be on for the key and your daily warming budget still applies; the console shows a “Warm hold active · until HH:MM” badge while it lasts
The message has to be short (≤ 80 characters) and clearly about the cache — anything that looks like a real coding request goes straight to the model, untouched. The pre-warm write is metered as a warming ping inside your daily budget, and repeating the command within a minute never pays for a second write.
Claude Code: fully automatic with the Cache Keeper plugin
Install once and your cache survives lunch, meetings, and long breaks: after every turn the plugin silently renews a warm hold (2 hours by default), so coming back never means paying a cold-cache re-write. The hold command is answered at the proxy — zero tokens — and warming spend always stays within the key's daily budget.
```
# inside Claude Code — install once
/plugin marketplace add caching-ai/caching.ai
/plugin install cache@caching-ai

# not routed through the proxy yet? the plugin sets everything up
/cache:setup

# manual controls
/cache:hold 8h     # hold the cache warm for a longer break
/cache:status      # routing, key, auto-hold, last hold
```
- Runs only in sessions actually routed through the proxy on an API key — anywhere else the plugin quietly does nothing and costs nothing
- Pick your window: CACHING_AUTO_HOLD in the settings env block (“4h”, up to 12h; “off” disables) — or /cache:hold 8h any time before a longer break
- /cache:setup connects a fresh machine end-to-end: settings backed up, env merged, then a real verification call
- Anthropic-only, by measurement — other providers hold their caches upstream long enough that pings would only burn budget, so Codex needs no warming in the first place
Open source like everything else — the plugin lives in the same GitHub repo (claude-plugin/). Self-hosting? Point it at your own proxy with CACHING_PROXY_URL.
Sub-tenants: one key, many end-customers
Serving many end-customers through a single ck_ key — a platform, an agency, an internal AI gateway? Tag each request with the customer it belongs to and the proxy keeps them apart: per-tenant cache policy, per-tenant usage and savings attribution, and per-tenant warm slots. There is no provisioning step — the header alone is enough:
```
# tag each request with the end-customer it belongs to — the headers alone work
curl https://proxy.caching.ai/v1/messages \
  -H "content-type: application/json" \
  -H "x-api-key: ck_your_key_here" \
  -H "anthropic-version: 2023-06-01" \
  -H "X-Cache-Tenant: acme-corp" \
  -H "X-Cache-Warm-Slot: user-42" \
  -H "X-Cache-Keepalive: on" \
  -d '{
    "model": "claude-sonnet-4-5",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "..."}]
  }'
```
- X-Cache-Tenant — which end-customer this request belongs to (1–120 chars: letters, digits, . _ : -). Unlocks policy resolution, attribution, and warm slots for that tenant
- X-Cache-Warm-Slot — a warm slot inside the tenant, e.g. one per end-user. Each slot keeps that user's latest prefix warm; slots are pruned to the most recent 16 by default (keepalive_max_slots, 1–128 per tenant)
- X-Cache-Keepalive: on|off, X-Cache-Injection: on|off, X-Cache-Ttl: 5m|1h — per-request overrides. A header beats the tenant policy, which beats the key setting
Manage tenants with the key itself
The management API authenticates with your ck_ key (x-api-key or Authorization: Bearer) — no console round-trip. GET /admin/v1/tenants lists your tenants; GET/PUT/DELETE /admin/v1/tenants/{tenant} reads, updates, or removes one. PUT is a partial upsert of auto_cache_control, keepalive_enabled, keepalive_budget_usd_daily, anthropic_cache_ttl, and keepalive_max_slots — set null to clear a field back to inheriting the key's setting. DELETE offboards the tenant: its policy row is dropped and its warming stops. POST /admin/v1/tenants/{tenant}/hold ({slot, hold_ms}) warm-holds that slot's saved conversations without a chat round-trip — the platform equivalent of an end user's "keep my cache warm" message; hold_ms=0 releases it. You can also attach X-Cache-Hold-Ms to a normal request to capture and hold that request's prefix in one shot — an explicit action that outranks X-Cache-Keepalive: off. GET /admin/v1/tenants/{tenant}/stats returns requests, warming pings, tokens, cache reads and writes, cost, and savings:
```
# per-tenant policy — partial upsert; null clears a field back to inherit
curl -X PUT https://proxy.caching.ai/admin/v1/tenants/acme-corp \
  -H "Authorization: Bearer ck_your_key_here" \
  -H "content-type: application/json" \
  -d '{"keepalive_enabled": true, "keepalive_budget_usd_daily": 1.0}'

# usage & savings attribution for one tenant
curl "https://proxy.caching.ai/admin/v1/tenants/acme-corp/stats?days=7" \
  -H "Authorization: Bearer ck_your_key_here"
```
Budgets stack: a tenant's keepalive_budget_usd_daily caps that tenant's warming pings, and the key's daily budget still caps the whole key; warming stays Anthropic-only, as everywhere else. Enterprise: if your Anthropic traffic runs through your own gateway, GET/PUT/DELETE /admin/v1/gateway sets a per-key upstream from an operator-approved allowlist — warming pings follow the gateway and replay your custom x-* headers, so gateway-side routing and attribution keep working. Contact support to have your gateway approved.

Verify it works

curl https://proxy.caching.ai/v1/messages \
  -H "content-type: application/json" \
  -H "x-api-key: ck_your_key_here" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-opus-4-8",
    "max_tokens": 128,
    "messages": [{"role": "user", "content": "Say hi"}]
  }'

The response is byte-identical to your provider's. Repeat the same call twice and check usage.cache_read_input_tokens — and your dashboard.

Self-hosting? Ops endpoints (healthz · readyz · Prometheus metrics) and every environment variable are covered in the GitHub README. github.com/caching-ai/caching.ai

Quickstart

1. Create a key

2. Swap the base URL

Also on OpenAI, Gemini & Grok

Connect any tool — copy-paste recipes

Claude Code

OpenAI Codex CLI

Cline / Roo Code (VS Code)

Continue (VS Code / JetBrains)

Aider

Gemini CLI

LangChain

LlamaIndex

Vercel AI SDK

OpenAI Agents SDK

Cursor

3. Watch the dashboard

What the proxy does

Hold the cache with a chat message

Claude Code: fully automatic with the Cache Keeper plugin

Sub-tenants: one key, many end-customers

Manage tenants with the key itself

Verify it works