Prompt caching

Prompt caching is automatic. No configuration, no extra parameters, and no extra charge: when the beginning of your prompt matches a recent request, the cached portion of your input is billed at a reduced rate. Only input tokens cache; output is always billed in full. To benefit, keep reusable content (system prompts, context) at the START of your prompt; even a single changed character invalidates the cached prefix. Cache entries expire after a period of inactivity, and hits are best-effort, never guaranteed. Savings are passed through to your billing automatically.

How to get cache hits

The cache matches on the prefix of your prompt. To maximize hits:

• Put stable content (system prompt, instructions, reference documents, few-shot examples) at the beginning of your messages.
• Put the variable part (the user's latest question) at the end.
• Keep the stable prefix byte-for-byte identical across calls. A single changed character invalidates the cached prefix from that point on.

Because only input tokens cache, workloads with a large fixed context and a small changing tail (chatbots with long system prompts, RAG over a fixed document, repeated classification) benefit the most.

Seeing your savings

Cache savings are applied to your bill automatically; there's nothing to reconcile. Your Usage page reports a per-model cache rate (cached input tokens as a share of total input tokens) so you can see how well your prompts are caching.

Cache behavior can vary with routing and load. Hits are best-effort and never guaranteed; design your cost estimates around the standard input rate and treat cache savings as upside.