Skip to main content

Prompt Caching & Cost Control — Production & Cost Optimization

[ ABORT TO HUD ]
SEQ. 1
SEQ. 2
SEQ. 3

Prompt Caching & Cost Control

💰 Production & Cost Optimization 12 min 250 BASE XP⌨ HANDS-ON LAB

Automatic Prompt Caching

The Responses API and Chat Completions automatically cache repeated prompt prefixes of 1,024 tokens or more. If your requests share a long developer/system prompt or tool catalog, subsequent requests pay dramatically reduced input token costs for cached reads.

How It Works (2026 Rates & Multipliers)

  • The API detects when multiple requests share identical prefix content (minimum 1,024 tokens)
  • GPT-6 Cache Discount (90%–95% off): Cached reads cost $1.00/M on gpt-6-astra ($10 standard), $0.10/M on gpt-6.1-sol ($2 standard), $0.05/M on gpt-6.1-terra ($1 standard), and $0.01/M on gpt-6-luna ($0.10 standard); earlier generations received 50%–75% off
  • Long-Context & Speed Modes: Prompts exceeding 272,000 input tokens incur a 2x input/cached multiplier (1.5x output), while Fast mode (renamed from Priority Processing on July 30, 2026) doubles rates for low-latency SLAs
  • Cache entries typically persist for 5–10 minutes of inactivity (with configurable retention options on supported tiers)

Maximizing Cache Hits

  1. Put static content first - system prompts, instructions, examples
  2. Put dynamic content last - user queries, variable data
  3. Keep system prompts identical across requests

Rate Limits & Tiers

TierRPMTPMHow to Upgrade
Free340K-
Tier 1500200K$5 paid
Tier 25,0002M$50+ paid, 7+ days
Tier 35,00010M$100+ paid, 7+ days
Tier 4+10,00050M+$250+ paid, 14+ days
🎯 Cost Formula: Total cost = (Uncached input tokens * input_rate) + (Cached tokens * cached_rate) + (Output + reasoning tokens * output_rate). With 90%–95% cached input savings on GPT-6 models for prefixes of 1,024+ tokens, keeping prefixes deterministic is critical.
⌨ HANDS-ON LABMeasure Automatic Prompt Caching Savings
⭐ +150 XP

OpenAI automatically caches prefixes of 1,024+ tokens (up to 90%–95% off on GPT-6 models). Inspect prompt_tokens_details.cached_tokens.

1POST a large context prompt to /v1/chat/completions and verify prompt token caching in the response usage.
lab-sandbox — simulated environment
INFINITY LAB SANDBOX v2.6 — simulated shell
Type the command for the current objective. Helpers: "hint", "solution", "clear".
$
OBJECTIVE 1 / 1 — type "hint" if stuck
SYNAPSE VERIFICATION
QUERY 1 // 3
How do you enable prompt caching in the Responses API?
Set cache: true
It's automatic for matching prefixes of 1,024+ tokens - no configuration needed
Use a special header
Pay for a premium plan