Token Economics and Caching — Cost Engineering

Cost Comes from Repetition and Over-Context

Large context payloads and repeated prompts dominate LLM cost curves. Because input tokens are billed on every request, a bloated system prompt is not a one-time cost - it is a tax on every single call, forever.

Where the Money Actually Goes

# A "small" inefficiency at scale
system prompt: 2,800 tokens (could be 900)
history sent:  6,000 tokens (relevant: ~1,500)
per-request waste: ~6,400 input tokens
× 500k requests/month = 3.2B wasted tokens/month
→ at typical mid-tier input pricing, thousands of dollars for zero quality gain

The Optimization Ladder (Cheapest First)

Prompt Caching Mechanics on Bedrock (cachePoint)

In the Converse API, place up to 4 cachePoint blocks right after static content in system, tools, or messages (default 5-minute sliding TTL, or optional "ttl": "1h" on supported Claude 5.x and Amazon Nova models):

response = client.converse(
    modelId="us.anthropic.claude-sonnet-5-5",
    system=[
        {"text": large_policy_handbook},          # must meet model min token threshold (e.g. 1,024+ tokens)
        {"cachePoint": {"type": "default"}},     # optional: {"type": "default", "ttl": "1h"}
    ],
    messages=[{"role": "user", "content": [{"text": user_question}]}],
)
usage = response["usage"]  # inspect cacheWriteInputTokens (miss) vs cacheReadInputTokens (hit)

• Use compact system prompts - every sentence should earn its tokens.
• Trim irrelevant history - summarise old turns instead of resending them.
• Reuse cached context for repetitive workflows where supported.