[ ABORT TO HUD ]
SEQ. 1
SEQ. 2
SEQ. 3
SEQ. 4
Scaling & Cost Optimization
Running Agents at Scale Without Going Broke
A single agent task might cost $0.05. At 10,000 tasks/day, that's $500/day or $180K/year. Cost optimization isn't optional - it's survival.
The Cost Optimization Toolkit
| Technique | Savings | Complexity | How It Works |
|---|---|---|---|
| Model Routing | 50-80% | Medium | Use Haiku for simple tasks, Sonnet for complex, Opus for critical |
| Prompt Caching | 60-90% | Low | Cache static prefixes (Anthropic reduces cached token cost by 90%) |
| Tool Result Caching | 20-50% | Low | Cache identical tool calls (same query = cached result) |
| Batch Processing | 50% | Low | Use Batch API for non-real-time tasks (Anthropic: 50% off) |
| Context Compaction | 40-70% | Medium | Summarize old messages, keep recent ones |
| Iteration Caps | Variable | Low | Hard limit on agent loops (prevent infinite spinning) |
Model Routing Architecture
// Route tasks to the cheapest capable model:
async function routeToModel(task) {
const complexity = await classifyComplexity(task); // Use Haiku to classify
switch (complexity) {
case "simple": return { model: "haiku", maxTokens: 1024 }; // ~$0.001
case "moderate": return { model: "sonnet", maxTokens: 4096 }; // ~$0.01
case "complex": return { model: "opus", maxTokens: 8192 }; // ~$0.10
}
}
Production Cost Monitoring
- Per-task budgets: Set a hard dollar limit per agent run. Kill the agent if exceeded.
- Daily burn rate alerts: Get notified if daily cost exceeds 2x the average.
- Per-model dashboards: Track which model is consuming the most budget.
- Anomaly detection: Flag tasks that cost 10x the median as potential infinite loops.
💰 Reality Check: The biggest cost savings come from model routing (use Haiku for 70% of tasks) and prompt caching (90% savings on cached tokens). Implement these two first before anything else.
⌨ HANDS-ON LABImplement Model Tier Routing for Cost Reduction
⭐ +150 XPRoute lightweight classification tasks to fast small models (Phi-4 / GPT-4o-mini) and complex reasoning tasks to frontier models.
1Run dynamic model tier router benchmark.
OBJECTIVE 1 / 1 — type "hint" if stuck
SYNAPSE VERIFICATION
QUERY 1 // 1
What is the most impactful cost optimization technique for agent systems?
Using longer prompts
Model routing (using cheaper models for simple tasks) combined with prompt caching
Running fewer agents
Using GPT-3.5 for everything