Choosing Models by Workload — Model Selection Strategy
← Back to Dashboard
1. Choosing Models by Workload2. A/B Testing and Rollout Gates
Choosing Models by Workload
📚 Model Selection Strategy⏱ 11 min⭐ 85 XP⌨ Hands-on lab
One Model Is Rarely Optimal
Bedrock hosts 100+ foundation models across Amazon, Anthropic, Meta, Mistral, DeepSeek, Moonshot AI (Kimi), MiniMax, Google DeepMind (Gemma), xAI (Grok), and OpenAI. Production systems route requests to different model tiers based on risk and required quality - paying frontier prices for trivial tasks is the most common AI budget leak.
A Practical Routing Matrix
| Workload | Risk/complexity | Model tier | Why |
|---|---|---|---|
| Summarization, tagging, classification, voice turn-taking | Low | Amazon Nova Micro / Nova 2 Lite, Claude Haiku 5.5, Nova 2.5 Sonic (voice) | Sub-second latency, low cost, configurable light thinking |
| Customer-facing drafting, grounded RAG, UI generation | Medium | Amazon Nova Pro / Nova 2 Pro, Claude Sonnet 5.5, Claude Fable 5.1 | Balanced reasoning, tool accuracy, and cost at volume |
| Complex reasoning, agentic coding, multi-step architecture | High | Claude Opus 5.5, Amazon Nova Premier, DeepSeek-R1 | Frontier reasoning depth is the bottleneck, not token price |
| Open-weight / portability mandates | Varies | Meta Llama 4 (Scout 17B, Maverick 17B), DeepSeek-V3.2, Mistral Large, GPT-OSS | Open-weight governance and cross-cloud portability |
| Regulated decisions (credit, medical, legal) | Critical | Strongest tier + Automated Reasoning Guardrails + human gates | Mathematical policy proof and audit trails outweigh token cost |
Routing Mechanics
- Static routing - endpoint/task type maps to a model in config. Simple, predictable, covers most systems.
- Dynamic routing - a cheap classifier scores request complexity, then dispatches. Adds cost/latency; justify with traffic diversity.
- Escalation routing - try the cheap tier; if confidence/validation fails, retry on the premium tier. Great when most requests are easy.
- Intelligent Prompt Routing (Bedrock feature) - serverless single-endpoint routing between models in the same family (e.g. Haiku ↔ Sonnet or Nova Lite ↔ Nova Pro) based on predicted prompt complexity, cutting spend by up to ~30%+ without sacrificing quality.
Keep the matrix versioned: the model space shifts fast - review your routing matrix quarterly with fresh eval scores. Last year's "best value" tier is often this year's overpriced one.
⌨ HANDS-ON LABBuild a Routing Matrix
⭐ +150 XPMap support, analytics, coding, and compliance tasks to model families.
1Create a workload-to-model matrix file.
2Add an escalation rule.
OBJECTIVE 1 / 2 — type "hint" if stuck
🧪 Knowledge Check
Press 1-4 to select1 of 2
Why do mature teams use model routing?
Because one model is always best
To optimize quality-latency-cost across workloads
To avoid testing
To remove telemetry