← Back to Dashboard
1. Choosing Models by Workload2. A/B Testing and Rollout Gates

Choosing Models by Workload

📚 Model Selection Strategy11 min85 XP⌨ Hands-on lab

One Model Is Rarely Optimal

Bedrock hosts 100+ foundation models across Amazon, Anthropic, Meta, Mistral, DeepSeek, Moonshot AI (Kimi), MiniMax, Google DeepMind (Gemma), xAI (Grok), and OpenAI. Production systems route requests to different model tiers based on risk and required quality - paying frontier prices for trivial tasks is the most common AI budget leak.

A Practical Routing Matrix

WorkloadRisk/complexityModel tierWhy
Summarization, tagging, classificationLowAmazon Nova Micro/Lite, Claude Haiku classFast, cheap, quality ceiling irrelevant here
Customer-facing drafting, RAG answersMediumNova Pro, Claude Sonnet classBalanced quality/cost at volume
Complex reasoning, agentic coding, multi-step plansHighClaude Opus/Sonnet frontier, Nova Premier, OpenAI frontier via Responses APICapability is the bottleneck, not price
Open-weight / self-hostable mandatesVariesDeepSeek, Meta Llama, Mistral, GPT-OSSLicensing and portability requirements
Regulated decisions (credit, medical, legal)CriticalStrongest tier + guardrails + human gatesAudit trail matters more than cost

Routing Mechanics

  • Static routing - endpoint/task type maps to a model in config. Simple, predictable, covers most systems.
  • Dynamic routing - a cheap classifier scores request complexity, then dispatches. Adds cost/latency; justify with traffic diversity.
  • Escalation routing - try the cheap tier; if confidence/validation fails, retry on the premium tier. Great when most requests are easy.
  • Intelligent Prompt Routing (Bedrock feature) - managed routing between models in a family based on predicted response quality per prompt.
Keep the matrix versioned: the model space shifts fast - review your routing matrix quarterly with fresh eval scores. Last year's "best value" tier is often this year's overpriced one.
⌨ HANDS-ON LABBuild a Routing Matrix
⭐ +150 XP

Map support, analytics, coding, and compliance tasks to model families.

1Create a workload-to-model matrix file.
2Add an escalation rule.
lab-sandbox — simulated environment
INFINITY LAB SANDBOX v2.6 — simulated shell
Type the command for the current objective. Helpers: "hint", "solution", "clear".
$
OBJECTIVE 1 / 2 — type "hint" if stuck
🧪 Knowledge Check
Press 1-4 to select1 of 2
Why do mature teams use model routing?
Because one model is always best
To optimize quality-latency-cost across workloads
To avoid testing
To remove telemetry