Skip to main content

Choosing Models by Workload — Model Selection Strategy

← Back to Dashboard
1. Choosing Models by Workload2. A/B Testing and Rollout Gates

Choosing Models by Workload

📚 Model Selection Strategy⏱ 11 min⭐ 85 XP⌨ Hands-on lab

One Model Is Rarely Optimal

Bedrock hosts 100+ foundation models across Amazon, Anthropic, Meta, Mistral, DeepSeek, Moonshot AI (Kimi), MiniMax, Google DeepMind (Gemma), xAI (Grok), and OpenAI. Production systems route requests to different model tiers based on risk and required quality - paying frontier prices for trivial tasks is the most common AI budget leak.

A Practical Routing Matrix

WorkloadRisk/complexityModel tierWhy
Summarization, tagging, classification, voice turn-takingLowAmazon Nova Micro / Nova 2 Lite, Claude Haiku 5.5, Nova 2.5 Sonic (voice)Sub-second latency, low cost, configurable light thinking
Customer-facing drafting, grounded RAG, UI generationMediumAmazon Nova Pro / Nova 2 Pro, Claude Sonnet 5.5, Claude Fable 5.1Balanced reasoning, tool accuracy, and cost at volume
Complex reasoning, agentic coding, multi-step architectureHighClaude Opus 5.5, Amazon Nova Premier, DeepSeek-R1Frontier reasoning depth is the bottleneck, not token price
Open-weight / portability mandatesVariesMeta Llama 4 (Scout 17B, Maverick 17B), DeepSeek-V3.2, Mistral Large, GPT-OSSOpen-weight governance and cross-cloud portability
Regulated decisions (credit, medical, legal)CriticalStrongest tier + Automated Reasoning Guardrails + human gatesMathematical policy proof and audit trails outweigh token cost

Routing Mechanics

  • Static routing - endpoint/task type maps to a model in config. Simple, predictable, covers most systems.
  • Dynamic routing - a cheap classifier scores request complexity, then dispatches. Adds cost/latency; justify with traffic diversity.
  • Escalation routing - try the cheap tier; if confidence/validation fails, retry on the premium tier. Great when most requests are easy.
  • Intelligent Prompt Routing (Bedrock feature) - serverless single-endpoint routing between models in the same family (e.g. Haiku ↔ Sonnet or Nova Lite ↔ Nova Pro) based on predicted prompt complexity, cutting spend by up to ~30%+ without sacrificing quality.
Keep the matrix versioned: the model space shifts fast - review your routing matrix quarterly with fresh eval scores. Last year's "best value" tier is often this year's overpriced one.
⌨ HANDS-ON LABBuild a Routing Matrix
⭐ +150 XP

Map support, analytics, coding, and compliance tasks to model families.

1Create a workload-to-model matrix file.
2Add an escalation rule.
lab-sandbox — simulated environment
INFINITY LAB SANDBOX v2.6 — simulated shell
Type the command for the current objective. Helpers: "hint", "solution", "clear".
$
OBJECTIVE 1 / 2 — type "hint" if stuck
🧪 Knowledge Check
Press 1-4 to select1 of 2
Why do mature teams use model routing?
Because one model is always best
To optimize quality-latency-cost across workloads
To avoid testing
To remove telemetry