← Back to Dashboard
1. Containerized Orchestration Services2. Release Strategies and Rollbacks

Containerized Orchestration Services

📚 Bedrock with Containers10 min110 XP

When Containers Beat Serverless

Choose ECS/EKS when you need long-lived workflows, custom runtime dependencies, or tighter runtime tuning. The 15-minute Lambda ceiling and per-invocation model are wrong for agent loops that run for an hour or services holding warm in-memory caches.

Decision Table

SignalPrefer LambdaPrefer ECS/EKS
Request durationSeconds, boundedMinutes-hours (agent plans, batch orchestration)
Traffic shapeSpiky, idle-heavySustained, predictable
Runtime needsStandard runtimes sufficeCustom binaries, big deps, sidecars
StateStateless per callWarm caches, connection pools, streaming hubs
Concurrency controlReserved concurrency is enoughFine-grained semaphores, priority queues in-process

Operational Guardrails for Containerized AI Services

  • Per-service IAM task roles - each ECS task definition / EKS service account (IRSA) gets its own least-privilege role with pinned model ARNs; never a shared node role.
  • Sidecar observability - OpenTelemetry collector or ADOT sidecar exports traces/metrics without polluting app code.
  • Separate control-plane and data-plane responsibilities - the service that invokes models should not also mutate guardrail configs or provision infrastructure.
  • Health checks that mean something - readiness should verify Bedrock reachability (a cheap control-plane call), not just "process is up".
  • Autoscale on the right signal - queue depth or in-flight inference count, not CPU: LLM orchestration is IO-bound and CPU-idle while awaiting tokens.
Hybrid is normal: many production stacks run API Gateway + Lambda for the thin synchronous API and an ECS worker pool for long agent workflows, both invoking the same Bedrock models with the same guardrails and telemetry stack.
🧪 Knowledge Check
Press 1-4 to select1 of 2
Containers are often preferred when:
Workflows need long-lived processes and deep runtime control
You want zero monitoring
No IAM is allowed
Only static HTML is served