Direct Inference Endpoints

Use InvokeModel for classic request/response patterns and streaming variants (InvokeModelWithResponseStream) for low-latency UI updates. Each model provider defines its own native request/response body schema for InvokeModel - Amazon Nova, Anthropic Claude, and Meta Llama all differ, so check the model's API reference before hardcoding a payload shape.

Provider Payloads Differ - See It Concretely

# Anthropic Claude via InvokeModel (native Messages schema)
{
  "anthropic_version": "bedrock-2023-05-31",
  "max_tokens": 512,
  "messages": [{"role": "user", "content": "Explain SQS DLQs"}]
}

# Amazon Nova via InvokeModel (different schema!)
{
  "messages": [{"role": "user", "content": [{"text": "Explain SQS DLQs"}]}],
  "inferenceConfig": {"maxTokens": 512, "temperature": 0.2}
}

This payload drift is exactly why the Converse API (next lesson) exists - one schema across providers.

Streaming: Perceived Latency Is Real Latency

With InvokeModelWithResponseStream (or ConverseStream), tokens arrive as server-sent chunks the moment the model emits them. A 20-second full completion can show first words in well under a second.

response = client.converse_stream(
    modelId="amazon.nova-lite-v1:0",
    messages=[{"role": "user", "content": [{"text": "Draft a runbook intro"}]}],
)
for event in response["stream"]:
    if "contentBlockDelta" in event:
        print(event["contentBlockDelta"]["delta"]["text"], end="", flush=True)
    elif "metadata" in event:
        print(event["metadata"]["usage"])  # tokens arrive in the final event

• Prefer strict request schemas per model family, or use the Converse API to avoid per-provider payload drift.
• Capture usage tokens (inputTokens/outputTokens) in telemetry for cost governance - in streams they arrive in the final metadata event.
• Pin exact model IDs in production configs - Amazon Nova (Micro, Lite, Pro, Premier) is AWS's current first-party model family.