Protect Public AI Endpoints

Expose AI APIs with identity checks, quotas, and abuse controls from day one. An unauthenticated LLM endpoint is a free compute faucet - it will be found, scripted, and drained, and the bill lands on you.

Auth Options on API Gateway

Layered Rate Limiting

edge:    WAF rate rules + bot control          # blunt abuse
gateway: usage plans - rate + burst + quota    # per key/tenant
app:     per-user token budgets (tokens ≠ requests!)
model:   Bedrock TPM/RPM quotas                # the real ceiling

Requests are the wrong unit for LLM cost - one request can be 200 or 200,000 tokens. Meter tokens per tenant per day at the application layer, and alert on outliers.

Abuse Patterns to Expect

• Scripted free-riding - your endpoint becomes someone's backend; token metering + auth kills it.
• Prompt-injection probing - automated jailbreak sweeps; guardrail intervention spikes are your detector.
• Oversized payloads - max-context requests as cost attacks; enforce input length caps at the gateway.

• Require auth for non-public workloads - "it's just a demo" endpoints get scraped too.
• Apply per-tenant rate limits and quotas; return 429 with Retry-After honestly.
• Return safe fallback responses under throttling - a cached or static answer beats a stack trace.