← Back to Dashboard
1. Guardrails Concepts2. Policy Testing and Failure Modes

Policy Testing and Failure Modes

📚 Safety and Guardrails10 min75 XP⌨ Hands-on lab

Policy Quality Is Measurable

Safety quality requires regression testing. Build a repeatable suite that includes allowed, risky, and prohibited requests, then score guardrail behavior over time - exactly like a unit-test suite for your safety posture.

The Four Outcomes Every Test Can Produce

OutcomeMeaningSeverity
True positiveUnsafe content correctly blockedWorking as intended
True negativeSafe content correctly allowedWorking as intended
False negativeUnsafe content allowed throughCritical - compliance/harm exposure
False positiveSafe content wrongly blockedErodes UX and user trust; teams then weaken policies

Building the Suite

  1. Benign set - realistic everyday requests that must always pass (regression canary for over-blocking).
  2. Borderline set - dual-use questions, aggressive-but-legitimate wording, policy edge cases.
  3. Prohibited set - known-bad prompts, including indirect phrasings and roleplay-wrapped attempts.
  4. Injection set - prompt-attack payloads (ignore-previous-instructions, tool-output poisoning, encoded instructions).

Score each release: block-rate on prohibited, pass-rate on benign, and drift versus the previous run. Bedrock's ApplyGuardrail API lets you test policies standalone - no model invocation required - which makes CI integration cheap.

  • Measure false negatives (unsafe content allowed).
  • Measure false positives (safe content blocked).
  • Track drift after prompt or model changes - a model upgrade can shift guardrail behavior even with identical policies.
CI rule: run the safety suite on every guardrail config change, every prompt release, and every model version bump. Any new false negative on the prohibited set is a blocking failure.
⌨ HANDS-ON LABBuild a Safety Regression Set
⭐ +150 XP

Create a red-team style prompt suite and validate expected block/allow outcomes.

1Create a CSV of test prompts and expected outcomes.
2Document block-rate and false-positive metrics.
lab-sandbox — simulated environment
INFINITY LAB SANDBOX v2.6 — simulated shell
Type the command for the current objective. Helpers: "hint", "solution", "clear".
$
OBJECTIVE 1 / 2 — type "hint" if stuck
🧪 Knowledge Check
Press 1-4 to select1 of 2
Which metric is most dangerous in regulated environments?
False negatives on unsafe content
UI load time
Icon size
Git commit frequency