โ Back to Dashboard
1. Offline Evaluation Design2. Online Eval and User Feedback Loops
Offline Evaluation Design
๐ Evaluation Frameworksโฑ 11 minโญ 90 XPโจ Hands-on lab
Offline Evals Before User Traffic
Offline evaluation lets you compare prompt/model variants before rollout.
- Use representative and adversarial datasets.
- Score factuality, instruction adherence, and policy compliance.
- Gate promotion with explicit pass thresholds.
โจ HANDS-ON LABCreate a Bedrock Eval Scorecard
โญ +150 XPDefine objective quality dimensions and pass thresholds before deployment.
1Create an evaluation rubric file.
2Set minimum release thresholds.
lab-sandbox โ simulated environment
INFINITY LAB SANDBOX v2.6 โ simulated shell
Type the command for the current objective. Helpers: "hint", "solution", "clear".
$
OBJECTIVE 1 / 2 โ type "hint" if stuck ๐งช Knowledge CheckPress 1-4 to select1 of 2
Why run offline evals first?
To skip monitoring
To reduce risk before exposing users
To bypass IAM
To avoid test data