[ ABORT TO HUD ]
SEQ. 1
SEQ. 2
SEQ. 3
Evals & Moderation
Evaluating AI Quality
Evals are automated tests that measure your AI system's quality. OpenAI provides an evaluation framework for testing model outputs against expected results.
Types of Evals
| Eval Type | Method | Best For |
|---|---|---|
| Exact Match | Output must match expected value exactly | Classification, structured data |
| LLM-as-Judge | A separate model scores the output quality | Creative writing, summaries |
| Semantic Similarity | Embedding distance between output and expected | Open-ended questions |
| Human Review | Manual scoring by domain experts | Complex, subjective tasks |
The Moderation API
The Moderation API is a free endpoint that classifies text into safety categories (hate, violence, self-harm, sexual content). Use it as a pre-filter before processing user input.
const moderation = await openai.moderations.create({
input: userMessage
});
if (moderation.results[0].flagged) {
return "This content violates our usage policy.";
}
🔒 Production Rule: Always run user inputs through the Moderation API before passing them to your main model. It's free and prevents harmful content from entering your pipeline.
⌨ HANDS-ON LABScreen Content with the Moderation API
⭐ +150 XPBefore user text reaches your agent, it must pass through a safety gate. Wire up the free Moderation endpoint and inspect its category scores.
1POST user text to /v1/moderations using the omni-moderation-latest model.
2Pipe the response through jq to inspect the numeric category_scores - your thresholds live here.
OBJECTIVE 1 / 2 — type "hint" if stuck
SYNAPSE VERIFICATION
QUERY 1 // 3
What is an 'LLM-as-Judge' eval?
A legal AI tool
Using a separate model to score the quality of another model's output
A benchmarking competition
A moderation filter