[ ABORT TO HUD ]
SEQ. 1
SEQ. 2
SEQ. 3
Model Evaluation
Measuring Model Quality
Foundry provides built-in evaluation tools to systematically measure your AI outputs.
Built-In Evaluators
| Evaluator | Measures | Scale |
|---|---|---|
| Groundedness | Is the response supported by the provided context? | 1-5 |
| Relevance | Does the response address the user's question? | 1-5 |
| Coherence | Is the response well-structured and logical? | 1-5 |
| Fluency | Is the language natural and grammatically correct? | 1-5 |
| Similarity | How close is the response to a ground-truth answer? | 0-1 |
Running Evaluations via SDK
from azure.ai.projects.models import Evaluation
evaluation = project.evaluations.create(
data="test_dataset.jsonl",
evaluators={
"groundedness": {"type": "groundedness"},
"relevance": {"type": "relevance"},
"coherence": {"type": "coherence"}
}
)
results = project.evaluations.get(evaluation.id)
print(f"Groundedness: {results.metrics['groundedness']}")
💡 Key Insight: Always evaluate before and after fine-tuning or RAG changes. Without baseline metrics, you can't prove your changes actually improved quality.
FOUNDRY VERIFICATION
QUERY 1 // 1
What does the 'Groundedness' evaluator measure?
Grammar quality
Whether the response is supported by the provided context/data
Response speed
Token count