Model Evaluation — Fine-Tuning & Customization

[ ABORT TO HUD ]
SEQ. 1
SEQ. 2
SEQ. 3

Model Evaluation

🎯 Fine-Tuning & Customization 9 min 80 BASE XP⌨ HANDS-ON LAB

Measuring Model Quality

Foundry provides built-in evaluation tools to systematically measure your AI outputs.

Built-In Evaluators

EvaluatorMeasuresScale
GroundednessIs the response supported by the provided context?1-5
RelevanceDoes the response address the user's question?1-5
CoherenceIs the response well-structured and logical?1-5
FluencyIs the language natural and grammatically correct?1-5
SimilarityHow close is the response to a ground-truth answer?0-1

Running Evaluations via SDK

import os
from azure.ai.evaluation import (
    evaluate,
    GroundednessEvaluator,
    RelevanceEvaluator,
    CoherenceEvaluator,
)

model_config = {
    "azure_endpoint": os.environ["AZURE_OPENAI_ENDPOINT"],
    "azure_deployment": "gpt-4.1",
}

result = evaluate(
    data="test_dataset.jsonl",
    evaluators={
        "groundedness": GroundednessEvaluator(model_config=model_config),
        "relevance": RelevanceEvaluator(model_config=model_config),
        "coherence": CoherenceEvaluator(model_config=model_config),
    },
    azure_ai_project=os.environ["AZURE_AI_PROJECT_ENDPOINT"],
)
print(f"Groundedness: {result['metrics']['groundedness.groundedness']}")
💡 Key Insight: Always evaluate before and after fine-tuning or RAG changes. Without baseline metrics, you can't prove your changes actually improved quality.
⌨ HANDS-ON LABRun Automated Groundedness and Coherence Evaluation
⭐ +200 XP

Evaluate model generation quality using Foundry's built-in evaluators: Groundedness, Relevance, and Coherence.

1Run evaluation pipeline using azure-ai-evaluation package in Python.
lab-sandbox — simulated environment
INFINITY LAB SANDBOX v2.6 — simulated shell
Type the command for the current objective. Helpers: "hint", "solution", "clear".
$
OBJECTIVE 1 / 1 — type "hint" if stuck
FOUNDRY VERIFICATION
QUERY 1 // 1
What does the 'Groundedness' evaluator measure?
Grammar quality
Whether the response is supported by the provided context/data
Response speed
Token count