[ ABORT TO HUD ]
SEQ. 1
SEQ. 2
SEQ. 3
Model Distillation
Shrink the Cost, Keep the Quality
Model Distillation is the process of using a large, expensive model (teacher) to generate training data, then fine-tuning a smaller, cheaper model (student) to replicate the teacher's behavior.
Distillation Pipeline
- Generate: Run flagship models (GPT-4o or o1) on 1,000 real-world queries. Save the outputs.
- Curate: Filter for high-quality responses. Remove errors.
- Fine-Tune: Train gpt-4o-mini on these curated examples.
- Evaluate: Compare gpt-4o-mini's outputs to GPT-4o on a held-out test set.
Cost Impact
| Metric | Teacher (GPT-4o) | Distilled Student (gpt-4o-mini) | Savings |
|---|---|---|---|
| Cost per 1M tokens | ~$10.00 (out) / $2.50 (in) | ~$0.60 (out) / $0.15 (in) | 94-97% |
| Latency | ~1-3s | ~0.2-0.4s | 85-90% |
| Quality (on your task) | 99% | 94-96% | Minimal loss |
💡 OpenAI Stored Completions: If you use
store: true in the Responses API, OpenAI stores your completions. You can then use these stored outputs directly as fine-tuning data for distillation - no manual data collection needed.⌨ HANDS-ON LABLaunch an Automated Distillation Job
⭐ +200 XPTransfer intelligence from GPT-4o into a fast, cheap gpt-4o-mini student model using OpenAI's distillation API.
1POST to /v1/fine_tuning/jobs specifying the training method distillation and the student model.
OBJECTIVE 1 / 1 — type "hint" if stuck
SYNAPSE VERIFICATION
QUERY 1 // 3
What is model distillation?
Compressing model weights
Using a large model's outputs to train a smaller model to replicate its behavior
Removing unused parameters
Converting to a different format