Model Distillation — Fine-Tuning & Distillation
[ ABORT TO HUD ]
SEQ. 1
SEQ. 2
SEQ. 3
Model Distillation
Shrink the Cost, Keep the Quality
Model Distillation is the process of using a large, expensive model (teacher) to generate training data, then fine-tuning a smaller, cheaper model (student) to replicate the teacher's behavior.
Distillation Pipeline
- Generate: Run flagship models (GPT-4o or o1) on 1,000 real-world queries. Save the outputs.
- Curate: Filter for high-quality responses. Remove errors.
- Fine-Tune: Train gpt-4o-mini on these curated examples.
- Evaluate: Compare gpt-4o-mini's outputs to GPT-4o on a held-out test set.
Cost Impact
| Metric | Teacher (GPT-4o) | Distilled Student (gpt-4o-mini) | Savings |
|---|---|---|---|
| Cost per 1M tokens | ~$10.00 (out) / $2.50 (in) | ~$0.60 (out) / $0.15 (in) | 94-97% |
| Latency | ~1-3s | ~0.2-0.4s | 85-90% |
| Quality (on your task) | 99% | 94-96% | Minimal loss |
💡 OpenAI Stored Completions & GPT-6 Economics: With
store: true in the Responses API, stored outputs from frontier models (like gpt-6-astra) can seed evaluation and distillation datasets. Note that gpt-4o / gpt-4o-mini fine-tuned models sunset on January 14, 2027, and gpt-6-luna ($0.10 in / $0.50 out per MTok) already delivers distilled-class unit economics out of the box.⌨ HANDS-ON LABLaunch an Automated Distillation Job
⭐ +200 XPTransfer intelligence from GPT-4o into a fast, cheap gpt-4o-mini student model using OpenAI's distillation API.
1POST to /v1/fine_tuning/jobs specifying the training method distillation and the student model.
OBJECTIVE 1 / 1 — type "hint" if stuck
SYNAPSE VERIFICATION
QUERY 1 // 3
What is model distillation?
Compressing model weights
Using a large model's outputs to train a smaller model to replicate its behavior
Removing unused parameters
Converting to a different format