[ ABORT TO HUD ]
SEQ. 1
SEQ. 2
SEQ. 3
Model Distillation
Shrink the Cost, Keep the Quality
Model Distillation is the process of using a large, expensive model (teacher) to generate training data, then fine-tuning a smaller, cheaper model (student) to replicate the teacher's behavior.
Distillation Pipeline
- Generate: Run OpenAI o3-mini Thinking on 1,000 real-world queries. Save the outputs.
- Curate: Filter for high-quality responses. Remove errors.
- Fine-Tune: Train OpenAI o3-mini Mini on these curated examples.
- Evaluate: Compare Mini's outputs to Thinking's on a held-out test set.
Cost Impact
| Metric | OpenAI o3-mini Thinking | Distilled Mini | Savings |
|---|---|---|---|
| Cost per 1M tokens | ~$15 | ~$0.40 | 97% |
| Latency | ~3-8s | ~0.3s | 90% |
| Quality (on your task) | 98% | 92-95% | Minimal loss |
💡 OpenAI Stored Completions: If you use
store: true in the Responses API, OpenAI stores your completions. You can then use these stored outputs directly as fine-tuning data for distillation - no manual data collection needed.SYNAPSE VERIFICATION
QUERY 1 // 3
What is model distillation?
Compressing model weights
Using a large model's outputs to train a smaller model to replicate its behavior
Removing unused parameters
Converting to a different format