loading
loading
Training a smaller, cheaper 'student' model to mimic a bigger, smarter 'teacher' model.
You run a large model over lots of prompts and train a small model to reproduce its outputs (or its full probability distribution), transferring much of the big model's capability into something faster and cheaper to serve. The student never matches the teacher exactly, but it can get surprisingly close on the tasks you distilled for — which is how you get small models that punch above their weight. Many of today's fast 'mini' or 'flash' tier models are distilled from their larger siblings.
Plainly
Think of Distillation as a simple recipe for doing the work better. Training a smaller, cheaper 'student' model to mimic a bigger, smarter 'teacher' model.
In practice
Use it when you need a repeatable method instead of guessing from vibes. In practice, define the owner, input, output, and failure mode before you rely on it.