Definition
Distillation in AI means training a smaller model to copy the useful skills of a larger model. The smaller model learns from the larger model’s answers, patterns, or predictions. This is important because smaller models can be faster, cheaper, and easier to use on phones or other devices. A downside is that the smaller model may lose some accuracy or miss details.
How to explain distillation to your child
AI distillation helps a smaller AI system learn from a larger one. The smaller system tries to keep the most useful lessons. It may work faster, but it might not know as much.
Try using this in a calm, casual conversation rather than turning it into a lesson.
Here’s how to think about it
Imagine a teacher turns a long textbook into a short study guide. The guide keeps the most important facts and removes many details. It is quicker to read, but it may leave out some useful information.
Learning Check
What happens during AI distillation?