Single-Teacher View Augmentation: Enhancing Knowledge Distillation with Student-Guided Perturbations

Knowledge distillation (KD) typically relies on the fixed perspective of a single teacher, limiting the diversity of supervisory signals. While multi-teacher distillation addresses this by aggregating knowledge from multiple models, it incurs prohibitive computational and storage costs. To balance e...