What Survives Distillation? A Mechanistic Interpretability Study of GPT-2 and DistilGPT-2

What Survives Distillation? A Mechanistic Interpretability Study of GPT-2 and DistilGPT-2. The story centres on GPT. Reported by arXiv. Bharat Hunt files it under AI Research and AI Models — the section covering papers, benchmarks, evaluations, interpretability and safety results.

Written by Bharat Hunt from the headline and the coverage below. The original reporting is the source of truth.