Large Language Models: DistilBERT - Smaller, Faster, Cheaper and Lighter | Towards Data Science

Unlocking the secrets of BERT compression: a student-teacher framework for maximum efficiency