Larger Models Yield Better Results? Streamlined Severity Classification of ADHD-Related Concerns Using BERT-Based Knowledge Distillation
Introduced LastBERT, a 29M-parameter student distilled from BERT-base with a 3.8× reduction in parameter count. Evaluated across GLUE tasks and ADHD concern-severity classification, with formal analysis supporting the argument that scale is not always a prerequisite for task competence.