On the weak link between importance and prunability of attention heads. - Aakriti Budhraja; Madhura Pande; Preksha Nema; Pratyush Kumar; Mitesh M. Khapra (EMNLP (1), 2020)

On the Importance of Local Information in Transformer Based Models. - Madhura Pande; Aakriti Budhraja; Preksha Nema; Pratyush Kumar; Mitesh M. Khapra (CoRR, 2020)

The Heads Hypothesis: A Unifying Statistical Approach Towards Understanding Multi-Headed Attention in BERT. - Madhura Pande; Aakriti Budhraja; Preksha Nema; Pratyush Kumar; Mitesh M. Khapra (AAAI, 2021)

On the Prunability of Attention Heads in Multilingual BERT. - Aakriti Budhraja; Madhura Pande; Pratyush Kumar; Mitesh M. Khapra (CoRR, 2021)