Exploiting Transformer Activation Sparsity with Dynamic Inference. - Mikolaj Piórczynski; Filip Szatkowski; Klaudia Balazy; Bartosz Wójcik (CoRR, 2023)

Exploiting Activation Sparsity with Dense to Dynamic-k Mixture-of-Experts Conversion. - Filip Szatkowski; Bartosz Wójcik; Mikolaj Piórczynski; Simone Scardapane (NeurIPS, 2024)