StructMoE: Structured Mixture of Experts Using Low Rank Experts. - Zain Sarwar; Ashwinee Panda; Benjamin Thérien; Stephen Rawls; Anirban Das; Kartik Balasubramaniam; Berkcan Kapusuzoglu; Shixiong Zhang; Sambit Sahu; Milind Naphade; Supriyo Chakraborty (ENLSP, 2024)