AquilaMoE: Efficient Training for MoE Models with Scale-Up and Scale-Out Strategies. - Bo-Wen Zhang 011; Liangdong Wang; Ye Yuan 016; Jijie Li; Shuhao Gu; Mengdi Zhao; Xinya Wu; Guang Liu 006; Chengwei Wu; Hanyu Zhao; Li Du; Yiming Ju; et al. (CoRR, 2024)