Compensate Quantization Errors: Make Weights Hierarchical to Compensate Each Other. - Yifei Gao; Jie Ou; Lei Wang 018; Yuting Xiao; Xiangzhiyuan Xiangzhiyuan; Ruiting Dai; Jun Cheng 002 (NAACL-HLT (Findings), 2024)