Reward-free Alignment for Conflicting Objectives. - Peter L. Chen; Xiaopeng Li; Xi Chen; Tianyi Lin (CoRR, 2026)