Self-Improving Pretraining: using post-trained models to pretrain better models. - Ellen Xiaoqing Tan; Shehzaad Dhuliawala; Jing Xu 014; Ping Yu; Sainbayar Sukhbaatar; Jason Weston; Olga Golovneva (CoRR, 2026)