Replacing Human Audio with Synthetic Audio for on-Device Unspoken Punctuation Prediction. - Daria Soboleva; Ondrej Skopek; Márius Sajgalík; Victor Carbune; Felix Weissenberger; Julia Proskurnia; Bogdan Prisacari; Daniel Valcarce; Justin Lu; Rohit Prabhavalkar; Balint Miklos (ICASSP, 2021)

SlimPajama-DC: Understanding Data Combinations for LLM Training. - Zhiqiang Shen; Tianhua Tao; Liqun Ma; Willie Neiswanger; Zhengzhong Liu 001; Hongyi Wang 001; Bowen Tan; Joel Hestness; Natalia Vassilieva; Daria Soboleva; Eric P. Xing (CoRR, 2023)

BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model. - Nolan Dey; Daria Soboleva; Faisal Al-Khateeb; Bowen Yang; Ribhu Pathria; Hemant Khachane; Shaheer Muhammad; Zhiming Chen; Robert Myers; Jacob Robert Steeves; Natalia Vassilieva; Marvin Tom; et al. (CoRR, 2023)

Position Interpolation Improves ALiBi Extrapolation. - Faisal Al-Khateeb; Nolan Dey; Daria Soboleva; Joel Hestness (CoRR, 2023)

Agent-Based Insight into Eco-Choices: Simulating the Fast Fashion Shift. - Daria Soboleva; Angel Sánchez (CoRR, 2024)

Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs. - Shane Bergsma; Nolan Simran Dey; Gurpreet Gosal; Gavia Gray; Daria Soboleva; Joel Hestness (ICLR, 2025)

Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training. - Shane Bergsma; Nolan Dey; Gurpreet Gosal; Gavia Gray; Daria Soboleva; Joel Hestness (CoRR, 2025)