LightOn Optical Processing Unit : Scaling-up AI and HPC with a Non von Neumann co-processor. - Charles Brossollet; Alessandro Cappelli; Igor Carron; Charidimos Chaintoutis; Amélie Chatelain; Laurent Daudet; Sylvain Gigan; Daniel Hesslow; Florent Krzakala; Julien Launay; Safa Mokaadi; Fabien Moreau; et al. (HCS, 2021)

Photonic co-processors in HPC: Using LightOn OPUs for Randomized Numerical Linear Algebra. - Daniel Hesslow; Alessandro Cappelli; Igor Carron; Laurent Daudet; Raphaël Lafargue; Kilian Müller; Ruben Ohana; Gustave Pariente; Iacopo Poli (HCS, 2021)

Is the Number of Trainable Parameters All That Actually Matters? - Amélie Chatelain; Amine Djeghri; Daniel Hesslow; Julien Launay (ICBINB@NeurIPS, 2021)

Contrastive Embeddings for Neural Architectures. - Daniel Hesslow; Iacopo Poli (CoRR, 2021)

Is the Number of Trainable Parameters All That Actually Matters? - Amélie Chatelain; Amine Djeghri; Daniel Hesslow; Julien Launay; Iacopo Poli (CoRR, 2021)

What Language Model to Train if You Have One Million GPU Hours? - Teven Le Scao; Thomas Wang; Daniel Hesslow; Lucile Saulnier; Stas Bekman; M. Saiful Bari; Stella Biderman; Hady Elsahar; Niklas Muennighoff; Jason Phang; Ofir Press; Colin Raffel; et al. (EMNLP (Findings), 2022)

What Language Model Architecture and Pretraining Objective Works Best for Zero-Shot Generalization? - Thomas Wang; Adam Roberts; Daniel Hesslow; Teven Le Scao; Hyung Won Chung; Iz Beltagy; Julien Launay; Colin Raffel (ICML, 2022)

What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization? - Thomas Wang; Adam Roberts; Daniel Hesslow; Teven Le Scao; Hyung Won Chung; Iz Beltagy; Julien Launay; Colin Raffel (CoRR, 2022)

RITA: a Study on Scaling Up Generative Protein Sequence Models. - Daniel Hesslow; Niccoló Zanichelli; Pascal Notin; Iacopo Poli; Debora S. Marks (CoRR, 2022)

Scaling Laws Beyond Backpropagation. - Matthew J. Filipovich; Alessandro Cappelli; Daniel Hesslow; Julien Launay (CoRR, 2022)

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. - Teven Le Scao; Angela Fan; Christopher Akiki; Ellie Pavlick; Suzana Ilic; Daniel Hesslow; Roman Castagné; Alexandra Sasha Luccioni; François Yvon; Matthias Gallé; Jonathan Tow; Alexander M. Rush; et al. (CoRR, 2022)

The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data Only. - Guilherme Penedo; Quentin Malartic; Daniel Hesslow; Ruxandra Cojocaru; Hamza Alobeidli; Alessandro Cappelli; Baptiste Pannier; Ebtesam Almazrouei; Julien Launay (NeurIPS, 2023)

The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only. - Guilherme Penedo; Quentin Malartic; Daniel Hesslow; Ruxandra Cojocaru; Alessandro Cappelli; Hamza Alobeidli; Baptiste Pannier; Ebtesam Almazrouei; Julien Launay (CoRR, 2023)

The Falcon Series of Open Language Models. - Ebtesam Almazrouei; Hamza Alobeidli; Abdulaziz Alshamsi; Alessandro Cappelli; Ruxandra Cojocaru; Mérouane Debbah; Étienne Goffinet; Daniel Hesslow; Julien Launay; Quentin Malartic; Daniele Mazzotta; Badreddine Noune; et al. (CoRR, 2023)

Limiting factors for the continued scaling of Large Langauge Models : Data Sets, efficient systems for training, model architecture and novel hardware. (Facteurs limitants pour la poursuite de l'escalade des Grands Modèles Linguistiques : Jeux de données, systèmes d'entraînement efficaces, architecture des modèles et matériel novateur). - Daniel Hesslow (2024)