Scalable Training of Language Models using JAX pjit and TPUv4. - Joanna Yoo; Kuba Perlin; Siddhartha Rao Kamalakara; João G. M. Araújo (CoRR, 2022)

Exploring Low Rank Training of Deep Neural Networks. - Siddhartha Rao Kamalakara; Acyr Locatelli; Bharat Venkitesh; Jimmy Ba; Yarin Gal; Aidan N. Gomez (CoRR, 2022)

Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models. - Laura Ruis; Maximilian Mozes; Juhan Bae; Siddhartha Rao Kamalakara; Dwarak Talupuru; Acyr Locatelli; Robert Kirk; Tim Rocktäschel; Edward Grefenstette; Max Bartolo (CoRR, 2024)

Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models. - Laura Ruis; Maximilian Mozes; Juhan Bae; Siddhartha Rao Kamalakara; Dwaraknath Gnaneshwar; Acyr Locatelli; Robert Kirk; Tim Rocktäschel; Edward Grefenstette; Max Bartolo (ICLR, 2025)