How You Start Matters for Generalization. - Sameera Ramasinghe; Lachlan E. MacDonald; Moshiur R. Farazi; Hemanth Saratchandran; Simon Lucey (CoRR, 2022)

A global analysis of global optimisation. - Lachlan Ewen MacDonald; Hemanth Saratchandran; Jack Valmadre; Simon Lucey (CoRR, 2022)

Curvature-Aware Training for Coordinate Networks. - Hemanth Saratchandran; Shin-Fang Ch'ng; Sameera Ramasinghe; Lachlan E. MacDonald; Simon Lucey (ICCV, 2023)

How much does Initialization Affect Generalization? - Sameera Ramasinghe; Lachlan Ewen MacDonald; Moshiur R. Farazi; Hemanth Saratchandran; Simon Lucey (ICML, 2023)

On skip connections and normalisation layers in deep optimisation. - Lachlan E. MacDonald; Jack Valmadre; Hemanth Saratchandran; Simon Lucey (NeurIPS, 2023)

On the effectiveness of neural priors in modeling dynamical systems. - Sameera Ramasinghe; Hemanth Saratchandran; Violetta Shevchenko; Simon Lucey (CoRR, 2023)

D'OH: Decoder-Only Random Hypernetworks for Implicit Neural Representations. - Cameron Gordon; Lachlan E. MacDonald; Hemanth Saratchandran; Simon Lucey (ACCV (7), 2024)

From Activation to Initialization: Scaling Insights for Optimizing Neural Fields. - Hemanth Saratchandran; Sameera Ramasinghe; Simon Lucey (CVPR, 2024)

Invertible Neural Warp for NeRF. - Shin-Fang Ch'ng; Ravi Garg; Hemanth Saratchandran; Simon Lucey (ECCV (17), 2024)

Weight Conditioning for Smooth Optimization of Neural Networks. - Hemanth Saratchandran; Thomas X. Wang; Simon Lucey (ECCV (85), 2024)

A sampling theory perspective on activations for implicit neural representations. - Hemanth Saratchandran; Sameera Ramasinghe; Violetta Shevchenko; Alexander Long; Simon Lucey (ICML, 2024)

Architectural Strategies for the optimization of Physics-Informed Neural Networks. - Hemanth Saratchandran; Shin-Fang Ch'ng; Simon Lucey (CoRR, 2024)

Analyzing the Neural Tangent Kernel of Periodically Activated Coordinate Networks. - Hemanth Saratchandran; Shin-Fang Ch'ng; Simon Lucey (CoRR, 2024)

Preconditioners for the Stochastic Training of Implicit Neural Representations. - Shin-Fang Ch'ng; Hemanth Saratchandran; Simon Lucey (CoRR, 2024)

Sine Activated Low-Rank Matrices for Parameter Efficient Learning. - Yiping Ji; Hemanth Saratchandran; Cameron Gordon; Zeyu Zhang 006; Simon Lucey (CoRR, 2024)

Rethinking Softmax: Self-Attention with Polynomial Activations. - Hemanth Saratchandran; Jianqiao Zheng; Yiping Ji; Wenbo Zhang; Simon Lucey (CoRR, 2024)

Preconditioners for the Stochastic Training of Neural Fields. - Shin-Fang Ch'ng; Hemanth Saratchandran; Simon Lucey (CVPR, 2025)

RandLoRA: Full rank parameter-efficient fine-tuning of large models. - Paul Albert; Frederic Z. Zhang; Hemanth Saratchandran; Cristian Rodriguez Opazo; Anton van den Hengel; Ehsan Abbasnejad (ICLR, 2025)

Efficient Learning with Sine-Activated Low-Rank Matrices. - Yiping Ji; Hemanth Saratchandran; Cameron Gordon; Zeyu Zhang 006; Simon Lucey (ICLR, 2025)

Always Skip Attention. - Yiping Ji; Hemanth Saratchandran; Peyman Moghaddam; Simon Lucey (CoRR, 2025)

Enhancing Transformers Through Conditioned Embedded Tokens. - Hemanth Saratchandran; Simon Lucey (CoRR, 2025)

Structured Initialization for Vision Transformers. - Jianqiao Zheng; Xueqian Li; Hemanth Saratchandran; Simon Lucey (CoRR, 2025)

Leaner Transformers: More Heads, Less Depth. - Hemanth Saratchandran; Damien Teney; Simon Lucey (CoRR, 2025)

Compressing Sine-Activated Low-Rank Adapters through Post-Training Quantization. - Cameron Gordon; Yiping Ji; Hemanth Saratchandran; Paul Albert; Simon Lucey (CoRR, 2025)

Transformers Pretrained on Procedural Data Contain Modular Structures for Algorithmic Reasoning. - Zachary Shinnick; Liangze Jiang; Hemanth Saratchandran; Anton van den Hengel; Damien Teney (CoRR, 2025)