softmax is not enough (for sharp out-of-distribution). - Petar Velickovic; Christos Perivolaropoulos; Federico Barbero; Razvan Pascanu (CoRR, 2024)

Round and Round We Go! What makes Rotary Positional Encodings useful? - Federico Barbero; Alex Vitvitskyi; Christos Perivolaropoulos; Razvan Pascanu; Petar Velickovic (ICLR, 2025)

Why do LLMs attend to the first token? - Federico Barbero; Alvaro Arroyo; Xiangming Gu; Christos Perivolaropoulos; Michael M. Bronstein; Petar Velickovic; Razvan Pascanu (CoRR, 2025)

Filter Equivariant Functions: A symmetric account of length-general extrapolation on lists. - Owen Lewis; Neil Ghani; Andrew Dudzik; Christos Perivolaropoulos; Razvan Pascanu; Petar Velickovic (CoRR, 2025)