Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators? - Tokio Kajitsuka; Issei Sato (ICLR, 2024)

On the Optimal Memorization Capacity of Transformers. - Tokio Kajitsuka; Issei Sato (ICLR, 2025)