Gravitational Clustering. - Armen Aghajanyan (CoRR, 2015)

SoftTarget Regularization: An Effective Technique to Reduce Over-Fitting in Neural Networks. - Armen Aghajanyan (CYBCONF, 2017)

Charged Point Normalization: An Efficient Solution to the Saddle Point Problem. - Armen Aghajanyan (ICLR (Workshop), 2017)

Convolution Aware Initialization. - Armen Aghajanyan (CoRR, 2017)

Towards Language Agnostic Universal Representations. - Armen Aghajanyan; Xia Song; Saurabh Tiwary (ACL (1), 2019)

Conversational Semantic Parsing. - Armen Aghajanyan; Jean Maillard; Akshat Shrivastava; Keith Diedrick; Michael Haeger; Haoran Li 007; Yashar Mehdad; Veselin Stoyanov; Anuj Kumar; Mike Lewis; Sonal Gupta (EMNLP (1), 2020)

Pre-training via Paraphrasing. - Mike Lewis; Marjan Ghazvininejad; Gargi Ghosh; Armen Aghajanyan; Sida Wang 001; Luke Zettlemoyer (NeurIPS, 2020)

Conversational Semantic Parsing. - Armen Aghajanyan; Jean Maillard; Akshat Shrivastava; Keith Diedrick; Mike Haeger; Haoran Li 007; Yashar Mehdad; Ves Stoyanov; Anuj Kumar; Mike Lewis; Sonal Gupta (CoRR, 2020)

Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning. - Armen Aghajanyan; Sonal Gupta; Luke Zettlemoyer (ACL/IJCNLP (1), 2021)

Muppet: Massive Multi-task Representations with Pre-Finetuning. - Armen Aghajanyan; Anchit Gupta; Akshat Shrivastava; Xilun Chen 002; Luke Zettlemoyer; Sonal Gupta (EMNLP (1), 2021)

VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding. - Hu Xu 001; Gargi Ghosh; Po-Yao Huang 001; Dmytro Okhonko; Armen Aghajanyan; Florian Metze; Luke Zettlemoyer; Christoph Feichtenhofer (EMNLP (1), 2021)

Better Fine-Tuning by Reducing Representational Collapse. - Armen Aghajanyan; Akshat Shrivastava; Anchit Gupta; Naman Goyal 001; Luke Zettlemoyer; Sonal Gupta (ICLR, 2021)

Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog. - Arun Babu; Akshat Shrivastava; Armen Aghajanyan; Ahmed Aly; Angela Fan; Marjan Ghazvininejad (NAACL-HLT, 2021)

RetroNLU: Retrieval Augmented Task-Oriented Semantic Parsing. - Vivek Gupta 001; Akshat Shrivastava; Adithya Sagar; Armen Aghajanyan; Denis Savenkov (ConvAI@ACL, 2022)

Improving Passage Retrieval with Zero-Shot Question Generation. - Devendra Singh Sachan; Mike Lewis; Mandar Joshi; Armen Aghajanyan; Wen-tau Yih; Joelle Pineau; Luke Zettlemoyer (EMNLP, 2022)

HTLM: Hyper-Text Pre-Training and Prompting of Language Models. - Armen Aghajanyan; Dmytro Okhonko; Mike Lewis; Mandar Joshi; Hu Xu 001; Gargi Ghosh; Luke Zettlemoyer (ICLR, 2022)

CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training. - Patrick Huber; Armen Aghajanyan; Barlas Oguz; Dmytro Okhonko; Scott Yih; Sonal Gupta; Xilun Chen 002 (NAACL-HLT (Findings), 2022)

Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models. - Kushal Tirumala; Aram H. Markosyan; Luke Zettlemoyer; Armen Aghajanyan (NeurIPS, 2022)

CM3: A Causal Masked Multimodal Model of the Internet. - Armen Aghajanyan; Bernie Huang; Candace Ross; Vladimir Karpukhin; Hu Xu 001; Naman Goyal 001; Dmytro Okhonko; Mandar Joshi; Gargi Ghosh; Mike Lewis; Luke Zettlemoyer (CoRR, 2022)

InCoder: A Generative Model for Code Infilling and Synthesis. - Daniel Fried; Armen Aghajanyan; Jessy Lin; Sida Wang 001; Eric Wallace; Freda Shi; Ruiqi Zhong; Scott Yih; Luke Zettlemoyer; Mike Lewis (ICLR, 2023)

Scaling Laws for Generative Mixed-Modal Language Models. - Armen Aghajanyan; Lili Yu; Alexis Conneau; Wei-Ning Hsu; Karen Hambardzumyan; Susan Zhang; Stephen Roller; Naman Goyal 001; Omer Levy; Luke Zettlemoyer (ICML, 2023)

Retrieval-Augmented Multimodal Language Modeling. - Michihiro Yasunaga; Armen Aghajanyan; Weijia Shi; Richard James 001; Jure Leskovec; Percy Liang; Mike Lewis; Luke Zettlemoyer; Wen-Tau Yih (ICML, 2023)

D4: Improving LLM Pretraining via Document De-Duplication and Diversification. - Kushal Tirumala; Daniel Simig; Armen Aghajanyan; Ari Morcos (NeurIPS, 2023)

MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers. - Lili Yu; Daniel Simig; Colin Flaherty; Armen Aghajanyan; Luke Zettlemoyer; Mike Lewis (NeurIPS, 2023)

Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning. - Lili Yu; Bowen Shi; Ramakanth Pasunuru; Benjamin Muller; Olga Golovneva; Tianlu Wang; Arun Babu; Binh Tang; Brian Karrer; Shelly Sheynin; Candace Ross; Adam Polyak; et al. (CoRR, 2023)

DOMINO: A Dual-System for Multi-step Visual Language Reasoning. - Peifang Wang; Olga Golovneva; Armen Aghajanyan; Xiang Ren 001; Muhao Chen 001; Asli Celikyilmaz; Maryam Fazel-Zarandi (CoRR, 2023)

Jointly Training Large Autoregressive Multimodal Models. - Emanuele Aiello; Lili Yu; Yixin Nie; Armen Aghajanyan; Barlas Oguz (ICLR, 2024)

Text Quality-Based Pruning for Efficient Training of Language Models. - Vasu Sharma; Karthik Padthe; Newsha Ardalani; Kushal Tirumala; Russell Howes; Hu Xu 001; Po-Yao Huang 001; Shang-Wen Li 001; Armen Aghajanyan; Gargi Ghosh; Luke Zettlemoyer (CoRR, 2024)

Small Molecule Optimization with Large Language Models. - Philipp Guevorguian; Menua Bedrosian; Tigran Fahradyan; Gayane Chilingaryan; Hrant Khachatrian; Armen Aghajanyan (CoRR, 2024)

MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts. - Xi Victoria Lin; Akshat Shrivastava; Liang Luo; Srinivasan Iyer 001; Mike Lewis; Gargi Ghosh; Luke Zettlemoyer; Armen Aghajanyan (CoRR, 2024)

When Worse is Better: Navigating the compression-generation tradeoff in visual tokenization. - Vivek Ramanujan; Kushal Tirumala; Armen Aghajanyan; Luke Zettlemoyer; Ali Farhadi (CoRR, 2024)

BartSmiles: Generative Masked Language Models for Molecular Representations. - Gayane Chilingaryan; Hovhannes Tamoyan; Ani Tevosyan; Nelly Babayan; Karen Hambardzumyan; Zaven Navoyan; Armen Aghajanyan; Hrant Khachatrian; Lusine Khondkaryan (J. Chem. Inf. Model., 2024)