Machine Learning for Reliable MMWAVE Systems: Blockage Prediction and Proactive Handoff. - Ahmed Alkhateeb; Iz Beltagy; Sam Alex (GlobalSIP, 2018)

Construction of the Literature Graph in Semantic Scholar. - Waleed Ammar; Dirk Groeneveld; Chandra Bhagavatula; Iz Beltagy; Miles Crawford; Doug Downey; Jason Dunkelberger; Ahmed Elgohary; Sergey Feldman; Vu Ha; Rodney Kinney; Sebastian Kohlmeier; et al. (NAACL-HLT (3), 2018)

Machine Learning for Reliable mmWave Systems: Blockage Prediction and Proactive Handoff. - Ahmed Alkhateeb; Iz Beltagy (CoRR, 2018)

Improving Distant Supervision with Maxpooled Attention and Sentence-Level Supervision. - Iz Beltagy; Kyle Lo; Waleed Ammar (CoRR, 2018)

ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing. - Mark Neumann; Daniel King; Iz Beltagy; Waleed Ammar (BioNLP@ACL, 2019)

SciBERT: A Pretrained Language Model for Scientific Text. - Iz Beltagy; Kyle Lo; Arman Cohan (EMNLP/IJCNLP (1), 2019)

Pretrained Language Models for Sequential Sentence Classification. - Arman Cohan; Iz Beltagy; Daniel King; Bhavana Dalvi; Daniel S. Weld (EMNLP/IJCNLP (1), 2019)

Combining Distant and Direct Supervision for Neural Relation Extraction. - Iz Beltagy; Kyle Lo; Waleed Ammar (NAACL-HLT (1), 2019)

SciBERT: Pretrained Contextualized Embeddings for Scientific Text. - Iz Beltagy; Arman Cohan; Kyle Lo (CoRR, 2019)

SPECTER: Document-level Representation Learning using Citation-informed Transformers. - Arman Cohan; Sergey Feldman; Iz Beltagy; Doug Downey; Daniel S. Weld (ACL, 2020)

Don't Stop Pretraining: Adapt Language Models to Domains and Tasks. - Suchin Gururangan; Ana Marasovic; Swabha Swayamdipta; Kyle Lo; Iz Beltagy; Doug Downey; Noah A. Smith (ACL, 2020)

SciREX: A Challenge Dataset for Document-Level Information Extraction. - Sarthak Jain; Madeleine van Zuylen; Hannaneh Hajishirzi; Iz Beltagy (ACL, 2020)

Longformer: The Long-Document Transformer. - Iz Beltagy; Matthew E. Peters; Arman Cohan (CoRR, 2020)

SciCo: Hierarchical Cross-Document Coreference for Scientific Concepts. - Arie Cattan; Sophie Johnson; Daniel S. Weld; Ido Dagan; Iz Beltagy; Doug Downey; Tom Hope (AKBC, 2021)

Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study. - Rahul Nadkarni; David Wadden; Iz Beltagy; Noah A. Smith; Hannaneh Hajishirzi; Tom Hope (AKBC, 2021)

CDLM: Cross-Document Language Modeling. - Avi Caciularu; Arman Cohan; Iz Beltagy; Matthew E. Peters; Arie Cattan; Ido Dagan (EMNLP (Findings), 2021)

MS ^2: Multi-Document Summarization of Medical Studies. - Jay DeYoung; Iz Beltagy; Madeleine van Zuylen; Bailey Kuehl; Lucy Lu Wang (EMNLP (1), 2021)

A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers. - Pradeep Dasigi; Kyle Lo; Iz Beltagy; Arman Cohan; Noah A. Smith; Matt Gardner 001 (NAACL-HLT, 2021)

FLEX: Unifying Evaluation for Few-Shot NLP. - Jonathan Bragg; Arman Cohan; Kyle Lo; Iz Beltagy (NeurIPS, 2021)

Cross-Document Language Modeling. - Avi Caciularu; Arman Cohan; Iz Beltagy; Matthew E. Peters; Arie Cattan; Ido Dagan (CoRR, 2021)

MS2: Multi-Document Summarization of Medical Studies. - Jay DeYoung; Iz Beltagy; Madeleine van Zuylen; Bailey Kuehl; Lucy Lu Wang (CoRR, 2021)

PRIMER: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization. - Wen Xiao; Iz Beltagy; Giuseppe Carenini; Arman Cohan (CoRR, 2021)

LongChecker: Improving scientific claim verification by modeling full-abstract context. - David Wadden; Kyle Lo; Lucy Lu Wang; Arman Cohan; Iz Beltagy; Hannaneh Hajishirzi (CoRR, 2021)

Zero- and Few-Shot NLP with Pretrained Language Models. - Iz Beltagy; Arman Cohan; Robert L. Logan IV; Sewon Min; Sameer Singh 001 (ACL (tutorial), 2022)

PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization. - Wen Xiao; Iz Beltagy; Giuseppe Carenini; Arman Cohan (ACL (1), 2022)

What Language Model to Train if You Have One Million GPU Hours? - Teven Le Scao; Thomas Wang; Daniel Hesslow; Lucile Saulnier; Stas Bekman; M. Saiful Bari; Stella Biderman; Hady Elsahar; Niklas Muennighoff; Jason Phang; Ofir Press; Colin Raffel; et al. (EMNLP (Findings), 2022)

SciFact-Open: Towards open-domain scientific claim verification. - David Wadden; Kyle Lo; Bailey Kuehl; Arman Cohan; Iz Beltagy; Lucy Lu Wang; Hannaneh Hajishirzi (EMNLP (Findings), 2022)

Continued Pretraining for Better Zero- and Few-Shot Promptability. - Zhaofeng Wu; Robert L. Logan IV; Pete Walsh; Akshita Bhagia; Dirk Groeneveld; Sameer Singh 001; Iz Beltagy (EMNLP, 2022)

Staged Training for Transformer Language Models. - Sheng Shen 001; Pete Walsh; Kurt Keutzer; Jesse Dodge; Matthew E. Peters; Iz Beltagy (ICML, 2022)

What Language Model Architecture and Pretraining Objective Works Best for Zero-Shot Generalization? - Thomas Wang; Adam Roberts; Daniel Hesslow; Teven Le Scao; Hyung Won Chung; Iz Beltagy; Julien Launay; Colin Raffel (ICML, 2022)

Few-Shot Self-Rationalization with Natural Language Prompts. - Ana Marasovic; Iz Beltagy; Doug Downey; Matthew E. Peters (NAACL-HLT (Findings), 2022)

MultiVerS: Improving scientific claim verification with weak supervision and full-document context. - David Wadden; Kyle Lo; Lucy Lu Wang; Arman Cohan; Iz Beltagy; Hannaneh Hajishirzi (NAACL-HLT (Findings), 2022)

Infrastructure for Rapid Open Knowledge Network Development. - Michael J. Cafarella; Michael R. Anderson; Iz Beltagy; Arie Cattan; Sarah E. Chasins; Ido Dagan; Doug Downey; Oren Etzioni; Sergey Feldman; Tian Gao; Tom Hope; Kexin Huang; et al. (AI Mag., 2022)

Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search. - Daniel King; Zejiang Shen 001; Nishant Subramani; Daniel S. Weld; Iz Beltagy; Doug Downey (CoRR, 2022)

What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization? - Thomas Wang; Adam Roberts; Daniel Hesslow; Teven Le Scao; Hyung Won Chung; Iz Beltagy; Julien Launay; Colin Raffel (CoRR, 2022)

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. - Teven Le Scao; Angela Fan; Christopher Akiki; Ellie Pavlick; Suzana Ilic; Daniel Hesslow; Roman Castagné; Alexandra Sasha Luccioni; François Yvon; Matthias Gallé; Jonathan Tow; Alexander M. Rush; et al. (CoRR, 2022)

Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations. - Xinxi Lyu; Sewon Min; Iz Beltagy; Luke Zettlemoyer; Hannaneh Hajishirzi (ACL (1), 2023)

FiD-ICL: A Fusion-in-Decoder Approach for Efficient In-Context Learning. - Qinyuan Ye; Iz Beltagy; Matthew E. Peters; Xiang Ren 001; Hannaneh Hajishirzi (ACL (1), 2023)

How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources. - Yizhong Wang; Hamish Ivison; Pradeep Dasigi; Jack Hessel; Tushar Khot; Khyathi Raghavi Chandu; David Wadden; Kelsey MacMillan; Noah A. Smith; Iz Beltagy; Hannaneh Hajishirzi (NeurIPS, 2023)

The Semantic Scholar Open Data Platform. - Rodney Kinney; Chloe Anastasiades; Russell Authur; Iz Beltagy; Jonathan Bragg; Alexandra Buraczynski; Isabel Cachola; Stefan Candra; Yoganand Chandrasekhar; Arman Cohan; Miles Crawford; Doug Downey; et al. (CoRR, 2023)

Large Language Model Distillation Doesn't Need a Teacher. - Ananya Harsh Jha; Dirk Groeneveld; Emma Strubell; Iz Beltagy (CoRR, 2023)

Efficiency Pentathlon: A Standardized Arena for Efficiency Evaluation. - Hao Peng 009; Qingqing Cao; Jesse Dodge; Matthew E. Peters; Jared Fernandez; Tom Sherborne; Kyle Lo; Sam Skjonsberg; Emma Strubell; Darrell Plessas; Iz Beltagy; Evan Pete Walsh; et al. (CoRR, 2023)

Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2. - Hamish Ivison; Yizhong Wang; Valentina Pyatkin; Nathan Lambert 001; Matthew E. Peters; Pradeep Dasigi; Joel Jang; David Wadden; Noah A. Smith; Iz Beltagy; Hannaneh Hajishirzi (CoRR, 2023)

Catwalk: A Unified Language Model Evaluation Framework for Many Datasets. - Dirk Groeneveld; Anas Awadalla; Iz Beltagy; Akshita Bhagia; Ian Magnusson; Hao Peng 009; Oyvind Tafjord; Pete Walsh; Kyle Richardson 001; Jesse Dodge (CoRR, 2023)

Transparency Helps Reveal When Language Models Learn Meaning. - Zhaofeng Wu; William Merrill; Hao Peng 009; Iz Beltagy; Noah A. Smith (Trans. Assoc. Comput. Linguistics, 2023)

OLMo: Accelerating the Science of Language Models. - Dirk Groeneveld; Iz Beltagy; Evan Pete Walsh; Akshita Bhagia; Rodney Kinney; Oyvind Tafjord; Ananya Harsh Jha; Hamish Ivison; Ian Magnusson; Yizhong Wang; Shane Arora; David Atkinson; et al. (ACL (1), 2024)

Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research. - Luca Soldaini; Rodney Kinney; Akshita Bhagia; Dustin Schwenk; David Atkinson; Russell Authur; Ben Bogin; Khyathi Raghavi Chandu; Jennifer Dumas; Yanai Elazar; Valentin Hofmann; Ananya Harsh Jha; et al. (ACL (1), 2024)

TESS: Text-to-Text Self-Conditioned Simplex Diffusion. - Rabeeh Karimi Mahabadi; Hamish Ivison; Jaesung Tae; James Henderson 001; Iz Beltagy; Matthew E. Peters; Arman Cohan (EACL (1), 2024)

Paloma: A Benchmark for Evaluating Language Model Fit. - Ian Magnusson; Akshita Bhagia; Valentin Hofmann; Luca Soldaini; Ananya Harsh Jha; Oyvind Tafjord; Dustin Schwenk; Evan Pete Walsh; Yanai Elazar; Kyle Lo; Dirk Groeneveld; Iz Beltagy; et al. (NeurIPS, 2024)

OLMo: Accelerating the Science of Language Models. - Dirk Groeneveld; Iz Beltagy; Pete Walsh; Akshita Bhagia; Rodney Kinney; Oyvind Tafjord; Ananya Harsh Jha; Hamish Ivison; Ian Magnusson; Yizhong Wang; Shane Arora; David Atkinson; et al. (CoRR, 2024)

Source-Aware Training Enables Knowledge Attribution in Language Models. - Muhammad Khalifa; David Wadden; Emma Strubell; Honglak Lee; Lu Wang 008; Iz Beltagy; Hao Peng 009 (CoRR, 2024)