Iz Beltagy

According to our database, Iz Beltagy authored at least 52 papers between 2018 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2024

OLMo: Accelerating the Science of Language Models.

Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, Hannaneh Hajishirzi

CoRR, 2024

Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research.

Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A. Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, Kyle Lo

CoRR, 2024

TESS: Text-to-Text Self-Conditioned Simplex Diffusion.

Rabeeh Karimi Mahabadi, Hamish Ivison, Jaesung Tae, James Henderson, Iz Beltagy, Matthew E. Peters, Arman Cohan

Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, 2024

2023

Paloma: A Benchmark for Evaluating Language Model Fit.

Ian Magnusson, Akshita Bhagia, Valentin Hofmann, Luca Soldaini, Ananya Harsh Jha, Oyvind Tafjord, Dustin Schwenk, Evan Pete Walsh, Yanai Elazar, Kyle Lo, Dirk Groeneveld, Iz Beltagy, Hannaneh Hajishirzi, Noah A. Smith, Kyle Richardson, Jesse Dodge

CoRR, 2023

Catwalk: A Unified Language Model Evaluation Framework for Many Datasets.

Dirk Groeneveld, Anas Awadalla, Iz Beltagy, Akshita Bhagia, Ian Magnusson, Hao Peng, Oyvind Tafjord, Pete Walsh, Kyle Richardson, Jesse Dodge

CoRR, 2023

Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2.

Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew E. Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A. Smith, Iz Beltagy, Hannaneh Hajishirzi

CoRR, 2023

Efficiency Pentathlon: A Standardized Arena for Efficiency Evaluation.

Hao Peng, Qingqing Cao, Jesse Dodge, Matthew E. Peters, Jared Fernandez, Tom Sherborne, Kyle Lo, Sam Skjonsberg, Emma Strubell, Darrell Plessas, Iz Beltagy, Evan Pete Walsh, Noah A. Smith, Hannaneh Hajishirzi

CoRR, 2023

How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources.

Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, Hannaneh Hajishirzi

CoRR, 2023

Large Language Model Distillation Doesn't Need a Teacher.

Ananya Harsh Jha, Dirk Groeneveld, Emma Strubell, Iz Beltagy

CoRR, 2023

TESS: Text-to-Text Self-Conditioned Simplex Diffusion.

Rabeeh Karimi Mahabadi, Jaesung Tae, Hamish Ivison, James Henderson, Iz Beltagy, Matthew E. Peters, Arman Cohan

CoRR, 2023

The Semantic Scholar Open Data Platform.

Rodney Kinney, Chloe Anastasiades, Russell Authur, Iz Beltagy, Jonathan Bragg, Alexandra Buraczynski, Isabel Cachola, Stefan Candra, Yoganand Chandrasekhar, Arman Cohan, Miles Crawford, Doug Downey, Jason Dunkelberger, Oren Etzioni, Rob Evans, Sergey Feldman, Joseph Gorney, David Graham, Fangzhou Hu, Regan Huff, Daniel King, Sebastian Kohlmeier, Bailey Kuehl, Michael Langan, Daniel Lin, Haokun Liu, Kyle Lo, Jaron Lochner, Kelsey MacMillan, Tyler Murray, Chris Newell, Smita Rao, Shaurya Rohatgi, Paul Sayre, Zejiang Shen, Amanpreet Singh, Luca Soldaini, Shivashankar Subramanian, Amber Tanaka, Alex D. Wade, Linda Wagner, Lucy Lu Wang, Chris Wilhelm, Caroline Wu, Jiangjiang Yang, Angele Zamarron, Madeleine van Zuylen, Daniel S. Weld

CoRR, 2023

How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources.

Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, Hannaneh Hajishirzi

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

FiD-ICL: A Fusion-in-Decoder Approach for Efficient In-Context Learning.

Qinyuan Ye, Iz Beltagy, Matthew E. Peters, Xiang Ren, Hannaneh Hajishirzi

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023

Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations.

Xinxi Lyu, Sewon Min, Iz Beltagy, Luke Zettlemoyer, Hannaneh Hajishirzi

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023

2022

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model.

Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilic, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot, Niklas Muennighoff, Albert Villanova del Moral, Olatunji Ruwase, Rachel Bawden, Stas Bekman, Angelina McMillan-Major, Iz Beltagy, Huu Nguyen, Lucile Saulnier, Samson Tan, Pedro Ortiz Suarez, Victor Sanh, Hugo Laurençon, Yacine Jernite, Julien Launay, Margaret Mitchell, Colin Raffel, Aaron Gokaslan, Adi Simhi, Aitor Soroa, Alham Fikri Aji, Amit Alfassy, Anna Rogers, Ariel Kreisberg Nitzav, Canwen Xu, Chenghao Mou, Chris Emezue, Christopher Klamm, Colin Leong, Daniel van Strien, David Ifeoluwa Adelani, et al.

CoRR, 2022

What Language Model to Train if You Have One Million GPU Hours?

Teven Le Scao, Thomas Wang, Daniel Hesslow, Lucile Saulnier, Stas Bekman, M. Saiful Bari, Stella Biderman, Hady Elsahar, Niklas Muennighoff, Jason Phang, Ofir Press, Colin Raffel, Victor Sanh, Sheng Shen, Lintang Sutawika, Jaesung Tae, Zheng Xin Yong, Julien Launay, Iz Beltagy

CoRR, 2022

Transparency Helps Reveal When Language Models Learn Meaning.

Zhaofeng Wu, William Merrill, Hao Peng, Iz Beltagy, Noah A. Smith

CoRR, 2022

What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?

Thomas Wang, Adam Roberts, Daniel Hesslow, Teven Le Scao, Hyung Won Chung, Iz Beltagy, Julien Launay, Colin Raffel

CoRR, 2022

Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search.

Daniel King, Zejiang Shen, Nishant Subramani, Daniel S. Weld, Iz Beltagy, Doug Downey

CoRR, 2022

Infrastructure for Rapid Open Knowledge Network Development.

Michael J. Cafarella, Michael R. Anderson, Iz Beltagy, Arie Cattan, Sarah E. Chasins, Ido Dagan, Doug Downey, Oren Etzioni, Sergey Feldman, Tian Gao, Tom Hope, Kexin Huang, Sophie Johnson, Daniel King, Kyle Lo, Yuze Lou, Matthew D. Shapiro, Dinghao Shen, Shivashankar Subramanian, Lucy Lu Wang, Yuning Wang, Yitong Wang, Daniel S. Weld, Jenny M. Vo-Phamhi, Anna Zeng, Jiayun Zou

AI Mag., 2022

MultiVerS: Improving scientific claim verification with weak supervision and full-document context.

David Wadden, Kyle Lo, Lucy Lu Wang, Arman Cohan, Iz Beltagy, Hannaneh Hajishirzi

Proceedings of the Findings of the Association for Computational Linguistics: NAACL 2022, 2022

Few-Shot Self-Rationalization with Natural Language Prompts.

Ana Marasovic, Iz Beltagy, Doug Downey, Matthew E. Peters

Proceedings of the Findings of the Association for Computational Linguistics: NAACL 2022, 2022

What Language Model Architecture and Pretraining Objective Works Best for Zero-Shot Generalization?

Thomas Wang, Adam Roberts, Daniel Hesslow, Teven Le Scao, Hyung Won Chung, Iz Beltagy, Julien Launay, Colin Raffel

Proceedings of the International Conference on Machine Learning, 2022

Staged Training for Transformer Language Models.

Sheng Shen, Pete Walsh, Kurt Keutzer, Jesse Dodge, Matthew E. Peters, Iz Beltagy

Proceedings of the International Conference on Machine Learning, 2022

Continued Pretraining for Better Zero- and Few-Shot Promptability.

Zhaofeng Wu, Robert L. Logan IV, Pete Walsh, Akshita Bhagia, Dirk Groeneveld, Sameer Singh, Iz Beltagy

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022

SciFact-Open: Towards open-domain scientific claim verification.

David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Iz Beltagy, Lucy Lu Wang, Hannaneh Hajishirzi

Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2022, 2022

What Language Model to Train if You Have One Million GPU Hours?

Teven Le Scao, Thomas Wang, Daniel Hesslow, Stas Bekman, M. Saiful Bari, Stella Biderman, Hady Elsahar, Niklas Muennighoff, Jason Phang, Ofir Press, Colin Raffel, Victor Sanh, Sheng Shen, Lintang Sutawika, Jaesung Tae, Zheng Xin Yong, Julien Launay, Iz Beltagy

Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2022, 2022

PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization.

Wen Xiao, Iz Beltagy, Giuseppe Carenini, Arman Cohan

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022

Zero- and Few-Shot NLP with Pretrained Language Models.

Iz Beltagy, Arman Cohan, Robert L. Logan IV, Sewon Min, Sameer Singh

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, 2022

2021

LongChecker: Improving scientific claim verification by modeling full-abstract context.

David Wadden, Kyle Lo, Lucy Lu Wang, Arman Cohan, Iz Beltagy, Hannaneh Hajishirzi

CoRR, 2021

PRIMER: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization.

Wen Xiao, Iz Beltagy, Giuseppe Carenini, Arman Cohan

CoRR, 2021

MS2: Multi-Document Summarization of Medical Studies.

Jay DeYoung, Iz Beltagy, Madeleine van Zuylen, Bailey Kuehl, Lucy Lu Wang

CoRR, 2021

Cross-Document Language Modeling.

Avi Caciularu, Arman Cohan, Iz Beltagy, Matthew E. Peters, Arie Cattan, Ido Dagan

CoRR, 2021

FLEX: Unifying Evaluation for Few-Shot NLP.

Jonathan Bragg, Arman Cohan, Kyle Lo, Iz Beltagy

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers.

Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, Matt Gardner

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021

MS\^2: Multi-Document Summarization of Medical Studies.

Jay DeYoung, Iz Beltagy, Madeleine van Zuylen, Bailey Kuehl, Lucy Lu Wang

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021

CDLM: Cross-Document Language Modeling.

Avi Caciularu, Arman Cohan, Iz Beltagy, Matthew E. Peters, Arie Cattan, Ido Dagan

Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2021, 2021

Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study.

Rahul Nadkarni, David Wadden, Iz Beltagy, Noah A. Smith, Hannaneh Hajishirzi, Tom Hope

Proceedings of the 3rd Conference on Automated Knowledge Base Construction, 2021

SciCo: Hierarchical Cross-Document Coreference for Scientific Concepts.

Arie Cattan, Sophie Johnson, Daniel S. Weld, Ido Dagan, Iz Beltagy, Doug Downey, Tom Hope

Proceedings of the 3rd Conference on Automated Knowledge Base Construction, 2021

2020

Longformer: The Long-Document Transformer.

Iz Beltagy, Matthew E. Peters, Arman Cohan

CoRR, 2020

SciREX: A Challenge Dataset for Document-Level Information Extraction.

Sarthak Jain, Madeleine van Zuylen, Hannaneh Hajishirzi, Iz Beltagy

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020

Don't Stop Pretraining: Adapt Language Models to Domains and Tasks.

Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, Noah A. Smith

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020

SPECTER: Document-level Representation Learning using Citation-informed Transformers.

Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, Daniel S. Weld

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020

2019

SciBERT: Pretrained Contextualized Embeddings for Scientific Text.

Iz Beltagy, Arman Cohan, Kyle Lo

CoRR, 2019

Combining Distant and Direct Supervision for Neural Relation Extraction.

Iz Beltagy, Kyle Lo, Waleed Ammar

Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019

Pretrained Language Models for Sequential Sentence Classification.

Arman Cohan, Iz Beltagy, Daniel King, Bhavana Dalvi, Daniel S. Weld

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019

SciBERT: A Pretrained Language Model for Scientific Text.

Iz Beltagy, Kyle Lo, Arman Cohan

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019

ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing.

Mark Neumann, Daniel King, Iz Beltagy, Waleed Ammar

Proceedings of the 18th BioNLP Workshop and Shared Task, 2019

2018

Improving Distant Supervision with Maxpooled Attention and Sentence-Level Supervision.

Iz Beltagy, Kyle Lo, Waleed Ammar

CoRR, 2018

Machine Learning for Reliable mmWave Systems: Blockage Prediction and Proactive Handoff.

Ahmed Alkhateeb, Iz Beltagy

CoRR, 2018

Construction of the Literature Graph in Semantic Scholar.

Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, Rodney Kinney, Sebastian Kohlmeier, Kyle Lo, Tyler Murray, Hsu-Han Ooi, Matthew E. Peters, Joanna Power, Sam Skjonsberg, Lucy Lu Wang, Chris Wilhelm, Zheng Yuan, Madeleine van Zuylen, Oren Etzioni

Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018

Machine Learning for Reliable MMWAVE Systems: Blockage Prediction and Proactive Handoff.

Ahmed Alkhateeb, Iz Beltagy, Sam Alex

Proceedings of the 2018 IEEE Global Conference on Signal and Information Processing, 2018