Publications

Papers

My full list of papers is here.

	Compute-Constrained Data Selection J. O. Yin, Alexander Rush. Preprint 2024
	Contextual Document Embeddings J. X. Morris, Alexander Rush. Preprint 2024
	A controlled study on long context extension and generalization in llms Y. Lu, J. N. Yan, S. Yang, J. T. Chiu, S. Ren, F. Yuan, W. Zhao, Z. Wu, Alexander Rush. Preprint 2024
	The mamba in the llama: Distilling and accelerating hybrid models J. Wang, D. Paliotta, A. May, Alexander Rush, T. Dao. NeurIPS 2024
	Great Memory, Shallow Reasoning: Limits of NN-LMs S. Geng, W. Zhao, Alexander Rush. Preprint 2024
	Predicting text preference via structured comparative reasoning J. N. Yan, T. Liu, J. Chiu, J. Shen, Z. Qin, Y. Yu, C. Lakshmanan, Y. Kurzion, Alexander Rush. ACL 2024
	I Could've Asked That: Reformulating Unanswerable Questions W. Zhao, G. Gao, C. Cardie, Alexander Rush. EMNLP 2024
	ShadowLLM: Predictor-based Contextual Sparsity for Large Language Models Yash Akhauri, Ahmed F AbouElhamayed, Jordan Dotzel, Zhiru Zhang, Alexander M Rush, Safeen Huda, Mohamed S Abdelfattah. EMNLP 2024
	Simple and Effective Masked Diffusion Language Models Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, Volodymyr Kuleshov. NeurIPS 2024
	Entity disambiguation via fusion entity decoding Junxiong Wang, Ali Mousavi, Omar Attia, Ronak Pradeep, Saloni Potdar, Alexander M Rush, Umar Farooq Minhas, Yunyao Li. NAACL 2024
	MambaByte: Token-free Selective State Space Model Junxiong Wang, Tushaar Gangavarapu, Jing Nathan Yan, Alexander M. Rush. COLM 2024
	Zephyr: Direct Distillation of LM Alignment Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, Thomas Wolf. COLM 2024
	Guess and Sketch: Language Model Guided Transpilation Celine Lee, Abdulrahman Mahmoud, Michal Kurek, Simone Campanoni, David Brooks, Stephen Chong, Gu-Yeon Wei, Alexander M. Rush. ICLR 2024
	Symbolic Planning and Code Generation for Grounded Dialogue Justin T. Chiu, Wenting Zhao, Derek Chen, Saujas Vaduguru, Alexander M. Rush, Daniel Fried. EMNLP 2023
	Teal: Learning-Accelerated Optimization of WAN Traffic Engineering Zhiying Xu, Francis Y. Yan, Rachee Singh, Justin T. Chiu, Alexander M. Rush, Minlan Yu. SIGCOMM 2023
	Text Embeddings Reveal (Almost) As Much As Text John X. Morris, Volodymyr Kuleshov, Vitaly Shmatikov, Alexander M. Rush. EMNLP 2023
	Tree Prompting: Efficient Task Adaptation without Fine-Tuning John X. Morris, Chandan Singh, Alexander M. Rush, Jianfeng Gao, Yuntian Deng. EMNLP 2023
	HOP, UNION, GENERATE: Explainable Multi-hop Reasoning without Rationale Supervision Wenting Zhao, Justin T. Chiu, Claire Cardie, Alexander M. Rush. EMNLP 2023
	Pretraining Without Attention Junxiong Wang, Jing Nathan Yan, Albert Gu, Alexander M. Rush. EMNLP 2023 Findings
	Scaling Data-Constrained Language Models Niklas Muennighoff, Alexander M. Rush, Boaz Barak, Teven Le Scao, Aleksandra Piktus, Nouamane Tazi, Sampo Pyysalo, Thomas Wolf, Colin Raffel. NeurIPS 2023 (Oral)
	OBELISC: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents Hugo Laurençon, Lucile Saulnier, Léo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth Karamcheti, Alexander M. Rush, Douwe Kiela, Matthieu Cord, Victor Sanh. NeurIPS 2023 Dataset
	Abductive Commonsense Reasoning Exploiting Mutually Exclusive Explanations Wenting Zhao, Justin T. Chiu, Claire Cardie, Alexander M. Rush. ACL 2023
	Markup-to-Image Diffusion Models with Scheduled Sampling Yuntian Deng, Noriyuki Kojima, Alexander M. Rush. ICLR 2023
	A 12nm 18.1TFLOPs/W Sparse Transformer Processor with Entropy-Based Early Exit, Mixed-Precision Predication and Fine-Grained Power Management Thierry Tambe, Jeff Zhang, Coleman Hooper, Tianyu Jia, Paul N. Whatmough, Joseph Zuckerman, Maico Cassel dos Santos, Erik Jens Loscalzo, Davide Giri, Kenneth L. Shepard, Luca P. Carloni, Alexander M. Rush, David Brooks, Gu-Yeon Wei. ISSCC 2023
	BLOOM: A 176B-Parameter Open-Access Multilingual Language Model BigScience Workshop. Arxiv Preprint
	Named Tensor Notation David Chiang, Alexander M. Rush, Boaz Barak. TMLR 2022
	Xatu: boosting existing DDoS detection systems using auxiliary signals Zhiying Xu, Sivaramakrishnan Ramanathan, Alexander Rush, Jelena Mirkovic, Minlan Yu. CoNEXT 2022
	Unsupervised Text Deidentification John X Morris, Justin T Chiu, Ramin Zabih, Alexander M Rush. EMNLP Findings 2022
	Model Criticism for Long-Form Text Generation Yuntian Deng, Volodymyr Kuleshov, Alexander M Rush. EMNLP 2022
	Evaluate and Evaluation on the Hub: Better Best Practices for Data and Model Measurement Leandro von Werra et al.. EMNLP Demos 2022 (Best Demo)
	Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Models Hendik Strobelt et al.. IEEE Trans on Visualization 2022
	A 16-nm SoC for Noise-Robust Speech and NLP Edge AI Inference With Bayesian Sound Source Separation and Attention-Based DNNs Thierry Tambe et al.. IEEE Solid-State Circuits 2022
	Promptsource: An integrated development environment and repository for natural language prompts Stephen Bach et al.. ACL Demo 2022
	End-to-end learning of multiple sequence alignments with differentiable Smith-Waterman Samantha Petti, et al.. Bioinformatics
	Multitask prompted training enables zero-shot task generalization Victor Sanh, et al.. ICLR 2022
	Developmental Stage Classification of Embryos Using Two-Stream Neural Network with Linear-Chain Conditional Random Field Stanislav Lukyanenko et al.. MICCAI 2021
	Rationales for sequential predictions Keyon Vafa, Yuntian Deng, David Blei, Alexander Rush. EMNLP 2021
	Low-Rank Constraints for Fast Inference in Structured Models Justin Chiu, Yuntian Deng, and Alexander M. Rush. NeurIPS 2021
	Conference demographics and footprint changed by virtual platforms Matthe Skiles et al.. Nature Sustainability
	Sequence-to-Lattice Models for Fast Translation Yuntian Deng and Alexander M. Rush. EMNLP Findings Short 2021
	Datasets: A Community Library for Natural Language Processing Quentin Lhoest et al. EMNLP Demos 2021 (Best Demo)
	EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference Thierry Tambe and Others. IEEE MICRO 2021
	GenNI: Human-AI Collaboration for Data-Backed Text Generation Hendrik Strobelt, Jambay Kinley, Robert Krueger, Johanna Beyer, Alexander M. Rush, Hanspeter Pfister. IEEE VIS 2021
	Parameter-efficient transfer learning with diff pruning Demi Guo, Alexander M. Rush, Yoon Kim. ACL 2021
	How many data points is a prompt worth? Teven Le Scao, Alexander M. Rush. NAACL Short 2021 (Best Paper - Runner-Up)
	Block pruning for faster transformers François Lagunas, Ella Charlaix, Victor Sanh, Alexander M Rush. ACL 2021
	Low-Complexity Probing via Finding Subnetworks Steven Cao, Victor Sanh, Alexander M. Rush. NAACL Short 2021
	Template Filling with Generative Transformers Xinya Du, Alexander M. Rush, Claire Cardie. NAACL Short 2021
	9.8 A 25mm2 SoC for IoT Devices with 18ms Noise-Robust Speech-to-Text Latency via Bayesian Speech Denoising and Attention-Based Sequence-to-Sequence DNN Speech Recognition in 16nm FinFET Thierry Tambe, En-Yu Yang, Glenn G Ko, Yuji Chai, Coleman Hooper, Marco Donato, Paul N Whatmough, Alexander M Rush, David Brooks, Gu-Yeon Wei. IEEE International Solid-State Circuits Conference 2021
	Cascaded Text Generation with Markov Transformers Yuntian Deng, Alexander M. Rush. NeurIPS 2020
	Latent Template Induction with Gumbel-CRFs Yao Fu, Chuanqi Tan, Bin Bi, Mosha Chen, Yansong Feng, Alexander Rush. NeurIPS 2020
	Movement Pruning: Adaptive Sparsity by Fine-Tuning Victor Sanh, Thomas Wolf, Alexander M. Rush. NeurIPS 2020
	Scaling Hidden Markov Language Models Justin T. Chiu, Alexander M. Rush. EMNLP 2020
	Adversarial Semantic Collisions Congzheng Song, Alexander M. Rush, Vitaly Shmatikov. EMNLP 2020
	Sequence-Level Mixed Sample Data Augmentation Demi Guo, Yoon Kim, Alexander M. Rush. EMNLP 2020
	AdaptivFloat: A Floating-point based Data Type for Resilient Deep Learning Inference Thierry Tambe, En-Yu Yang, Zishen Wan, Yuntian Deng, Vijay Janapa Reddi, Alexander Rush, David Brooks, Gu-Yeon Wei. DAC 2020 (Best Paper)
	Transformers: State-of-the-art Natural Language Processing Thomas Wolf et al. EMNLP Demos 2020 (Best Demo)
	Torch-Struct: Deep Structured Prediction Library Alexander Rush. ACL Demos 2020 (Best Demo Honorable Mention)
	What is Learned in Visually Grounded Neural Syntax Acquisition Noriyuki Kojima, Hadar Averbuch-Elor, Alexander M. Rush, Yoav Artzi. ACL 2020 (Short)
	Posterior Control of Blackbox Generation Xiang Lisa Li, Alexander M. Rush. ACL 2020
	Automating Botnet Detection with Graph Neural Networks Jiawei Zhou, Zhiying Xu, Alexander M. Rush, Minlan Yu. AutoML for Networking and Systems Workshop
	LAN -- A materials notation for 2D layered assemblies Georgios A. Tritsaris, Yiqi Xie, Alexander M. Rush, Stephen Carr, Marios Mattheakis, Efthimios Kaxiras.
	MASR: A Modular Accelerator for Sparse RNNs Udit Gupta, Brandon Reagen, Lillian Pentecost, Marco Donato, Thierry Tambe, Alexander M. Rush, Gu-Yeon Wei, David Brooks. PACT 2019
	Commonsense Knowledge Mining from Pretrained Models Joe Davison, Joshua Feldman and Alexander Rush. EMNLP 2019
	Neural Linguistic Steganography Zachary Ziegler, Yuntian Deng and Alexander Rush. EMNLP 2019
	Compound Probabilistic Context-Free Grammars for Grammar Induction Yoon Kim, Chris Dyer, Alexander M. Rush. ACL 2019
	Visual Interaction with Deep Learning Models through Collaborative Semantic Inference Gehrmann S, Strobelt H, Krueger R, Pfister H, and Alexander M. Rush. InfoVis 2019
	Simple Unsupervised Summarization by Contextual Matching Jiawei Zhou, Alexander M. Rush. ACL 2019
	GLTR: Statistical Detection and Visualization of Generated Text Sebastian Gehrmann, Hendrik Strobelt, Alexander M Rush. ACL Demo 2019 (Best Demo Honorable Mention)
	Unsupervised Recurrent Neural Network Grammars Yoon Kim, Alexander M. Rush, Lei Yu, Adhiguna Kuncoro, Chris Dyer, Gabor Melis. NAACL 2019
	Avoiding Latent Variable Collapse With Generative Skip Models Adji B. Dieng, Yoon Kim, Alexander M. Rush, David M. Blei. AISTATS 2019
	Tensor Variable Elimination for Plated Factor Graphs Fritz Obermeyer, Eli Bingham, Martin Jankowiak, Justin Chiu, Neeraj Pradhan, Alexander Rush, Noah Goodman. ICML 2019
	Latent Normalizing Flows for Discrete Sequences Zachary M. Ziegler, Alexander M. Rush. ICML 2019
	Deep Latent-Variable Models for Natural Language Yoon Kim, Sam Wiseman, Alexander M. Rush. EMNLP 2018 (Tutorial)
	End-to-End Content and Plan Selection for Data-to-Text Generation Sebastian Gehrmann, Falcon Z. Dai, Henry Elder, Alexander M. Rush. INLG 2018
	Latent Alignment and Variational Attention Yuntian Deng, Yoon Kim, Justin Chiu, Demi Guo, Alexander M. Rush. NIPS 2018
	Learning Neural Templates for Text Generation Sam Wiseman, Stuart M. Shieber, Alexander Rush. EMNLP 2018
	Bottom-Up Abstractive Summarization Sebastian Gehrmann, Yuntian Deng, Alexander Rush. EMNLP 2018
	Training for Diversity in Image Paragraph Captioning Luke Melas-Kyriazi, George Han, Alexander Rush. EMNLP 2018 (Short)
	Entity Tracking Improves Cloze-style Reading Comprehension Luong Hoang, Sam Wiseman, Alexander Rush. EMNLP 2018 (Short)
	Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models Hendrik Strobelt, Sebastian Gehrmann, Michael Behrisch, Adam Perer, Hanspeter Pfister, Alexander M. Rush. VAST 2018, EMNLP-BlackBox 2018 (Best Paper - Honorable Mention)
	The Annotated Transformer Alexander M. Rush. ACL NLP-OSS 2018
	OpenNMT System Description for WNMT 2018: 800 words/sec on a single-core CPU Jean Senellart, Dakun Zhang, Bo Wang, Guillaume Klein, J.P. Ramatchandirin, Josep Crego, Alexander M. Rush. WNMT 2018 (First-Place CPU Speed/Memory)
	Semi-Amortized Variational Autoencoders Yoon Kim, Sam Wiseman, Andrew C. Miller, David Sontag, Alexander M. Rush. ICML 2018
	Compressing Deep Neural Networks with Probabilistic Data Structures Brandon Reagen, Udit Gupta, Robert Adolf, Michael M. Mitzenmacher, Alexander M. Rush, Gu-Yeon Wei, David Brooks. ICML 2018, SysML 2018
	Adapting Sequence Models for Sentence Correction Allen Schmaltz, Yoon Kim, Alexander M. Rush, Stuart M. Shieber. EMNLP 2017
	Challenges in Data-to-Document Generation Sam Wiseman, Stuart M Shieber Alexander M. Rush. EMNLP 2017
	Adversarially Regularized Autoencoders Junbo Zhao, Yoon Kim, Kelly Zhang, Alexander M. Rush, Yann LeCun. ICML 2018, NIPS 2017 Workshop
	OpenNMT: Open-Source Toolkit for Neural Machine Translation Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, Alexander M. Rush. ACL Demo 2017 (Best Demo Runner-up)
	Dilated Convolutions for Modeling Long-Distance Genomic Dependencies Ankit Gupta, Alexander M. Rush. ICML CompBio 2017 (Best Poster)
	Image-to-Markup Generation with Coarse-to-Fine Attention Yuntian Deng, Anssi Kanervisto, Jeffrey Ling, and Alexander M. Rush. ICML 2017
	LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks Hendrik Strobelt, Sebastian Gehrmann, Hanspeter Pfister, and Alexander M. Rush. InfoVis 2017
	Structured Attention Networks Yoon Kim, Carl Denton, Luong Hoang, and Alexander M. Rush. ICLR 2017
	Lie-Access Neural Turing Machines Greg Yang and Alexander M. Rush. ICLR 2017
	Sequence-Level Knowledge Distillation Yoon Kim and Alexander M. Rush. EMNLP 2016
	Sequence-to-Sequence Learning as Beam-Search Optimization Sam Wiseman and Alexander M. Rush. EMNLP 2016 (Best Paper Runner-Up)
	An Embedding Model for Predicting Roll-Call Votes Peter Kraft, Hirsh Jain, and Alexander M. Rush. Proceedings of EMNLP 2016
	Word Ordering Without Syntax Allen Schmaltz, Alexander M. Rush, and Stuart M. Shieber. EMNLP 2016
	Sentence-Level Grammatical Error Identification as Sequence-to-Sequence Correction Allen Schmaltz, Yoon Kim, Alexander M. Rush, and Stuart M. Shieber. Workshop Submission for AESW 2016 (Top Performing System)
	Learning Global Features for Coreference Resolution Sam Wiseman, Alexander M. Rush, and Stuart M. Shieber. NAACL 2016
	Abstractive Sentence Summarization with Attentive Recurrent Neural Networks Sumit Chopra, Michael Auli, and Alexander M. Rush. NAACL 2016
	Character-Aware Neural Language Models Yoon Kim, Yacine Jernite, David Sontag, and Alexander M. Rush. AAAI 2016
	A Neural Attention Model for Abstractive Sentence Summarization Alexander M. Rush, Sumit Chopra, and Jason Weston. EMNLP 2015.
	Towards AI-Complete Question Answering A Set of Prerequisite Toy Tasks Jason Weston, Antoine Bordes, Sumit Chopra, Tomas Mikolov, and Alexander M. Rush. ArXiv Preprint
	Learning Anaphoricity and Antecedent Ranking Features for Coreference Resolution Sam Wiseman, Alexander M. Rush, Jason Weston, and Stuart M. Shieber. ACL 2015.
	A Fast Variational Approach for Learning Markov Random Field Language Models Yacine Jernite, Alexander M. Rush, and David Sontag. ICML 2015.
	Transforming Dependencies into Phrase Structures Lingpeng Kong, Alexander M. Rush, and Noah A. Smith. NAACL 2015.