Alexander RushAssociate Professor.Cornell University CS+Cornell Tech (NYC), Cornell NLP [arush@cornell.edu] [cv] Researcher. Hugging Face |
My research group aims to build and improve language models. Methodologically, we study data-driven methods that combine deep-learning based models with probabilistic controls. We are interested in applications in improved scaling, efficiency, model reasoning, and long-context generation.
I am also interested in open-source deep learning LLMs, and develop projects to make systems safer, more clear, and easier to use. I work part-time at Hugging Face and like to release various software projects to support NLP and DL research. I host a YouTube channel with technical talks about topics I am interested in.
Current Research Areas
- Alternative architectures for deep learning on language .
- Methods for scaling open language models.
- Efficient algorithms and hardware for language tasks.
- Security in AI systems.
Recognition
My group's work has been recognized with an NSF CAREER Award and a Sloan Fellowship. We have won paper awards at conferences for NLP, Hardware, and Visualization, as well as awards for best demonstrations for open-source software.
Selected Papers
A selection of papers that represent my research interests and style.
Simple and Effective Masked Diffusion Language Models
Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, Volodymyr Kuleshov. NeurIPS 2024 | |
Zephyr: Direct Distillation of LM Alignment
Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, Thomas Wolf. COLM 2024 | |
Pretraining Without Attention
Junxiong Wang, Jing Nathan Yan, Albert Gu, Alexander M. Rush. EMNLP 2023 Findings | |
Multitask prompted training enables zero-shot task generalization
Victor Sanh, et al.. ICLR 2022 | |
How many data points is a prompt worth?
Teven Le Scao, Alexander M. Rush. NAACL Short 2021 | |
Transformers: State-of-the-art Natural Language Processing
Thomas Wolf et al. EMNLP Demos 2020 | |
Compound Probabilistic Context-Free Grammars for Grammar Induction
Yoon Kim, Chris Dyer, Alexander M. Rush. ACL 2019 | |
Learning Neural Templates for Text Generation
Sam Wiseman, Stuart M. Shieber, Alexander Rush. EMNLP 2018 | |
LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks
Hendrik Strobelt, Sebastian Gehrmann, Hanspeter Pfister, and Alexander M. Rush. InfoVis 2017 | |
OpenNMT: Open-Source Toolkit for Neural Machine Translation
Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, Alexander M. Rush. ACL Demo 2017 | |
Sequence-Level Knowledge Distillation
Yoon Kim and Alexander M. Rush. EMNLP 2016 | |
Character-Aware Neural Language Models
Yoon Kim, Yacine Jernite, David Sontag, and Alexander M. Rush. AAAI 2016 | |
A Neural Attention Model for Abstractive Sentence Summarization
Alexander M. Rush, Sumit Chopra, and Jason Weston. EMNLP 2015. |