Publications

* and ^ represent equal-contribution groups.

Non-Archival Tech Reports

  1. ๐Ÿฆ WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild
    Bill Yuchen Lin, Yuntian Deng, Khyathi Chandu, Faeze Brahman, Abhilasha Ravichander, Valentina Pyatkin, Nouha Dziri, Ronan Le Bras, Yejin Choi
    ๐Ÿขใ€€arXiv
    [๐Ÿค— Leaderboard] [๐Ÿ’พ Github] [๐Ÿฆ Tweet]

  2. ๐Ÿ† RewardBench: Evaluating Reward Models for Language Modeling
    Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, Hannaneh Hajishirzi
    ๐Ÿขใ€€arXiv

  3. ๐Ÿ”ฅ Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models
    Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, Minjoon Seo
    ๐Ÿขใ€€arXiv

  4. ๐Ÿ•ธ๏ธ VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?
    Junpeng Liu, Yifan Song, Bill Yuchen Lin, Wai Lam, Graham Neubig, Yuanzhi Li, Xiang Yue
    ๐Ÿขใ€€arXiv

  5. โš›๏ธ LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition
    โœ๏ธใ€€Chengsong Huang*, Qian Liu*, Bill Yuchen Lin*, Tianyu Pang, Chao Du, Min Lin
    ๐Ÿขใ€€arXiv
    [๐Ÿ’พ Demo] [๐Ÿ’พ Github] [๐Ÿฆ Tweet]

  6. ๐Ÿงฉ L3GO: Language Agents with Chain-of-3D-Thoughts for Generating Unconventional Objects
    โœ๏ธใ€€Yutaro Yamada, Khyathi Chandu, Bill Yuchen Lin, Jack Hessel, Ilker Yildirim, Yejin Choi
    ๐Ÿขใ€€arXiv

  7. ๐Ÿ’ป OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement
    โœ๏ธใ€€Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, Xiang Yue
    ๐Ÿขใ€€arXiv
    [๐Ÿ’พ Website] [๐Ÿ’พ Demo] [๐Ÿ’พ Code] [๐Ÿ’พ Models]

  8. ๐Ÿฒ Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging
    โœ๏ธใ€€Joel Jang, Seungone Kim, Bill Yuchen Lin, Yizhong Wang, Jack Hessel, Luke Zettlemoyer, Hannaneh Hajishirzi, Yejin Choi, Prithviraj Ammanabrolu
    ๐Ÿขใ€€arXiv
    [๐Ÿ’พ Github]

  9. ๐Ÿƒ Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT4
    Jiaxian Guo*, Bo Yang*, Paul Yoo, Bill Yuchen Lin, Yusuke Iwasawa, Yutaka Matsuo
    ๐Ÿขใ€€arXiv
    [๐Ÿ’พ Github] [๐Ÿฆ Tweet]

  10. ๐Ÿ‘€ Selective โ€œSelective Predictionโ€: Reducing Unnecessary Abstention in Vision-Language Reasoning
    Tejas Srinivasan, Jack Hessel, Tanmay Gupta, Bill Yuchen Lin, Yejin Choi, Jesse Thomason, Khyathi Raghavi Chandu
    ๐Ÿขใ€€arXiv

2024

  1. ๐Ÿช„ Agent Lumos: Unified and Modular Training for Open-Source Language Agents
    โœ๏ธใ€€Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, Bill Yuchen Lin
    ๐Ÿขใ€€ACL 2024 Main Conference
    [๐Ÿ“ƒ Website] [๐Ÿ’พ Github] [๐Ÿฆ Tweet]

  2. ๐Ÿค– Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents
    โœ๏ธใ€€Yifan Song, Da Yin, Xiang Yue, Jie Huang, Sujian Li, Bill Yuchen Lin
    ๐Ÿขใ€€ACL 2024 Main Conference
    [๐Ÿ’พ Github]

  3. ๐Ÿ›ก๏ธ SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding
    โœ๏ธใ€€Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, Radha Poovendran
    ๐Ÿขใ€€ACL 2024 Main Conference
    [๐Ÿ’พ Github]

  4. ๐Ÿฏ TIGERScore: Towards Building Explainable Metric for All Text Generation Tasks
    โœ๏ธใ€€Dongfu Jiang*, Yishan Li*, Ge Zhang, Wenhao Huang, Bill Yuchen Lin, Wenhu Chen
    ๐Ÿขใ€€TMLR
    [๐Ÿ’พ Github] [๐Ÿ“ƒ Website] [๐Ÿฆ Tweet]

  5. ๐Ÿ The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning
    โœ๏ธใ€€Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, Yejin Choi
    ๐Ÿขใ€€ICLR 2024
    [๐Ÿ“ƒ Website] [๐Ÿ’พ Github]

2023

  1. ๐Ÿ”ฅ SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks
    โœ๏ธใ€€Bill Yuchen Lin, Yicheng Fu, Karina Yang, Prithviraj Ammanabrolu, Faeze Brahman, Shiyu Huang, Chandra Bhagavatula, Yejin Choi, Xiang Ren
    ๐Ÿขใ€€NeurIPS 2023 (spotlight)
    [๐Ÿ“ƒ Website] [๐Ÿ’พ Github] [๐Ÿฆ Tweet] [๐Ÿ“ฐ Blog]

  2. ๐Ÿ”ฅ Faith and Fate: Limits of Transformers on Compositionality
    โœ๏ธใ€€Nouha Dziri*, Ximing Lu*, Melanie Sclar*, Xiang Lorraine Li^, Liwei Jiang^, Bill Yuchen Lin^,
    Peter West, Chandra Bhagavatula, Ronan Le Bras,Jena Hwang,Soumya Sanyal,Sean Welleck,Xiang Ren, Allyson Ettinger, Zaid Harchaoui, Yejin Choi
    ๐Ÿขใ€€NeurIPS 2023 (spotlight)
    [๐Ÿฆ Tweet]

  3. ๐Ÿ”ฅ LLM-Blender: Ensembling Large Language Models with Pairwise Comparison and Generative Fusion
    โœ๏ธใ€€Dongfu Jiang, Xiang Ren, Bill Yuchen Lin
    ๐Ÿขใ€€to appear in Proc. of ACL 2023
    [๐Ÿ“ƒ Website] [๐Ÿ’พ Github] [๐Ÿฆ Tweet]
    Media coverage : MarkTechPost

  4. ๐Ÿบ Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuning
    โœ๏ธใ€€Ximing Lu, Faeze Brahman, Peter West, Jaehun Jang, Khyathi Chandu, Abhilasha Ravichander, Lianhui Qin, Prithviraj Ammanabrolu,
    Liwei Jiang, Sahana Ramnath, Nouha Dziri, Jillian Fisher, Bill Yuchen Lin, Skyler Hallinan, Xiang Ren, Sean Welleck, Yejin Choi
    ๐Ÿขใ€€EMNLP 2023 (Main)

  5. NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation
    โœ๏ธใ€€Peter West, Ronan Le Bras, Taylor Sorensen, Bill Yuchen Lin, Liwei Jiang, Ximing Lu, Khyathi Chandu, Jack Hessel, Ashutosh Baheti, Chandra Bhagavatula, Yejin Choi
    ๐Ÿขใ€€EMNLP 2023 (Findings)

  6. On Grounded Planning for Embodied Tasks with Language Models
    โœ๏ธใ€€Bill Yuchen Lin*, Chengsong Huang*, Qian Liu, Wenda Gu, Sam Sommerer, Xiang Ren
    ๐Ÿขใ€€in Proc. of AAAI 2023
    [๐Ÿ“ƒ Website] [๐Ÿ’พ Github] [๐Ÿค— Data]
    Media coverage : USC Viterbi News

  7. AutoTriggER: Named Entity Recognition with Auxiliary Trigger Extraction
    โœ๏ธใ€€Dong-Ho Lee, Ravi Kiran Selvam, Sheikh Muhammad Sarwar, Bill Yuchen Lin,
    Mahak Agarwal, Fred Morstatter, Jay Pujara, Elizabeth Boschee, James Allan, Xiang Ren
    ๐Ÿขใ€€in Proc. of EACL 2023, also presented at TrustNLP @ NAACL 2021 (best paper award)

  8. Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
    โœ๏ธใ€€442 authors including Bill Yuchen Lin
    ย [๐Ÿ’พ Github]
    ๐Ÿขใ€€in TMLR

2022

  1. Reflect, Not Reflex: Inference-Based Common Ground Improves Dialogue Response Quality
    โœ๏ธใ€€Pei Zhou, Hyundong J. Cho, Pegah Jandaghi, Dong-Ho Lee, Bill Yuchen Lin, Jay Pujara, Xiang Ren
    ๐Ÿขใ€€in Proc. of EMNLP 2022
    [๐Ÿฆ Tweet]

  2. Unsupervised Cross-Task Generalization via Retrieval Augmentation
    โœ๏ธใ€€Bill Yuchen Lin, Kangmin Tan, Chris Miller, Beiwen Tian, Xiang Ren
    ๐Ÿขใ€€in Proc. of NeurIPS 2022
    ย [๐Ÿ“ƒ Website] ย [๐Ÿ’พ Github] [๐Ÿ–ผ๏ธ Slides] ย [๐ŸŽฆ Video] [๐Ÿฆ Tweet]

  3. On Continual Model Refinement in Out-of-Distribution Data Streams
    โœ๏ธใ€€Bill Yuchen Lin, Sida Wang, Xi Victoria Lin, Robin Jia, Lin Xiao, Xiang Ren, Scott Yih
    ๐Ÿขใ€€in Proc. of ACL 2022
    ย [๐Ÿ“ƒ Website] ย [๐Ÿ’พ Github] [๐Ÿ–ผ๏ธ Slides] ย [๐ŸŽฆ Video] [๐Ÿฆ Tweet]

  4. FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks
    โœ๏ธใ€€Bill Yuchen Lin*, Chaoyang He*, Zihang Zeng, Hulin Wang, Yufen Huang, Mahdi Soltanolkotabi, Xiang Ren^, Salman Avestimehr^
    ๐Ÿขใ€€in Proc. of NAACL 2022 Findings
    [๐Ÿ’พ Github] [๐Ÿฆ Tweet]

  5. On the Robustness of Reading Comprehension Models to Entity Renaming
    โœ๏ธใ€€Jun Yan, Yang Xiao, Sagnik Mukherjee, Bill Yuchen Lin, Robin Jia, Xiang Ren
    ๐Ÿขใ€€in Proc. of NAACL 2022

2021

  1. CrossFit: A Few-shot Learning Challenge for Cross-Task Generalization in NLP
    โœ๏ธใ€€Qinyuan Ye, Bill Yuchen Lin, Xiang Ren
    ๐Ÿขใ€€in Proc. of EMNLP 2021
    [๐Ÿ’พ Github] [๐Ÿฆ Tweet]

  2. Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning
    โœ๏ธใ€€Xisen Jin, Bill Yuchen Lin, Mohammad Rostami, Xiang Ren
    ๐Ÿขใ€€in Proc. of EMNLP 2021 Findings
    [๐Ÿ’พ Github]

  3. RockNER: A Simple Method to Create Adversarial Examples for Evaluating the Robustness of NER Models
    โœ๏ธใ€€Bill Yuchen Lin, Wenyang Gao, Jun Yan, Ryan Moreno, Xiang Ren
    ๐Ÿขใ€€in Proc. of EMNLP 2021 (short)
    ย [๐Ÿ“ƒ Website]

  4. RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms
    โœ๏ธใ€€Pei Zhou, Rahul Khanna, Seyeon Lee, Bill Yuchen Lin, Daniel Ho, Jay Pujara, Xiang Ren
    ๐Ÿขใ€€in Proc. of EMNLP 2021
    ย [๐Ÿ“ƒ Website]

  5. Probing Commonsense Explanation in Dialogue Response Generation
    โœ๏ธใ€€Pei Zhou, Pegah Jandaghi, Hyundong Cho, Bill Yuchen Lin, Jay Pujara, Xiang Ren
    ๐Ÿขใ€€in Proc. of EMNLP 2021 Findings

  6. Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoning
    โœ๏ธใ€€Bill Yuchen Lin, Seyeon Lee, Xiaoyang Qiao, Xiang Ren
    ๐Ÿขใ€€in Proc. of ACL 2021
    [๐Ÿ’พ Github] ย ย [๐Ÿ“ƒ Website]

  7. RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledge
    โœ๏ธใ€€Bill Yuchen Lin, Ziyi Wu, Yichi Yang, Dong-Ho Lee, Xiang Ren
    ๐Ÿขใ€€in Proc. of ACL 2021 Findings
    [๐Ÿ’พ Github] ย ย [๐Ÿ“ƒ Website]

  8. Differentiable Open-Ended Commonsense Reasoning
    โœ๏ธใ€€Bill Yuchen Lin, Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Xiang Ren, William W. Cohen
    ๐Ÿขใ€€in Proc. of NAACL 2021
    [๐Ÿ–ผ๏ธ Slides] ย ย [๐ŸŽฆ Video] ย ย [๐Ÿ’พ Github] ย ย [๐Ÿ“ƒ Website]

  9. Pre-training Text-to-Text Transformers for Concept-Centric Common Sense
    โœ๏ธใ€€Wangchunshu Zhou, Dong-Ho Lee, Ravi Kiran Selvam, Seyeon Lee, Bill Yuchen Lin, Xiang Ren
    ๐Ÿขใ€€in Proc. of ICLR 2021 ย 
    [๐Ÿ’พ Github]

  10. IsoBN: Fine-Tuning BERT with Isotropic Batch Normalization
    โœ๏ธใ€€Wenxuan Zhou, Bill Yuchen Lin, Xiang Ren
    ๐Ÿขใ€€in Proc. of AAAI 2021

2020

  1. CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning
    โœ๏ธใ€€Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi, Xiang Ren
    ๐Ÿขใ€€in Proc. of EMNLP 2020 Findings ย ย  (presented at AKBC 2020 as a non-archival paper.)
    [๐Ÿ“ƒ Website]
    Media coverage : The Register , Tech Xplore , Techzine , Radio.com , ScienceDaily , USC Viterbi

  2. Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models
    โœ๏ธใ€€Bill Yuchen Lin, Seyeon Lee, Rahul Khanna, Xiang Ren
    ๐Ÿขใ€€in Proc. of EMNLP 2020 (short)
    [๐Ÿ“ƒ Website]

  3. Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering
    โœ๏ธใ€€Yanlin Feng*, Xinyue Chen*, Bill Yuchen Lin, Peifeng Wang, Jun Yan, Xiang Ren
    ๐Ÿขใ€€in Proc. of EMNLP 2020
    [๐Ÿ’พ Github]

  4. FreeDOM: A Transferable Neural Architecture for Structured Information Extraction on Web Documents
    โœ๏ธใ€€Bill Yuchen Lin, Ying Sheng, Nguyen Vo and Sandeep Tata
    ๐Ÿขใ€€in Proc. of KDD 2020 (Research Track)
    [๐Ÿ–ผ๏ธ Slides] [๐ŸŽฆ Video]
  5. TriggerNER: Learning with Entity Triggers as Explanations for Named Entity Recognition
    โœ๏ธใ€€Bill Yuchen Lin*, Dongho Lee*, Ming Shen, Ryan Moreno, Xiao Huang, Prashant Shiralkar, Xiang Ren
    ๐Ÿขใ€€in Proc. of ACL 2020 (short)
    [๐Ÿ–ผ๏ธ Slides] ย ย [๐ŸŽฆ Video] ย ย [๐Ÿ’พ Github] ย ย [๐Ÿ“ƒ Website]

  6. Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labeling.
    โœ๏ธใ€€Ouyu Lan, Xiao Huang, Bill Yuchen Lin, He Jiang, Liyuan Liu, Xiang Ren
    ๐Ÿขใ€€in Proc. of ACL 2020
    [๐Ÿ’พ Github]

  7. LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation
    โœ๏ธใ€€Dong-Ho Lee, Rahul Khanna, Bill Yuchen Lin, Jamin Chen, Seyeon Lee, Qinyuan Ye, Elizabeth Boschee, Leonardo Neves, Xiang Ren
    ๐Ÿขใ€€in Proc. of ACL 2020 (Demo Track)
    [๐Ÿ“ƒ Website]

  8. NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extraction.
    โœ๏ธใ€€Wenxuan Zhou, Hongtao Lin, Bill Yuchen Lin, Ziqi Wang, Junyi Du, Leonardo Neves, Xiang Ren
    ๐Ÿขใ€€in Proc. of TheWebConf (WWW) 2020
    Best Paper Runner-up (2/1500+) ย ย  [๐Ÿ’พ Github]

2019

  1. KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning.
    โœ๏ธใ€€Bill Yuchen Lin, Xinyue Chen, Jamin Chen, Xiang Ren
    ๐Ÿขใ€€in Proc. of EMNLP-IJCNLP 2019
    [๐Ÿ’พ Github]
  2. AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging.
    โœ๏ธใ€€Bill Yuchen Lin*, Dongho Lee*, Frank F. Xu, Ouyu Lan, Xiang Ren
    ๐Ÿขใ€€in Proc. of ACL 2019 (Demo Track)
    [๐Ÿ“ƒ Website]

2018

  1. Neural Adaptation Layers for Cross-domain Named Entity Recognition.
    โœ๏ธใ€€Bill Yuchen Lin, Wei Lu
    ๐Ÿขใ€€in Proc. of EMNLP 2018
    [๐Ÿ’พ Github]
  2. ExtRA: Extracting Prominent Review Aspects from Customer Feedback.
    โœ๏ธใ€€Zhiyi Luo, Shanshan Huang, Frank F. Xu, Bill Yuchen Lin, Hanyuan Shi, Kenny Q. Zhu
    ๐Ÿขใ€€in Proc. of EMNLP 2018
    [๐Ÿ’พ Github]
  3. Mining Cross-Cultural Differences and Similarities in Social Media.
    โœ๏ธใ€€Bill Yuchen Lin*, Frank F. Xu*, Kenny Q. Zhu, Seung-won Hwang
    ๐Ÿขใ€€in Proc. of ACL 2018
    [๐Ÿ’พ Github]
  4. Automatic Extraction of Commonsense LocatedNear Knowledge.
    โœ๏ธใ€€Frank F. Xu*, Bill Yuchen Lin*, Kenny Q. Zhu
    ๐Ÿขใ€€in Proc. of ACL 2018 (short)
    [๐Ÿ’พ Github]

2017

  1. Multi-channel BiLSTM-CRF Model for Emerging Named Entity Recognition in Social Media.
    โœ๏ธใ€€Bill Y. Lin*, Frank F. Xu*, Zhiyi Luo, Kenny Q. Zhu
    ๐Ÿขใ€€in Proc. of EMNLP 2017, Workshop on Noisy User-generated Text
    [๐Ÿ’พ Github]