Publications

* and ^ represent equal-contribution groups.

Non-Archival Tech Reports

1. 🐦‍⬛ Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing
   Zhangchen Xu, Fengqing Jiang, Luyao Niu, Yuntian Deng, Radha Poovendran, Yejin Choi, Bill Yuchen Lin
   🏢　arXiv
   [💻 Website] [🤗 HF] [🤗 Demo (by @davanstrien)] [💾 Github] [🐦 Tweet]

2. 👀 WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences
   Yujie Lu, Dongfu Jiang, Wenhu Chen, William Yang Wang, Yejin Choi, Bill Yuchen Lin
   🏢　arXiv
   [🤗 Leaderboard] [💾 Github] [🐦 Tweet]

3. 🦁 WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild
   Bill Yuchen Lin, Yuntian Deng, Khyathi Chandu, Faeze Brahman, Abhilasha Ravichander, Valentina Pyatkin, Nouha Dziri, Ronan Le Bras, Yejin Choi
   🏢　arXiv
   [🤗 Leaderboard] [💾 Github] [🐦 Tweet]

4. 🎭 SimulBench: Evaluating Language Models with Creative Simulation Tasks
   Qi Jia, Xiang Yue, Tianyu Zheng, Jie Huang, Bill Yuchen Lin
   🏢　arXiv
   [💻 Website]

5. 🚨 WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs
   Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, Nouha Dziri
   🏢　arXiv
   [💾 Github]

6. 💼 OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automation
   Zilong Wang, Yuedong Cui, Li Zhong, Zimin Zhang, Da Yin, Bill Yuchen Lin, Jingbo Shang
   🏢　arXiv
   [💾 Github]

7. 🏆 RewardBench: Evaluating Reward Models for Language Modeling
   Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, Hannaneh Hajishirzi
   🏢　arXiv

8. 🔥 Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models
   Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, Minjoon Seo
   🏢　arXiv

9. 🧩 L3GO: Language Agents with Chain-of-3D-Thoughts for Generating Unconventional Objects
   ✍️　Yutaro Yamada, Khyathi Chandu, Bill Yuchen Lin, Jack Hessel, Ilker Yildirim, Yejin Choi
   🏢　arXiv

10. 🍲 Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging
    ✍️　Joel Jang, Seungone Kim, Bill Yuchen Lin, Yizhong Wang, Jack Hessel, Luke Zettlemoyer, Hannaneh Hajishirzi, Yejin Choi, Prithviraj Ammanabrolu
    🏢　arXiv
    [💾 Github]

2024

1. 🐏 The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning
   ✍️　Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, Yejin Choi
   🏢　ICLR 2024
   [📃 Website] [💾 Github]
   [🤗 Demo (BaseChat)] [🤗 URIAL-Bench] [🐦 Tweet 1] [🐦 2]

2. 🪄 Agent Lumos: Unified and Modular Training for Open-Source Language Agents
   ✍️　Da Yin, Faeze Brahman, Abhilasha Ravichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, Bill Yuchen Lin
   🏢　ACL 2024 Main Conference
   [📃 Website] [💾 Github] [🐦 Tweet]

3. 🤖 Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents
   ✍️　Yifan Song, Da Yin, Xiang Yue, Jie Huang, Sujian Li, Bill Yuchen Lin
   🏢　ACL 2024 Main Conference
   [💾 Github]

4. 🛡️ SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding
   ✍️　Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, Radha Poovendran
   🏢　ACL 2024 Main Conference
   [💾 Github]

5. 👀 Selective “Selective Prediction”: Reducing Unnecessary Abstention in Vision-Language Reasoning
   Tejas Srinivasan, Jack Hessel, Tanmay Gupta, Bill Yuchen Lin, Yejin Choi, Jesse Thomason, Khyathi Raghavi Chandu
   🏢　ACL 2024 Findings

6. 💻 OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement
   ✍️　Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, Xiang Yue
   🏢　ACL 2024 Findings [💾 Website] [💾 Demo] [💾 Code] [💾 Models]

7. 🃏 Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT4
   Jiaxian Guo*, Bo Yang*, Paul Yoo, Bill Yuchen Lin, Yusuke Iwasawa, Yutaka Matsuo
   🏢　COLM 2024
   [💾 Github] [🐦 Tweet]

8. ⚛️ LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition
   ✍️　Chengsong Huang*, Qian Liu*, Bill Yuchen Lin*, Tianyu Pang, Chao Du, Min Lin
   🏢　COLM 2024
   [💾 Demo] [💾 Github] [🐦 Tweet]

9. 🕸️ VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?
   Junpeng Liu, Yifan Song, Bill Yuchen Lin, Wai Lam, Graham Neubig, Yuanzhi Li, Xiang Yue
   🏢　COLM 2024

10. 🐯 TIGERScore: Towards Building Explainable Metric for All Text Generation Tasks
    ✍️　Dongfu Jiang*, Yishan Li*, Ge Zhang, Wenhao Huang, Bill Yuchen Lin, Wenhu Chen
    🏢　TMLR
    [💾 Github] [📃 Website] [🐦 Tweet]

2023

1. 🔥 SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Tasks
   ✍️　Bill Yuchen Lin, Yicheng Fu, Karina Yang, Prithviraj Ammanabrolu, Faeze Brahman, Shiyu Huang, Chandra Bhagavatula, Yejin Choi, Xiang Ren
   🏢　NeurIPS 2023 (spotlight)
   [📃 Website] [💾 Github] [🐦 Tweet] [📰 Blog]

2. 🔥 Faith and Fate: Limits of Transformers on Compositionality
   ✍️　Nouha Dziri*, Ximing Lu*, Melanie Sclar*, Xiang Lorraine Li^, Liwei Jiang^, Bill Yuchen Lin^,
   Peter West, Chandra Bhagavatula, Ronan Le Bras,Jena Hwang,Soumya Sanyal,Sean Welleck,Xiang Ren, Allyson Ettinger, Zaid Harchaoui, Yejin Choi
   🏢　NeurIPS 2023 (spotlight)
   [🐦 Tweet]

3. 🔥 LLM-Blender: Ensembling Large Language Models with Pairwise Comparison and Generative Fusion
   ✍️　Dongfu Jiang, Xiang Ren, Bill Yuchen Lin
   🏢　to appear in Proc. of ACL 2023
   [📃 Website] [💾 Github] [🐦 Tweet]
   Media coverage : MarkTechPost

4. 🍺 Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuning
   ✍️　Ximing Lu, Faeze Brahman, Peter West, Jaehun Jang, Khyathi Chandu, Abhilasha Ravichander, Lianhui Qin, Prithviraj Ammanabrolu,
   Liwei Jiang, Sahana Ramnath, Nouha Dziri, Jillian Fisher, Bill Yuchen Lin, Skyler Hallinan, Xiang Ren, Sean Welleck, Yejin Choi
   🏢　EMNLP 2023 (Main)

5. NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation
   ✍️　Peter West, Ronan Le Bras, Taylor Sorensen, Bill Yuchen Lin, Liwei Jiang, Ximing Lu, Khyathi Chandu, Jack Hessel, Ashutosh Baheti, Chandra Bhagavatula, Yejin Choi
   🏢　EMNLP 2023 (Findings)

6. On Grounded Planning for Embodied Tasks with Language Models
   ✍️　Bill Yuchen Lin*, Chengsong Huang*, Qian Liu, Wenda Gu, Sam Sommerer, Xiang Ren
   🏢　in Proc. of AAAI 2023
   [📃 Website] [💾 Github] [🤗 Data]
   Media coverage : USC Viterbi News

7. AutoTriggER: Named Entity Recognition with Auxiliary Trigger Extraction
   ✍️　Dong-Ho Lee, Ravi Kiran Selvam, Sheikh Muhammad Sarwar, Bill Yuchen Lin,
   Mahak Agarwal, Fred Morstatter, Jay Pujara, Elizabeth Boschee, James Allan, Xiang Ren
   🏢　in Proc. of EACL 2023, also presented at TrustNLP @ NAACL 2021 (best paper award)

8. Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
   ✍️　442 authors including Bill Yuchen Lin
    [💾 Github]
   🏢　in TMLR

2022

1. Reflect, Not Reflex: Inference-Based Common Ground Improves Dialogue Response Quality
   ✍️　Pei Zhou, Hyundong J. Cho, Pegah Jandaghi, Dong-Ho Lee, Bill Yuchen Lin, Jay Pujara, Xiang Ren
   🏢　in Proc. of EMNLP 2022
   [🐦 Tweet]

2. Unsupervised Cross-Task Generalization via Retrieval Augmentation
   ✍️　Bill Yuchen Lin, Kangmin Tan, Chris Miller, Beiwen Tian, Xiang Ren
   🏢　in Proc. of NeurIPS 2022
    [📃 Website]  [💾 Github] [🖼️ Slides]  [🎦 Video] [🐦 Tweet]

3. On Continual Model Refinement in Out-of-Distribution Data Streams
   ✍️　Bill Yuchen Lin, Sida Wang, Xi Victoria Lin, Robin Jia, Lin Xiao, Xiang Ren, Scott Yih
   🏢　in Proc. of ACL 2022
    [📃 Website]  [💾 Github] [🖼️ Slides]  [🎦 Video] [🐦 Tweet]

4. FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks
   ✍️　Bill Yuchen Lin*, Chaoyang He*, Zihang Zeng, Hulin Wang, Yufen Huang, Mahdi Soltanolkotabi, Xiang Ren^, Salman Avestimehr^
   🏢　in Proc. of NAACL 2022 Findings
   [💾 Github] [🐦 Tweet]

5. On the Robustness of Reading Comprehension Models to Entity Renaming
   ✍️　Jun Yan, Yang Xiao, Sagnik Mukherjee, Bill Yuchen Lin, Robin Jia, Xiang Ren
   🏢　in Proc. of NAACL 2022

2021

1. CrossFit: A Few-shot Learning Challenge for Cross-Task Generalization in NLP
   ✍️　Qinyuan Ye, Bill Yuchen Lin, Xiang Ren
   🏢　in Proc. of EMNLP 2021
   [💾 Github] [🐦 Tweet]

2. Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning
   ✍️　Xisen Jin, Bill Yuchen Lin, Mohammad Rostami, Xiang Ren
   🏢　in Proc. of EMNLP 2021 Findings
   [💾 Github]

3. RockNER: A Simple Method to Create Adversarial Examples for Evaluating the Robustness of NER Models
   ✍️　Bill Yuchen Lin, Wenyang Gao, Jun Yan, Ryan Moreno, Xiang Ren
   🏢　in Proc. of EMNLP 2021 (short)
    [📃 Website]

4. RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axioms
   ✍️　Pei Zhou, Rahul Khanna, Seyeon Lee, Bill Yuchen Lin, Daniel Ho, Jay Pujara, Xiang Ren
   🏢　in Proc. of EMNLP 2021
    [📃 Website]

5. Probing Commonsense Explanation in Dialogue Response Generation
   ✍️　Pei Zhou, Pegah Jandaghi, Hyundong Cho, Bill Yuchen Lin, Jay Pujara, Xiang Ren
   🏢　in Proc. of EMNLP 2021 Findings

6. Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoning
   ✍️　Bill Yuchen Lin, Seyeon Lee, Xiaoyang Qiao, Xiang Ren
   🏢　in Proc. of ACL 2021
   [💾 Github]   [📃 Website]

7. RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledge
   ✍️　Bill Yuchen Lin, Ziyi Wu, Yichi Yang, Dong-Ho Lee, Xiang Ren
   🏢　in Proc. of ACL 2021 Findings
   [💾 Github]   [📃 Website]

8. Differentiable Open-Ended Commonsense Reasoning
   ✍️　Bill Yuchen Lin, Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Xiang Ren, William W. Cohen
   🏢　in Proc. of NAACL 2021
   [🖼️ Slides]   [🎦 Video]   [💾 Github]   [📃 Website]

9. Pre-training Text-to-Text Transformers for Concept-Centric Common Sense
   ✍️　Wangchunshu Zhou, Dong-Ho Lee, Ravi Kiran Selvam, Seyeon Lee, Bill Yuchen Lin, Xiang Ren
   🏢　in Proc. of ICLR 2021  
   [💾 Github]

10. IsoBN: Fine-Tuning BERT with Isotropic Batch Normalization
    ✍️　Wenxuan Zhou, Bill Yuchen Lin, Xiang Ren
    🏢　in Proc. of AAAI 2021

2020

1. CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning
   ✍️　Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi, Xiang Ren
   🏢　in Proc. of EMNLP 2020 Findings    (presented at AKBC 2020 as a non-archival paper.)
   [📃 Website]
   Media coverage : The Register , Tech Xplore , Techzine , Radio.com , ScienceDaily , USC Viterbi

2. Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models
   ✍️　Bill Yuchen Lin, Seyeon Lee, Rahul Khanna, Xiang Ren
   🏢　in Proc. of EMNLP 2020 (short)
   [📃 Website]

3. Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering
   ✍️　Yanlin Feng*, Xinyue Chen*, Bill Yuchen Lin, Peifeng Wang, Jun Yan, Xiang Ren
   🏢　in Proc. of EMNLP 2020
   [💾 Github]

4. FreeDOM: A Transferable Neural Architecture for Structured Information Extraction on Web Documents
   ✍️　Bill Yuchen Lin, Ying Sheng, Nguyen Vo and Sandeep Tata
   🏢　in Proc. of KDD 2020 (Research Track)
   [🖼️ Slides] [🎦 Video]

5. TriggerNER: Learning with Entity Triggers as Explanations for Named Entity Recognition
   ✍️　Bill Yuchen Lin*, Dongho Lee*, Ming Shen, Ryan Moreno, Xiao Huang, Prashant Shiralkar, Xiang Ren
   🏢　in Proc. of ACL 2020 (short)
   [🖼️ Slides]   [🎦 Video]   [💾 Github]   [📃 Website]

6. Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labeling.
   ✍️　Ouyu Lan, Xiao Huang, Bill Yuchen Lin, He Jiang, Liyuan Liu, Xiang Ren
   🏢　in Proc. of ACL 2020
   [💾 Github]

7. LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation
   ✍️　Dong-Ho Lee, Rahul Khanna, Bill Yuchen Lin, Jamin Chen, Seyeon Lee, Qinyuan Ye, Elizabeth Boschee, Leonardo Neves, Xiang Ren
   🏢　in Proc. of ACL 2020 (Demo Track)
   [📃 Website]

8. NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extraction.
   ✍️　Wenxuan Zhou, Hongtao Lin, Bill Yuchen Lin, Ziqi Wang, Junyi Du, Leonardo Neves, Xiang Ren
   🏢　in Proc. of TheWebConf (WWW) 2020
   Best Paper Runner-up (2/1500+)    [💾 Github]

2019

1. KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning.
   ✍️　Bill Yuchen Lin, Xinyue Chen, Jamin Chen, Xiang Ren
   🏢　in Proc. of EMNLP-IJCNLP 2019
   [💾 Github]
2. AlpacaTag: An Active Learning-based Crowd Annotation Framework for Sequence Tagging.
   ✍️　Bill Yuchen Lin*, Dongho Lee*, Frank F. Xu, Ouyu Lan, Xiang Ren
   🏢　in Proc. of ACL 2019 (Demo Track)
   [📃 Website]

2018

1. Neural Adaptation Layers for Cross-domain Named Entity Recognition.
   ✍️　Bill Yuchen Lin, Wei Lu
   🏢　in Proc. of EMNLP 2018
   [💾 Github]
2. ExtRA: Extracting Prominent Review Aspects from Customer Feedback.
   ✍️　Zhiyi Luo, Shanshan Huang, Frank F. Xu, Bill Yuchen Lin, Hanyuan Shi, Kenny Q. Zhu
   🏢　in Proc. of EMNLP 2018
   [💾 Github]
3. Mining Cross-Cultural Differences and Similarities in Social Media.
   ✍️　Bill Yuchen Lin*, Frank F. Xu*, Kenny Q. Zhu, Seung-won Hwang
   🏢　in Proc. of ACL 2018
   [💾 Github]
4. Automatic Extraction of Commonsense LocatedNear Knowledge.
   ✍️　Frank F. Xu*, Bill Yuchen Lin*, Kenny Q. Zhu
   🏢　in Proc. of ACL 2018 (short)
   [💾 Github]

2017

1. Multi-channel BiLSTM-CRF Model for Emerging Named Entity Recognition in Social Media.
   ✍️　Bill Y. Lin*, Frank F. Xu*, Zhiyi Luo, Kenny Q. Zhu
   🏢　in Proc. of EMNLP 2017, Workshop on Noisy User-generated Text
   [💾 Github]