Publications

A System for Microserving of LLMs
Hongyi Jin, Ruihang Lai, Charlie F. Ruan, Yingcheng Wang, Todd Mowry, Xupeng Miao, Zhihao Jia, Tianqi Chen
PDF Code Blog
WebLLM: A High-Performance In-Browser LLM Inference Engine
Charlie F. Ruan, Yucheng Qin, Xun Zhou, Ruihang Lai, Hongyi Jin, Yixin Dong, Bohan Hou, Meng-Shiun Yu, Yiyan Zhai, Sudeep Agarwal, Hangrui Cao, Siyuan Feng, Tianqi Chen
PDF Code Blog Talk
XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models
Yixin Dong, Charlie F. Ruan, Yaxing Cai, Ruihang Lai, Ziyi Xu, Yilong Zhao, Tianqi Chen
PDF Code Blog
Local deployment of large-scale music AI models on commodity hardware
Xun Zhou, Charlie F. Ruan, Zihe Zhao, Tianqi Chen, Chris Donahue
PDF
Emerging Platforms Meet Emerging LLMs: A Year-Long Journey of Top-Down Development
Siyuan Feng, Jiawei Liu, Ruihang Lai, Charlie F. Ruan, Yong Yu, Lingming Zhang, Tianqi Chen
PDF
Coordinating Distributed Example Orders for Provably Accelerated Training
A. Feder Cooper, Wentao Guo, Khiem Pham, Tiancheng Yuan, Charlie F. Ruan, Yucheng Lu, Christopher De Sa
PDF Code Poster