Charlie Ruan

MSCS @ CMU

Carnegie Mellon University

Biography

I am a second-year MSCS student at Carnegie Mellon University. I am fortunate to work with Prof. Tianqi Chen as part of the Catalyst Group, where I have also collaborated with Prof. Zhihao Jia. I am currently visiting UC Berkeley, working with Prof. Ion Stoica as part of the Sky Computing Lab.

I focus on the systems problems in machine learning systems, with various open-source development experience. I am the current lead of WebLLM and a core contributor to MLC-LLM.

Prior to coming to CMU, I obtained my B.S. degree in Computer Science and Operations Research from Cornell University, where I was fortunate to work with Prof. Christopher De Sa on distributed training and with Prof. Jim Dai on stochastic processes.

I am applying for Fall 2025 PhD programs in machine learning systems. Please feel free to contact me!

Interests

Machine Learning Systems
Distributed Systems

Education

MS in Computer Science, 2025
Carnegie Mellon University
BS in Computer Science & Operations Research, 2023
Cornell University

Publications

A System for Microserving of LLMs

Hongyi Jin, Ruihang Lai, Charlie F. Ruan, Yingcheng Wang, Todd Mowry, Xupeng Miao, Zhihao Jia, Tianqi Chen

WebLLM: A High-Performance In-Browser LLM Inference Engine

Charlie F. Ruan, Yucheng Qin, Xun Zhou, Ruihang Lai, Hongyi Jin, Yixin Dong, Bohan Hou, Meng-Shiun Yu, Yiyan Zhai, Sudeep Agarwal, Hangrui Cao, Siyuan Feng, Tianqi Chen

XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models

Yixin Dong, Charlie F. Ruan, Yaxing Cai, Ruihang Lai, Ziyi Xu, Yilong Zhao, Tianqi Chen

Local deployment of large-scale music AI models on commodity hardware

Xun Zhou, Charlie F. Ruan, Zihe Zhao, Tianqi Chen, Chris Donahue

Emerging Platforms Meet Emerging LLMs: A Year-Long Journey of Top-Down Development

Siyuan Feng, Jiawei Liu, Ruihang Lai, Charlie F. Ruan, Yong Yu, Lingming Zhang, Tianqi Chen

Coordinating Distributed Example Orders for Provably Accelerated Training

A. Feder Cooper, Wentao Guo, Khiem Pham, Tiancheng Yuan, Charlie F. Ruan, Yucheng Lu, Christopher De Sa

Projects

Core Contributor

MLC-LLM

June 2023 – Present Pittsburgh, PA

Enable universal native deployment for LLMs through machine learning compilation techniques. GitHub (19.3k stars)

Project Lead

WebLLM

June 2023 – Present Pittsburgh, PA

Leading the project to bring LLMs to run locally in client-side browser with WebGPU acceleration. GitHub (13.9k stars); talk at Google WebAI Summit ‘24

Research Experience

Visiting Student Researcher

Prof. Ion Stoica, UC Berkeley

August 2024 – Present Berkeley, CA

Sky Computing Lab; Investigating GPU kernel language/compiler

Research Assistant

Prof. Tianqi Chen & Prof. Zhihao Jia, Carnegie Mellon University

March 2024 – Present Pittsburgh, PA

Catalyst Group; Investigating distributed LLM serving systems

Research Assistant

Prof. Christopher De Sa, Cornell University

September 2022 – June 2023 Ithaca, NY

Investigated finding provably better data permutations in distributed learning. CD-GraB was accepted by NeurIPS'23

Research Assistant

Prof. Jim Dai, Cornell University

November 2021 – September 2022 Pittsburgh, PA

Investigated using variance-reduction method approximating martingale-process in reinforcement learning with large state space

Industry Experience

Software Engineer Intern

Google

June 2023 – August 2023 Sunnyvale, CA

Worked on Core ML’s Distributed Runtime team, optimizing TensorFlow’s checkpoint to reduce wasted TPU cycles

Software Engineer Intern

Google Cloud

August 2022 – October 2022 Sunnyvale, CA

Worked on Technical Infrastructure’s Platform team, deploying accelerators in Google data centers using OpenBMC, implementing Linux daemon and firmware update APIs