Qinghao Hu

Qinghao Hu (胡擎昊)

I am a Research Assistant Professor at S-Lab, Nanyang Technological University, Singapore. I obtained my Ph.D. degree from NTU in 2023, advised by Prof. Tianwei Zhang and Prof. Yonggang Wen. Before that, I received my M.Sc. degree from National University of Singapore in 2020 and my B.Eng. degree from Zhejiang University in 2018. I am fortunate to be awarded the Google PhD Fellowship.
Email: [email protected]
Address:
S-Lab for Advanced Intelligence
ABN-02b-11, 61 Nanyang Avenue, Singapore 637335

Research Interest

Systems for Large Models

Datacenter Management and Scheduling

Machine Learning for Systems

Publications

[NSDI '24]	Characterization of Large Language Model Development in the Datacenter Qinghao Hu, Zhisheng Ye, Zerui Wang*, Guoteng Wang, Meng Zhang, Qiaoling Chen, Peng Sun, Dahua Lin, Xiaolin Wang, Yingwei Luo, Yonggang Wen, Tianwei Zhang Paper / System / Data / USENIX ;login:
[SC '24]	TorchGT: A Holistic System for Large-scale Graph Transformer Training Meng Zhang, Jie Sun, Qinghao Hu, Peng Sun, Zeke Wang, Yonggang Wen, Tianwei Zhang To Appear
[ICDE '24]	Sylvie: 3D-adaptive and Universal System for Large-scale Graph Neural Network Training Meng Zhang, Qinghao Hu, Cheng Wan, Haozhao Wang, Peng Sun, Yonggang Wen, Tianwei Zhang Paper / Code
[CSUR '24]	Deep Learning Workload Scheduling in GPU Datacenters: A Survey Zhisheng Ye, Wei Gao, Qinghao Hu*, Peng Sun, Xiaolin Wang, Yingwei Luo, Tianwei Zhang, Yonggang Wen Paper / Awesome List / ACM Computing Surveys
[OSDI '23]	Hydro: Surrogate-Based Hyperparameter Tuning Service in Datacenters Qinghao Hu, Zhisheng Ye, Meng Zhang, Qiaoling Chen, Peng Sun, Yonggang Wen, Tianwei Zhang Paper / Code / Artifact Badges: Available Functional Reproduced
[ASPLOS '23]	Lucid: A Non-Intrusive, Scalable and Interpretable Scheduler for Deep Learning Training Jobs Qinghao Hu, Meng Zhang, Peng Sun, Yonggang Wen, Tianwei Zhang Paper / Code / Artifact Badges: Available Functional Reproduced Distinguished Paper Award
[ATC '22]	Primo: Practical Learning-Augmented Systems with Interpretable Models Qinghao Hu, Harsha Nori, Peng Sun, Yonggang Wen, Tianwei Zhang Paper / Code / Artifact Badges: Available Functional Reproduced
[SC '21]	Characterization and Prediction of Deep Learning Workloads in Large-Scale GPU Datacenters Qinghao Hu, Peng Sun, Shengen Yan, Yonggang Wen, Tianwei Zhang Paper / Code / Data / Artifact Badges: Available Functional Reproduced

Preprint

[arXiv '24]	LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism Diandian Gu, Peng Sun, Qinghao Hu, Ting Huang, Xun Chen, Yingtong Xiong, Guoteng Wang, Qiaoling Chen, Shangchun Zhao, Jiarui Fang, Yonggang Wen, Tianwei Zhang, Xin Jin, Xuanzhe Liu Paper / Submitted to a Conference
[arXiv '24]	InternEvo: Efficient Long-Sequence Large Language Model Training via Hybrid Parallelism and Redundant Sharding Qiaoling Chen, Diandian Gu, Guoteng Wang, Xun Chen, Yingtong Xiong, Ting Huang, Qinghao Hu, Xin Jin, Yonggang Wen, Tianwei Zhang, Peng Sun Liu Paper / Submitted to a Conference
[arXiv '23]	AMSP: Super-Scaling LLM Training via Advanced Model States Partitioning Qiaoling Chen, Qinghao Hu, Zhisheng Ye, Guoteng Wang, Peng Sun, Yonggang Wen, Tianwei Zhang Paper / Submitted to a Conference

Professional Services

[EuroSys '24]	Shadow Committee Member
[EuroSys '23]	Shadow Committee Member
[OSDI '22]	AE Committee Member
[ATC '22]	AE Committee Member
[EuroSys '22]	AE Committee Member
[SOSP '21]	AE Committee Member

Awards

ML and Systems Rising Stars	2024
Outstanding PhD Thesis Award	2024
National Scholarship for Outstanding Graduates	2024
Google PhD Fellowship	2023
Distinguished Paper Award of ASPLOS	2023
Youth Outstanding Paper Award of WAIC	2023
Student Travel Grant of OSDI	2023
Best Undergraduate Thesis Award	2018
Outstanding Graduates of Zhejiang University	2018