About Me
I am currently a Professor at School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen. Previously, I was a Research Assistant Professor at Department of Computer Science & Engineering of The Hong Kong University of Science and Technology. I completed my Ph.D. study with Hong Kong Baptist University under the supervision of Prof. Xiaowen Chu in 2020. I received a B.Eng. degree in software engineering from South China University of Technology in 2010, and an M.Sc. degree in computer science from Harbin Institute of Technology under the supervision of Prof. Xuan Wang in 2013. My current research focus is distributed machine learning systems.
News
- 16/01/2024: Our paper “FedImpro: Measuring and Improving Client Update in Federated Learning” has been accepted by ICLR 2024.
- 10/01/2024: Our paper “ScheMoE: An Extensible Mixture-of-Experts Distributed Training System with Tasks Scheduling” has been accepted by EuroSys 2024.
- 01/12/2023: Our paper “Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules” has been accepted by IEEE INFOCOM 2024.
- 12/11/2023: Our paper “Performance Analysis and Optimizations of Matrix Multiplications on ARMv8 Processors” has been accepted by DATE 2024.
- 10/04/2023: Two papers have been accepted by IEEE ICDCS 2023.
- 21/01/2023: Our paper “Eva: Practical Second-order Optimization with Kronecker-vectorized Approximation” has been accepted by ICLR 2023.
- 06/12/2022: Our paper “GossipFL: A Decentralized Federated Learning Framework with Sparsified and Adaptive Communication” has been accepted by IEEE TPDS.
- 02/12/2022: Two papers have been accepted by IEEE INFOCOM 2023.
- 06/09/2022: Our paper “Scalable K-FAC Training for Deep Neural Networks with Distributed Preconditioning” has been accepted by IEEE Transactions on Cloud Computing.
- 01/09/2022: I joined Harbin Institute of Technology, Shenzhen as an Assistant Professor.
- 04/07/2022: Our paper “EASNet:Searching Elastic and Accurate Network Architecture for Stereo Matching” has been accepted by ECCV 2022.
- 15/05/2022: Our paper “Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learning” has been accepted by ICML 2022.
- 11/05/2021: Our paper “Exploiting Simultaneous Communications to Accelerate Data Parallel Distributed Deep Learning” has recieved the Best Paper Award (3 out of 252 accepted papers) by IEEE INFOCOM 2021.
- 18/03/2021: Our paper “Accelerating Distributed K-FAC with Smart Parallelism of Computing and Communication Tasks” has been accepted by IEEE ICDCS 2021.
- 17/03/2021: We are organizing the 5th International Workshop on Embedded and Mobile Deep Learning: https://emdl21.github.io/, co-located with ACM MobiSys 2021. Please consider submitting your work to the venue.
- 19/01/2021: Our paper “Towards Scalable Distributed Training of Deep Learning on Public Cloud Clusters” [PDF] has been accepted by MLSys 2021.
- 14/01/2021: Our paper “MG-WFBP: Merging Gradients Wisely for Efficient Communication in Distributed Deep Learning” [Code, PDF] has been accepted by IEEE TPDS.
- 05/12/2020: Our paper “Exploiting Simultaneous Communications to Accelerate Data Parallel Distributed Deep Learning” has been accepted by IEEE INFOCOM 2021.
- 02/12/2020: Our paper “Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans” has been accepted by AAAI 2021.
- 21/10/2020: Our survey paper “A Quantitative Survey of Communication Optimizations in Distributed Deep Learning” [Code, PDF] has been accepted by IEEE Network Magazine.
- 28/09/2020: Awarded CCF-Baidu Open Fund (CCF-百度松果基金) [Link].
- 01/09/2020: I start my academic position as a RAP at HKUST.
Work Experience
- 10/2023-present: Professor, HITSZ.
- 09/2022-09/2023: Assistant Professor, HITSZ.
- 09/2020-08/2022: Research Assistant Professor, HKUST.
- 04/2019-05/2020: Deep Learning Intern, NVIDIA.
- 02/2014-03/2016: Senior Research Assistant, Hong Kong Baptist University.
- 02/2013-02/2014: Research Assistant, Hong Kong Baptist University.
Research Interests (Publications)
- Distributed machine learning systems
- GPU computing
- Parallel and distributed systems
- Deep learning
Teaching
- Instructor at HITSZ
- 2023 Fall Semester: Parallel Processing and Computer Architecture
- 2023 Fall Semester: Computer Architecture
- 2022 Spring Semester: Computer Systems (CSAPP)
- Instructor at HKUST
- Teaching Assistant at HKBU
- 2019 Spring Semester, IT Forum
- 2018 Fall Semester, Software Engineering
- 2018 Spring Semester, Advanced Programming for Software Development
- 2018 Spring Semester, Data Communications and Networking
- 2017 Fall Semester, Enterprise Networking and Cloud Computing
- 2017 Spring Semester, Cloud Computing
- 2016 Fall Semester, Mobile Computing
- 2016 Fall Semester, iMake Apps
Professional Activities
- Organization
- 5th International Workshop on Embedded and Mobile Deep Learning, co-located with ACM MobiSys 2021, https://emdl21.github.io/.
- Invited Program Committee Member for Conferences
- ICLR ‘24, AAAI ‘24
- NeurIPS ‘23, IJCAI ‘23, AAAI ‘23, HiPC ‘22, ICML ‘22, IJCAI ‘22, AAAI ‘22, DSS ‘20, ICC Workshop ‘21, IJCAI ‘21, ICDCS ‘21, HiPC ‘21, ICPADS ‘21, HPCC ‘21, DSS ‘21
- Invited Reviewer for Journals
- ACM Computing Surveys (CSUR)
- IEEE Network Magazine
- IEEE Transactions on Mobile Computing (TMC)
- IEEE Transactions on Cloud Computing (TCC)
- IEEE Transactions on Parallel and Distributed Systems (TPDS)
- IEEE Transactions on Network Science and Engineering (TNSE)
- IEEE Transactions on Industrial Informatics (TII)
- IEEE Journal on Selected Areas in Communications (JSAC)
- Journal of Parallel and Distributed Computing (JPDC)
- IEEE Acess
- MDPI Sensors
Awards and Prizes
- 2021, Best Paper Award of IEEE INFOCOM 2021.
- 2020, Yakun Scholarship Scheme for Mainland Postgraduate Students, Hong Kong Baptist University. [Link]
- 2018-2020, RPg Performance Award Scheme, Hong Kong Baptist University. [Link]
- 2018, Best Paper Award of IEEE DataCom 2018. [Link]
- 2018, Teaching Assistant Performance Award, Hong Kong Baptist University. [Link]
- 2017, Alibaba Tianchi Healthcare AI Competition, Ranked 7th out of 2887. [Link]
- 2012, Graduate National Scholarship, Harbin Institute of Technology. [Link]
- 2010, First Prize of the Second National CUDA Programming Competition, NVIDIA.
- 2009, Outstanding Prize of the First National CUDA Programming Competition, NVIDIA.
- 2007-2010, National Scholarship and Merit Student, South China University of Technology.
Technical Skills
- General: C/C++, Python, and Linux Shell.
- Parallel and Distributed Systems: CUDA, OpenCL, SSE, OpenMP, MPI, and Horovod.
- Deep Learning Frameworks: PyTorch, TensorFlow, and Caffe.