Seminar: Haiying Shen (University of Virginia), Resource Optimization for ML Inference Serving
Content
Speaker
Haiying Shen (University of Virginia)
Abstract
My research focuses on job scheduling and resource management in Machine Learning (ML) and Large Language Model (LLM) systems. With the growing popularity of deep learning models, minimizing the monetary costs and maximizing the goodput of inference-serving systems have become critical challenges. Addressing these challenges requires efficient task scheduling within and across nodes, as well as optimized resource management to ensure high performance, resource utilization, and adherence to Service Level Objectives (SLOs). However, current approaches often fall short of fully addressing these challenges. In this talk, I will present our novel methods designed to address these gaps and enable the efficient execution of ML/LLM workloads. I will also briefly discuss my ongoing and future research plans for advancing LLM systems.
Bio
Dr. Haiying Shen is currently an Associate Professor in the Department of Computer Science at the University of Virginia and she was a Consulting Researcher at Microsoft on LLM systems in 2024. Her research area is distributed systems, focusing on LLM/LM systems, cloud computing, big data and edge computing. Dr. Shen has made significant contributions to her field, with an H-index of 50 and over 370 publications in top conferences and journals such as SIGCOMM, OSDI, EuroSys, ASPLOS, CoNext, Infocom, ICDCS, IPDPS, IEEE/ACM Transactions on Networking (TON), IEEE Transactions on Parallel and Distributed Systems (TPDS), and IEEE Transactions on Mobile Computing (TMC). Her papers have received the George N. Saridis Best Transactions Paper Award 2021, best paper awards at CloudCom 2016 and NAS 2018, a best paper runner-up award at ICCCN 2015, best paper award nominations at ICPP 2021, MASS 2011, and CCGrid 2009, and a best-in-session presentation award at INFOCOM 2017. She received the 2010 Microsoft Faculty Fellowship Award, the 2015 IEEE Technical Committee on Scalable Computing (TCSC) Mid-career Award, the 2015 IBM Faculty Award, the 2013 NSF CAREER Award, and the 2013 Sigma Xi Clemson Chapter Young Investigator Award. She currently advises ten Ph.D. students, one MS student and two postdoctoral fellows. She is an Associate Editor for IEEE/ACM Transactions on Networking (TON), IEEE Transactions on Mobile Computing (TMC), and IEEE Networking Letters (NL). Dr. Shen has served on the program committees of numerous leading conferences and has been a program co-chair and general co-chair for several international conferences.
Faculty host