PhD Dissertation Proposal Defense: Arisa Tajima, Advancing End-to-End Privacy in Machine Learning: Input, Output, and Beyond
Content
Speaker
Abstract
As machine learning becomes integral to modern applications, ensuring privacy while maintaining utility and efficiency remains a critical challenge. Privacy-enhancing technologies (PETs) such as differential privacy (DP) and secure computation provide solutions, but their integration often involves trade-offs between privacy, accuracy, and efficiency. This dissertation introduces novel techniques to enhance privacy in machine learning while achieving improved accuracy and efficiency over existing methods.
The first part of this work focuses on protecting output privacy in machine learning predictions. We propose a DP mechanism for random forest classifiers that leverages a novel matrix representation to optimize DP noise addition, resulting in improved accuracy. Additionally, we explore the application of DP to released census data, a critical downstream task for fair redistricting, and design mechanisms that enhance equal voting power and fair representation.
Next, we address input privacy, developing a secure protocol for fine-tuning language models using secure multiparty computation (MPC). Through an adaptive loss-scaling algorithm, we address numerical instability, a key challenge in MPC-based training of large-scale models, enabling efficient and privacy-preserving fine-tuning.
Finally, we propose hybrid methods that combine DP and cryptographic techniques to ensure both input and output privacy. Balancing efficiency, accuracy and privacy, we introduce optimizations that relax cryptographic requirements to DP in specific tasks, such as counting queries, a foundational operation in many machine learning algorithms. This approach significantly improves the efficiency of existing cryptographic protocols.
By advancing PETs for machine learning, this dissertation demonstrates how carefully designed privacy techniques can achieve superior accuracy and efficiency, bridging the gap between robust privacy protection and practical usability.
Advisor
Amir Houmansadr