Variational inference (VI) is a prominent framework that approximates the posterior distribution of a probabilistic model by optimizing an objective over a tractable distribution family. Black-box VI (BBVI) goes a step further and abstracts away model specifics, requiring only the ability to evaluate the log density or its gradient, allowing wide applicability. Recently, there has been interest in automating BBVI to facilitate easy use for non-inference experts. However, versatile applications render many current BBVI methods unreliable in practice. In this thesis, we propose to improve the robustness, scalability, evaluation, and accuracy of BBVI by combining different algorithmic techniques, exploiting the structure of probabilistic models, and carefully optimizing neural network-based variational families.
Advisor: Justin Domke