Machine Learning and Friends Lunch: Xinya Du, Synergizing Knowledge and Large Language Models

Content

Abstract

Large Language Models (LLMs) have revolutionized the field of natural language processing and reshaped how humans acquire and interact with knowledge. In this talk, I will discuss my research on synergizing LMs and knowledge — where LLMs not only extract and discover knowledge, but also continually improve by integrating new knowledge. First, I will cover our work on improving knowledge extraction from the vast amount of existing literature, with a particular focus on enabling models to better understand long documents in a cost-efficient and comprehensive manner. I will describe a novel paradigm for representing document-level structured information as question answer pairs, and how we extract them by leveraging global context. Next, I will introduce our pioneering investigation into using LLMs for new scientific knowledge discovery. We explore a multi-stage, LLM-based framework to generate and iteratively refine natural language scientific hypotheses. Finally, building on the above efforts, I will complete the virtuous cycle by demonstrating how LLMs can integrate the knowledge they acquire to continuously enhance their reasoning capabilities and their ability to learn new knowledge.

Bio

Xinya Du is a tenure-track assistant professor at UT Dallas Computer Science Department. He earned a Ph.D. degree from Cornell University and was a Postdoctoral Research Associate at the University of Illinois (UIUC). He has also worked at Microsoft Research, Google Research, and Allen Institute AI. His research is on natural language processing, deep learning, and large language models. The goal is to build intelligent machines with both faithful knowledge & reasoning capabilities.

His work has been published in leading NLP and ML conferences (ACL, EMNLP, ICLR). His work was included in the list of Most Influential ACL Papers by Paper Digest and has been covered by major media like New Scientist. He was named a Spotlight Rising Star in Data Science by the University of Chicago and was selected for the New Faculty Highlights program by AAAI. He is the recipient of the 2024 Amazon Research Award and the 2024 NSF CAREER Award.