UMass AI&Sec SP'25 Seminar: Sahar Abdelnabi, Evaluating and Securing LLM-Agentic Networks
Content
Speaker
Bio
Sahar Abdelnabi is an AI security researcher at Microsoft's Security Response Center. She completed her PhD at the CISPA Helmholtz Center for Information Security under the supervision of Prof. Dr. Mario Fritz and holds an MSc from Saarland University. Her research focuses on the intersection of machine learning with security, safety, and sociopolitical aspects, including understanding and mitigating failure modes of machine learning models, addressing biases, and exploring emergent safety challenges posed by large language models.
Abstract
There is an increasing interest in using LLM agents to autonomously automate tasks and workflows. For example, the new OpenAI operator may be used to design travel plans for the user. Service providers now use LLM chatbots to assist users as well. Soon, it is very likely that these two sides are going to communicate, forming agentic networks. Such paradigms will unlock new use cases where agents can negotiate, deliberate, adapt, and find creative solutions on behalf of entities they represent. In this talk, I will discuss our work on evaluating multi-agent negotiations, and how that can be beneficial to test reasoning and create evolving, dynamic benchmarks. We use this benchmark to study manipulation and safety risks, such as how cooperative agents can be steered by greedy or adversarial ones. In the second part of the talk, I will present our new work to identify security and privacy risks in adaptive agentic networks where an assistant communicates with an external party to fulfil a multi-goal task. The assistant must perform actions that are entailed by the goal, not over share information, and maintain utility against greedy agents. We create a firewalling mitigation that allow agents to dynamically communicate and adapt, while balancing these security and privacy risks.
Host