Bagdasarian Wins Distinguished Paper Award at 2024 USENIX Security Symposium
Content
Eugene Bagdasarian, assistant professor in the Manning College of Information and Computer Sciences (CICS) at UMass Amherst, has been awarded a Distinguished Paper Award at the 2024 USENIX Security Symposium for “Adversarial Illusions in Multi-Modal Embeddings, which explores the vulnerabilities of multi-modal embeddings.
The research, written in collaboration with Tingwei Zhang and Rishi Jha of Cornell University and Vitaly Shmatikov of Cornell Tech, demonstrates that multi-modal embeddings, which encode images, sounds, text, and videos into a single format, can be vulnerable to adversarial illusions—an attack that slightly alters an image or sound, tricking the system into thinking it matches with something else.
Bagdasarian reveals how he and Shmatikov initially drew inspiration from the art of Belgian surrealist René Magritte: “As a kid, I attended art school in Tashkent [Uzbekistan] and my main influence was from surrealist artists like Dali and Bosch. [Shmatikov] proposed [we] connect [our research] to Magritte's art that showed how images and texts could contradict each other—we even used one of Magritte's quotes as an attack input and his painting as a target! It was really exciting to bring these ideas to our paper.”
By studying multi-modal embedding models like ImageBind, the researchers demonstrated how adversaries could manipulate embeddings to deceive downstream tasks like image, text generation, and zero-shot classification. This kind of attack can confuse various functions that rely on these embeddings, making it a significant concern for future technology.
“Reliance on external multi-modal embeddings to represent input data has opened even private pipelines to adversarial attacks,” says Bagdasarian. “Our illusion attack created inputs that literally act as a wolf in sheep’s clothing. This attack could mislead recommender systems, classifiers, and various GenAI use cases, and requires further study on practical defenses.”
By exploring how these targeted attacks may work across different embedding systems, the researchers explored possible defenses against these attacks, as well as ways to bypass those defenses. “I was pleased that our community has recognized this problem and its potential to affect applications relying on third-party multi-modal embeddings,” says Bagdasarian.
Bagdasarian joined the CICS faculty in the fall of 2024. His work focuses on security and privacy in emerging AI-based systems under real-life conditions and attacks. He completed his PhD at Cornell Tech and received an Apple Scholars in AI/ML Fellowship during his studies.
