About
People
Publications
Events
Blog
Careers
Visiting researchers
Contact
English
English
Français
ServiceNow
ServiceNow AI Research
Tags
Adversarial Attacks
ServiceNow AI Research
Adversarial Attacks
Visual Memory Attacks Can Persist Through Context Masking
Vision-language models (VLMs) are increasingly deployed as autonomous agents that ingest untrusted content, including images. Recent …
David Dobre
,
Leo Schwinn
,
Gauthier Gidel
,
Spandana Gella
,
Perouz Taslakian
,
Pierre-André Noël
COLM Workshops, 2026.
Cite
Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chain
The practice of fine-tuning AI agents on data from their own interactions—such as web browsing or tool use—, while being a strong …
Léo Boisvert
,
Abhay Puri
,
Chandra Kiran Reddy Evuru
,
Nicolas Chapados
,
Quentin Cappart
,
Alexandre Lacoste
,
Nazanin Sepahvand
,
Krishnamurthy (Dj) Dvijotham
,
Alexandre Drouin
,
Jason Stanley
ACM Conference on AI and Agentic Systems, 2026.
Paper
Cite
Code
Keeping up with dynamic attackers: Certifying robustness to adaptive online data poisoning
The rise of foundation models fine-tuned on human feedback from potentially untrusted users has increased the risk of adversarial data …
Avinandan Bose
,
Laurent Lessard
,
Maryam Fazel
,
Krishnamurthy (Dj) Dvijotham
International Conference on Artificial Intelligence and Statistics (AISTATS), 2025.
Paper
Cite
Code
Video
Constraining Representations Yields Models That Know What They Don't Know
A well-known failure mode of neural networks is that they may confidently return erroneous predictions. Such unsafe behaviour is …
João Monteiro
,
Pau Rodriguez
,
Pierre-André Noël
,
Issam H. Laradji
,
David Vazquez
International Conference of Learning Representations (ICLR), 2023.
Paper
Cite
Code
Model Card
Slides
Video
Maximal Jacobian-based Saliency Map Attack
The Jacobian-based Saliency Map Attack is a family of adversarial attack methods for fooling classification models, such as deep neural …
Rey Reza Wiyatno
,
Anqi Xu
Montreal AI Symposium (MAIS), 2018.
Paper
Cite
Code
Cite
×