ServiceNow AI Research

Adversarial Attacks

Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chain
The practice of fine-tuning AI agents on data from their own interactions—such as web browsing or tool use—, while being a strong …
Keeping up with dynamic attackers: Certifying robustness to adaptive online data poisoning
The rise of foundation models fine-tuned on human feedback from potentially untrusted users has increased the risk of adversarial data …
Constraining Representations Yields Models That Know What They Don't Know
A well-known failure mode of neural networks is that they may confidently return erroneous predictions. Such unsafe behaviour is …
Maximal Jacobian-based Saliency Map Attack
The Jacobian-based Saliency Map Attack is a family of adversarial attack methods for fooling classification models, such as deep neural …