ServiceNow AI Research

Agents

FM2DS: Few-Shot Multimodal Multihop Data Synthesis with Knowledge Distillation for Question Answering
FM2DS: Few-Shot Multimodal Multihop Data Synthesis with Knowledge Distillation for Question Answering
We introduce DRBench, a benchmark for evaluating AI agents on complex, open-ended enterprise deep research tasks. Unlike existing …
Amirhossein Abaskohi, Spandana Gella, Giuseppe Carenini, Issam H. Laradji
Conference on Empirical Methods in Natural Language Processing (EMNLP),  2025.
Paper Cite Code
WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
Understanding diverse web data and automating web development presents an exciting challenge for agentic multimodal models. While …
Rabiul Awal, Mahsa Massoud, Zichao Li, Aarash Feizi, Suyuchen Wang, Christopher Pal, Aishwarya Agrawal, David Vazquez, Perouz Taslakian, Spandana Gella, Sai Rajeswar Mudumba
Conference on Empirical Methods in Natural Language Processing (EMNLP),  2025.
Paper Cite Code Video
BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuning
BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuning
Charts are essential to data analysis, transforming raw data into clear visual representations that support human decision-making. …
Ahmed Masry, Abhay Puri, Masoud Hashemi, Juan A. Rodriguez, Megh Thakkar, Khyati Mahajan, Vikas Yadav, Sathwik Tejaswi Madhusudhan, Alexandre Piche, Dzmitry Bahdanau, Christopher Pal, David Vazquez, Enamul Hoque Prince , Perouz Taslakian, Sai Rajeswar Mudumba, Spandana Gella
Conference on Language Modeling (COLM),  2025.
Paper Cite Code Video
DoomArena: A framework for Testing AI Agents Against Evolving Security Threats
DoomArena: A framework for Testing AI Agents Against Evolving Security Threats
We present DoomArena, a security evaluation framework for AI agents. DoomArena is designed on three principles: 1) It is a …
Léo Boisvert, Mihir Bansal, Chandra Kiran Reddy Evuru, Gabriel Huang, Abhay Puri, Avinandan Bose, Maryam Fazel, Quentin Cappart, Jason Stanley, Alexandre Lacoste, Alexandre Drouin, Krishnamurthy (Dj) Dvijotham
Conference on Language Modeling (COLM),  2025.
Paper Cite Code
FocusAgent: Simple Yet Effective Ways of Trimming the Large Context of Web Agents
FocusAgent: Simple Yet Effective Ways of Trimming the Large Context of Web Agents
Web agents powered by large language models (LLMs) must process lengthy web page observations to complete user goals; these pages often …
Imene Kerboua, Sahar Omidi Shayegan, Xing Han Lu, Léo Boisvert, Megh Thakkar, Massimo Caccia, Jérémy Espinas, Alex Aussem, Véronique Eglin, Alexandre Lacoste
Transactions on Machine Learning Research (TMLR),  2025.
Paper Cite Code
Fine-Tune an SLM or Prompt an LLM? The Case of Generating Low-Code Workflows
Fine-Tune an SLM or Prompt an LLM? The Case of Generating Low-Code Workflows
Large Language Models (LLMs) such as GPT-4o can handle a wide range of complex tasks with the right prompt. As per token costs are …
Orlando Marquez, Patrice Béchard, Emily Chen, Maggie Baird, JingFei Chen
Knowledge Discovery and Data Mining,  2025.
Paper Cite
AgentAda: Skill-Adaptive Data Analytics for Tailored Insight Discovery
AgentAda: Skill-Adaptive Data Analytics for Tailored Insight Discovery
We introduce AgentAda, the first LLM-powered analytics agent that can learn and use new analytics skills to extract more specialized …
Amirhossein Abaskohi, Amrutha Ramesh, Shailesh Nanisetty, Chirag Goel, David Vazquez, Christopher Pal, Spandana Gella, Giuseppe Carenini, Issam H. Laradji
Workshop at the Annual Meeting of the Association for Computational Linguistics (ACL),  2025.
Paper Cite
DoomArena: A framework for Testing AI Agents Against Evolving Security Threats
DoomArena: A framework for Testing AI Agents Against Evolving Security Threats
We present DoomArena, a security evaluation framework for AI agents. DoomArena is designed on three principles: 1) It is a …
Léo Boisvert, Abhay Puri, Gabriel Huang, Mihir Bansal, Chandra Kiran Reddy Evuru, Avinandan Bose, Quentin Cappart, Maryam Fazel, Alexandre Lacoste, Alexandre Drouin, Jason Stanley, Krishnamurthy (Dj) Dvijotham
Workshop at the International Conference of Machine Learning (ICML),  2025.
Paper Cite Code
Silent Sabotage: Injecting Backdoors into AI Agents Through Fine-Tuning
Silent Sabotage: Injecting Backdoors into AI Agents Through Fine-Tuning
The rise of AI agents that can use tools, browse the web and interact with computers on behalf of a user, has sparked strong interest …
Léo Boisvert, Abhay Puri, Chandra Kiran Reddy Evuru, Joshua Kazdan, Avinandan Bose, Quentin Cappart, Maryam Fazel, Sai Rajeswar Mudumba, Jason Stanley, Nicolas Chapados, Alexandre Drouin, Krishnamurthy (Dj) Dvijotham
Workshop at the International Conference of Machine Learning (ICML),  2025.
Paper Cite
SafeArena: Evaluating the Safety of Autonomous Web Agents
SafeArena: Evaluating the Safety of Autonomous Web Agents
LLM-based agents are becoming increasingly proficient at solving web-based tasks. With this capability comes a greater risk of misuse …
Ada Tur, Nicholas Meade, Xing Han Lu, Alejandra Zambrano, Arkil Patel, Esin Durmus, Spandana Gella, Karolina Stanczak, Siva Reddy
International Conference on Machine Learning (ICML),  2025.
Paper Cite Code