About
People
Publications
Open Source
Demos
Events
Blog
Careers
Contact
English
English
Français
ServiceNow
ServiceNow AI Research
Tags
Agents
ServiceNow AI Research
Agents
SafeArena: Evaluating the Safety of Autonomous Web Agents
LLM-based agents are becoming increasingly proficient at solving web-based tasks. With this capability comes a greater risk of misuse …
Ada Tur
,
Nicholas Meade
,
Xing Han Lu
,
Alejandra Zambrano
,
Arkil Patel
,
Esin Durmus
,
Spandana Gella
,
Karolina Stanczak
,
Siva Reddy
International Conference on Machine Learning (ICML), 2025.
PDF
Cite
Code
UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction
Developing autonomous agents that can navigate diverse Graphical User Interfaces (GUIs) and solve complex tasks is essential for …
Shravan Nayak
,
Xiangru Jian
,
Kevin Lin
,
Juan A. Rodriguez
,
Motek Kalsi
,
Nicolas Chapados
,
Tamer Özsu
,
Aishwarya Agrawal
,
David Vazquez
,
Christopher Pal
,
Perouz Taslakian
,
Spandana Gella
,
Sai Rajeswar Mudumba
International Conference on Machine Learning (ICML), 2025.
PDF
Cite
Code
Video
WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
Understanding diverse web data and automating web development presents an exciting challenge for agentic AI. While existing benchmarks …
Rabiul Awal
,
Mahsa Massoud
,
Zichao Li
,
Aarash Feizi
,
Suyuchen Wang
,
Christopher Pal
,
Aishwarya Agrawal
,
David Vazquez
,
Siva Reddy
,
Juan A. Rodriguez
,
Perouz Taslakian
,
Sai Rajeswar Mudumba
Workshop at the Computer Vision and Pattern Recognition Conference (CVPR), 2025.
PDF
Cite
StarVector: Generating Scalable Vector Graphics Code from Images and Text
Scalable Vector Graphics (SVGs) are vital for modern image rendering due to their scalability and versatility. Previous SVG generation …
Juan A. Rodriguez
,
Abhay Puri
,
Shubham Agarwal
,
Issam H. Laradji
,
Pau Rodriguez
,
Sai Rajeswar Mudumba
,
David Vazquez
,
Christopher Pal
,
Marco Pedersoli
Computer Vision and Pattern Recognition (CVPR), 2025.
PDF
Cite
Code
Video
Keeping up with dynamic attackers: Certifying robustness to adaptive online data poisoning
The rise of foundation models fine-tuned on human feedback from potentially untrusted users has increased the risk of adversarial data …
Avinandan Bose
,
Laurent Lessard
,
Maryam Fazel
,
Krishnamurthy (Dj) Dvijotham
International Conference on Artificial Intelligence and Statistics (AISTATS), 2025.
PDF
Cite
Code
Video
A Guide To Effectively Leveraging LLMs for Low-Resource Text Summarization: Data Augmentation and Semi-supervised Approaches
Existing approaches for low-resource text summarization primarily employ large language models (LLMs) like GPT-3 or GPT-4 at inference …
Gaurav Sahu
,
Olga Vechtomova
,
Issam H. Laradji
North American Chapter of the Association for Computational Linguistics (NAACL), 2025.
PDF
Cite
Code
Generating a Low-code Complete Workflow via Task Decomposition and RAG
AI technologies are moving rapidly from research to production. With the popularity of Foundation Models (FMs) that generate text, …
Orlando Marquez
,
Patrice Béchard
Conference on AI Engineering (CAIN), 2025.
PDF
Cite
Societal Alignment Frameworks Can Improve LLM Alignment
Recent progress in large language models (LLMs) has focused on producing responses that meet human expectations and align with shared …
Karolina Stanczak
,
Nicholas Meade
,
Mehar Bhatia
,
Hattie Zhou
,
Konstantin Böttinger
,
Jeremy Barns
,
Jason Stanley
,
Nicolas Papernot
,
Nicolas Chapados
,
Denis Therien
,
Timothy P Lillicrap
,
Ana Marasovic
,
Sylvie Delacroix
,
Gillian K Hadfield
,
Siva Reddy
Workshop at the International Conference of Learning Representation (ICLR), 2025.
PDF
Cite
BigDocs: An Open and Permissively-Licensed Dataset for Training Multimodal Models on Document and Code Tasks
Multimodal AI has the potential to significantly enhance document-understanding tasks, such as processing receipts, understanding …
Juan A. Rodriguez
,
Xiangru Jian
,
Siba Smarak Panigrahi
,
Tianyu Zhang
,
Aarash Feizi
,
Abhay Puri
,
Akshay Kalkunte
,
Francois Savard
,
Ahmed Masry
,
Shravan Nayak
,
Rabiul Awal
,
Mahsa Massoud
,
Amirhossein Abaskohi
,
Zichao Li
,
Suyuchen Wang
,
Pierre-André Noël
,
Mats L. Richter
,
Saverio Vadacchino
,
Shubham Agarwal
,
Sanket Biswas
,
Sara Shanian
,
Ying Zhang
,
Sathwik Tejaswi Madhusudhan
,
João Monteiro
,
Krishnamurthy (Dj) Dvijotham
,
Torsten Scholak
,
Nicolas Chapados
,
Sepideh Kharaghani
,
Sean Hughes
,
Tamer Özsu
,
Siva Reddy
,
Marco Pedersoli
,
Yoshua Bengio
,
Christopher Pal
,
Issam H. Laradji
,
Spandana Gella
,
Perouz Taslakian
,
David Vazquez
,
Sai Rajeswar Mudumba
International Conference of Learning Representations (ICLR), 2025.
PDF
Cite
Code
Video
InsightBench: Evaluating Business Analytics Agents Through Multi-Step Insight Generation
Data analytics is essential for extracting valuable insights from data that can assist organizations in making effective decisions. We …
Gaurav Sahu
,
Abhay Puri
,
Juan A. Rodriguez
,
Amirhossein Abaskohi
,
Mohammad (Aaron) Chegini
,
Alexandre Drouin
,
Perouz Taslakian
,
Valentina Zantedeschi
,
Alexandre Lacoste
,
David Vazquez
,
Nicolas Chapados
,
Christopher Pal
,
Sai Rajeswar Mudumba
,
Issam H. Laradji
International Conference of Learning Representations (ICLR), 2025.
PDF
Cite
Code
«
»
Cite
×