ServiceNow AI Research

Multi-Modal Learning

WebArena-Pro: A Heterogeneous, Multimodal, Reproducible Benchmark for Web Agents
WebArena-Pro: A Heterogeneous, Multimodal, Reproducible Benchmark for Web Agents
Web agents powered by large language and vision-language models are increasingly applied to realistic browser work that spans …
Imene Kerboua, Fatemeh Pesaran, Xing Han Lu, Weijian Qi, Alexander Miller, Junyi Song, Yunjia Tian, Dongjin Kang, Seyeon Choi, Marzia Nouri, Ewen Gueguen, Matteo Boglioni, Fengyuan Liu, Zeyi Liao, Mengqi Yuan, Yue Li, Alexandre Lacoste, Alexandre Drouin, Spandana Gella, Huan Sun, Gunhee Kim, Siva Reddy
NeurIPS Datasets and Benchmarks Track (NeurIPS Datasets),  2026.
Paper Cite Code
VectorGym: A Multi-Task Benchmark for SVG Code Generation and Manipulation
VectorGym: A Multi-Task Benchmark for SVG Code Generation and Manipulation
We introduce VectorGym, a new comprehensive multi-task benchmark for evaluating Vision-Language Models (VLMs) on Scalable Vector …
Juan A. Rodriguez, Haotian Zhang, Abhay Puri, Tianyang Zhang, Rishav Pramanik, Meng Lin, Xiaoqing Xie, Marco Terral, Darsh Kaushik, Aly Shariff, Perouz Taslakian, Spandana Gella, Sai Rajeswar Mudumba, David Vazquez, Christopher Pal, Marco Pedersoli
Conference on Empirical Methods in Natural Language Processing (EMNLP),  2026.
Paper Cite Code
Improving GUI Grounding with Explicit Position-to-Coordinate Mapping
Improving GUI Grounding with Explicit Position-to-Coordinate Mapping
GUI grounding, the task of mapping natural-language instructions to pixel coordinates, is crucial for autonomous agents, yet remains …
Suyuchen Wang, Tianyu Zhang, Ahmed Masry, Christopher Pal, Spandana Gella, Bang Liu, Perouz Taslakian
Conference on Language Modeling (COLM),  2026.
Paper Cite Code
Grounding Computer Use Agents on Human Demonstrations
Grounding Computer Use Agents on Human Demonstrations
Building reliable computer-use agents requires grounding: accurately connecting natural language instructions to the correct on-screen …
Aarash Feizi, Shravan Nayak, Xiangru Jian, Kevin Qinghong Lin, Kaixin Li, Rabiul Awal, Xing Han Lu, Johan Obando, Juan A. Rodriguez, Nicolas Chapados, David Vazquez, Adriana Romero Soriano, Reihaneh Rabbany, Perouz Taslakian, Christopher Pal, Spandana Gella, Sai Rajeswar Mudumba
International Conference on Learning Representations,  2026.
Paper Cite Code
StarFlow: Generating Structured Workflow Outputs From Sketch Images
StarFlow: Generating Structured Workflow Outputs From Sketch Images
Workflows are a fundamental component of automation in enterprise platforms, enabling the orchestration of tasks, data processing, and …
Patrice Béchard, Chao Wang, Juan A. Rodriguez, Amirhossein Abaskohi, Christopher Pal, David Vazquez, Spandana Gella, Sai Rajeswar Mudumba, Perouz Taslakian
European Chapter of the Association for Computational Linguistics (EACL),  2026.
Paper Cite Code Video
AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal Document Understanding
AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal Document Understanding
Aligning visual features with language embeddings is a key challenge in vision-language models (VLMs). The performance of such models …
Ahmed Masry, Juan A. Rodriguez, Tianyu Zhang, Suyuchen Wang, Chao Wang, Aarash Feizi, Akshay Kalkunte, Abhay Puri, Xiangru Jian, Pierre-André Noël, Sathwik Madhusudhan, Marco Pedersoli, Bang Liu, Nicolas Chapados, Yoshua Bengio, Enamul Hoque Prince , Christopher Pal, Issam H. Laradji, David Vazquez, Perouz Taslakian, Spandana Gella, Sai Rajeswar Mudumba
Neural Information Processing Systems (NeurIPS),  2025.
Paper Cite Code Video
Rendering-Aware Reinforcement Learning for Vector Graphics Generation
Rendering-Aware Reinforcement Learning for Vector Graphics Generation
Scalable Vector Graphics (SVG) offer a powerful format for representing visual designs as interpretable code. Recent advances in …
Juan A. Rodriguez, Haotian Zhang, Abhay Puri, Rishav Pramanik, Aarash Feizi, Pascal Wichmann, Arnab Mondal, Mohammad Reza Samsami, Rabiul Awal, Perouz Taslakian, Spandana Gella, Sai Rajeswar Mudumba, David Vazquez, Christopher Pal, Marco Pedersoli
Neural Information Processing Systems (NeurIPS),  2025.
Paper Cite
The Promise of RL for Autoregressive Image Editing
The Promise of RL for Autoregressive Image Editing
While image generation techniques are now capable of producing high quality images that respect prompts which span multiple sentences, …
Saba Ahmadi, Rabiul Awal, Ankur Sikarwar, Amirhossein Kazemnejad, Ge Ya Luo, Juan A. Rodriguez, Sai Rajeswar Mudumba, Siva Reddy, Christopher Pal, Benno Krojer, Aishwarya Agrawal
Neural Information Processing Systems (NeurIPS),  2025.
Paper Cite Code
ColMate: Contrastive Late Interaction and Masked Text for Multimodal Document Retrieval
ColMate: Contrastive Late Interaction and Masked Text for Multimodal Document Retrieval
Retrieval-augmented generation has proven practical when models require specialized knowledge or access to the latest data. However, …
Ahmed Masry, Megh Thakkar, Patrice Béchard, Sathwik Madhusudhan, Rabiul Awal, Shambhavi Mishra, Akshay Kalkunte, Enamul Hoque Prince , Spandana Gella, Torsten Scholak, Sai Rajeswar Mudumba
Conference on Empirical Methods in Natural Language Processing (EMNLP),  2025.
Paper Cite
WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
Understanding diverse web data and automating web development presents an exciting challenge for agentic multimodal models. While …
Rabiul Awal, Mahsa Massoud, Zichao Li, Aarash Feizi, Suyuchen Wang, Christopher Pal, Aishwarya Agrawal, David Vazquez, Perouz Taslakian, Spandana Gella, Sai Rajeswar Mudumba
Conference on Empirical Methods in Natural Language Processing (EMNLP),  2025.
Paper Cite Code Video