ServiceNow Research

Benchmark

GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks
GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks
While numerous recent benchmarks focus on evaluating generic Vision-Language Models (VLMs), they fall short in addressing the unique …
International Conference on Computer Vision (ICCV),  2025.
PDF Cite
WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
Understanding diverse web data and automating web development presents an exciting challenge for agentic AI. While existing benchmarks …
Workshop at the Computer Vision and Pattern Recognition Conference (CVPR),  2025.
PDF Cite
The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applications
The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applications
Causal discovery aims to automatically uncover causal relationships from data, a capability with significant potential across many …
Causal Learning and Reasoning (CLeaR),  2025.
PDF Cite
WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
Understanding diverse web data and automating web development presents an exciting challenge for agentic AI. While existing benchmarks …
Workshop at the International Conference of Learning Representation (ICLR),  2025.
PDF Cite
MMTEB: Massive Multilingual Text Embedding Benchmark
MMTEB: Massive Multilingual Text Embedding Benchmark

Text embeddings are typically evaluated on a narrow set of tasks, limited in terms of languages, domains, and task types. To circumvent …

International Conference of Learning Representations (ICLR),  2025.
PDF Cite Code
EarthView: A Large Scale Remote Sensing Dataset for Self-Supervision
EarthView: A Large Scale Remote Sensing Dataset for Self-Supervision
This paper presents EarthView, a comprehensive dataset specifically designed for self-supervision on remote sensing data, intended to …
Workshop at the Winter Conference on Applications of Computer Vision (WACV),  2025.
PDF Cite Code
Context is Key: A Benchmark for Forecasting with Essential Textual Information
Context is Key: A Benchmark for Forecasting with Essential Textual Information
Forecasting is a critical task in decision making across various domains. While numerical data provides a foundation, it often lacks …
Workshop at the Neural Information Processing Systems (NeurIPS),  2024.
PDF Cite Code Video
Context is Key: A Benchmark for Forecasting with Essential Textual Information
Context is Key: A Benchmark for Forecasting with Essential Textual Information
Forecasting is a critical task in decision making across various domains. While numerical data provides a foundation, it often lacks …
Montreal AI Symposium (MAIS),  2024.
PDF Cite Code
Benchmarking Bayesian Causal Discovery Methods for Downstream Treatment Effect Estimation
Benchmarking Bayesian Causal Discovery Methods for Downstream Treatment Effect Estimation
The practical utility of causality in decision-making is widely recognized, with causal discovery and inference being inherently …
Workshop on Structured Probabilistic Inference & Generative Modeling (ICML),  2023.
PDF Cite
Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels
Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels
Controlling artificial agents from visual sensory data is an arduous task. Reinforcement learning (RL) algorithms can succeed but …
International Conference on Machine Learning (ICML),  2023.
PDF Cite Code