ServiceNow IA recherche

Benchmark

GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks
GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks
While numerous recent benchmarks focus on evaluating generic Vision-Language Models (VLMs), they fall short in addressing the unique …
International Conference on Computer Vision (ICCV),  2025.
PDF Citation Code
WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
Understanding diverse web data and automating web development presents an exciting challenge for agentic AI. While existing benchmarks …
Workshop at the Computer Vision and Pattern Recognition Conference (CVPR),  2025.
PDF Citation
The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applications
The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applications
Causal discovery aims to automatically uncover causal relationships from data, a capability with significant potential across many …
Causal Learning and Reasoning (CLeaR),  2025.
PDF Citation Code
WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation
Understanding diverse web data and automating web development presents an exciting challenge for agentic AI. While existing benchmarks …
Workshop at the International Conference of Learning Representation (ICLR),  2025.
PDF Citation
MMTEB: Massive Multilingual Text Embedding Benchmark
MMTEB: Massive Multilingual Text Embedding Benchmark

Text embeddings are typically evaluated on a narrow set of tasks, limited in terms of languages, domains, and task types. To circumvent …

Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzeminski, Genta Indra Winata, Saba Sturua, Saiteja Utpala, Mathieu Ciancone, Marion Schaeffer, Gabriel Sequeira, Diganta Misra, Shreeya Dhakal, Jonathan Rystrøm, Roman Solomatin, Ömer Çagatan, Akash Kundu, Martin Bernstorff, Shitao Xiao, Akshita Sukhlecha, Bhavish Pahwa, Rafał Poswiata, Kranthi Kiran GV, Shawon Ashraf, Daniel Auras, Björn Plüster, Jan Philipp Harries, Loïc Magne, Isabelle Mohr, Mariya Hendriksen, Dawei Zhu, Hippolyte Gisserot-Boukhlef, Tom Aarsen, Jan Kostkan, Konrad Wojtasik, Taemin Lee, Marek Šuppa, Crystina Zhang, Roberta Rocca, Mohammed Hamdy, Andrianos Michail, John Yang, Manuel Faysse, Aleksei Vatolin, Nandan Thakur, Manan Dey, Dipam Vasani, Pranjal Chitale, Simone Tedeschi, Nguyen Tai, Artem Snegirev, Michael Günther, Mengzhou Xia, Weijia Shi, Xing Han Lu, Jordan Clive, Gayatri Krishnakumar, Anna Maksimova, Silvan Wehrli, Maria Tikhonova, Henil Pancha, Aleksandr Abramov, Malte Ostendorff, Zheng Liu, Simon Clematide, Lester James Miranda, Alena Fenogenova, Guangyu Song, Ruqiya Bin Saf, Wen-Ding Li, Alessia Borghini, Federico Cassano, Hongjin Su, Jimmy Lin, Howard Yen, Lasse Hansen, Sara Hooker, Chenghao Xiao, Vaibhav Adlakha, Orion Weller, Siva Reddy, Niklas Muennighoff
International Conference of Learning Representations (ICLR),  2025.
PDF Citation Code
EarthView: A Large Scale Remote Sensing Dataset for Self-Supervision
EarthView: A Large Scale Remote Sensing Dataset for Self-Supervision
This paper presents EarthView, a comprehensive dataset specifically designed for self-supervision on remote sensing data, intended to …
Workshop at the Winter Conference on Applications of Computer Vision (WACV),  2025.
PDF Citation Code
Context is Key: A Benchmark for Forecasting with Essential Textual Information
Context is Key: A Benchmark for Forecasting with Essential Textual Information
Forecasting is a critical task in decision making across various domains. While numerical data provides a foundation, it often lacks …
Workshop at the Neural Information Processing Systems (NeurIPS),  2024.
PDF Citation Code Vidéo
Context is Key: A Benchmark for Forecasting with Essential Textual Information
Context is Key: A Benchmark for Forecasting with Essential Textual Information
Forecasting is a critical task in decision making across various domains. While numerical data provides a foundation, it often lacks …
Montreal AI Symposium (MAIS),  2024.
PDF Citation Code
Benchmarking Bayesian Causal Discovery Methods for Downstream Treatment Effect Estimation
Benchmarking Bayesian Causal Discovery Methods for Downstream Treatment Effect Estimation
The practical utility of causality in decision-making is widely recognized, with causal discovery and inference being inherently …
Workshop on Structured Probabilistic Inference & Generative Modeling (ICML),  2023.
PDF Citation
Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels
Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels
Controlling artificial agents from visual sensory data is an arduous task. Reinforcement learning (RL) algorithms can succeed but …
International Conference on Machine Learning (ICML),  2023.
PDF Citation Code