ServiceNow AI Research

Benchmark

Context is Key: A Benchmark for Forecasting with Essential Textual Information
Context is Key: A Benchmark for Forecasting with Essential Textual Information
Forecasting is a critical task in decision making across various domains. While numerical data provides a foundation, it often lacks …
Andrew Williams, Arjun Ashok, Étienne Marcotte, Valentina Zantedeschi, Jithendaraa Subramanian, Roland Riachi, James Requeima, Alexandre Lacoste, Irina Rish, Nicolas Chapados, Alexandre Drouin
Montreal AI Symposium (MAIS),  2024.
Paper Cite Code
Benchmarking Bayesian Causal Discovery Methods for Downstream Treatment Effect Estimation
Benchmarking Bayesian Causal Discovery Methods for Downstream Treatment Effect Estimation
The practical utility of causality in decision-making is widely recognized, with causal discovery and inference being inherently …
Chris Chinenye Emezue, Alexandre Drouin, Tristan Deleu, Stefan Bauer , Yoshua Bengio
Workshop on Structured Probabilistic Inference & Generative Modeling (ICML),  2023.
Paper Cite
Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels
Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels
Controlling artificial agents from visual sensory data is an arduous task. Reinforcement learning (RL) algorithms can succeed but …
Sai Rajeswar Mudumba, Pietro Mazzaglia, Tim Verbelen, Alexandre Piche, Bart Dhoedt, Aaron Courville, Alexandre Lacoste
International Conference on Machine Learning (ICML),  2023.
Paper Cite Code
The Stack: 3 TB of permissively licensed source code
The Stack: 3 TB of permissively licensed source code
Large Language Models (LLMs) play an ever-increasing role in the field of Artificial Intelligence (AI)–not only for natural …
Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Carlos Muñoz Ferrandis, Yacine Jernite, Margaret Mitchell, Sean Hughes, Thomas Wolf , Dzmitry Bahdanau, Leandro von Werra, Harm de Vries
Transactions on Machine Learning Research (TMLR),  2022.
Paper Cite Code
In search of robust measures of generalization
In search of robust measures of generalization
One of the principal scientific challenges in deep learning is explaining generalization, i.e., why the particular way the community …
Gintare Karolina Dziugaite, Alexandre Drouin, Brayden (Brady) Neal, Nitarshan Rajkumar, Ethan Victor Caballero, Linbo Wang, Ioannis Mitliagkas, Daniel M. Roy
Conference on Neural Information Processing Systems (NeurIPS),  2020.
Paper Cite Code Video