About
People
Publications
Events
Blog
Careers
Contact
English
English
Français
ServiceNow
ServiceNow AI Research
Tags
Large Language Models
ServiceNow AI Research
Large Language Models
Context is Key: A Benchmark for Forecasting with Essential Textual Information
Forecasting is a critical task in decision making across various domains. While numerical data provides a foundation, it often lacks …
Andrew Williams
,
Arjun Ashok
,
Étienne Marcotte
,
Valentina Zantedeschi
,
Jithendaraa Subramanian
,
Roland Riachi
,
James Requeima
,
Alexandre Lacoste
,
Irina Rish
,
Nicolas Chapados
,
Alexandre Drouin
Workshop at the Neural Information Processing Systems (NeurIPS), 2024.
Paper
Cite
Code
Video
Evaluating Interventional Reasoning Capabilities of Large Language Models
Numerous decision-making tasks require estimating causal effects under interventions on different parts of a system. As practitioners …
Tejas Kasetty
,
Divyat Mahajan
,
Gintare Karolina Dziugaite
,
Alexandre Drouin
,
Dhanya Sridhar
Workshop at the Neural Information Processing Systems (NeurIPS), 2024.
Paper
Cite
RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Content
Large Language Models (LLMs) are trained on vast amounts of data, most of which is automatically scraped from the internet. This data …
João Monteiro
,
Pierre-André Noël
,
Étienne Marcotte
,
Sai Rajeswar Mudumba
,
Valentina Zantedeschi
,
David Vazquez
,
Nicolas Chapados
,
Christopher Pal
,
Perouz Taslakian
NeurIPS Datasets and Benchmarks Track (NeurIPS Datasets), 2024.
Paper
Cite
Code
Video
XC-Cache: Cross-Attending to Cached Context for Efficient LLM Inference
In-context learning (ICL) approaches typically leverage prompting to condition decoder-only language model generation on reference …
João Monteiro
,
Étienne Marcotte
,
Pierre-André Noël
,
Valentina Zantedeschi
,
David Vazquez
,
Nicolas Chapados
,
Christopher Pal
,
Perouz Taslakian
Workshop at the Neural Information Processing Systems (NeurIPS), 2024.
Paper
Cite
Code
Change Is the Only Constant: Dynamic LLM Slicing based on Layer Redundancy
This paper introduces a novel model compression approach through dynamic layer-specific pruning in Large Language Models (LLMs), …
Razvan-Gabriel Dumitru
,
Paul-Ioan Clotan
,
Vikas Yadav
,
Darius Peteleaza
,
Mihai Surdeanu
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024.
Paper
Cite
Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferences
Direct Preference Optimization (DPO) is an effective technique that leverages pairwise preference data (usually one chosen and rejected …
Pulkit Pattnaik
,
Rishabh Maheshwary
,
Kelechi Ogueji
,
Vikas Yadav
,
Sathwik Tejaswi Madhusudhan
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024.
Paper
Cite
XC-Cache: Cross-Attending to Cached Context for Efficient LLM Inference
In-context learning (ICL) approaches typically leverage prompting to condition decoder-only language model generation on reference …
João Monteiro
,
Étienne Marcotte
,
Pierre-André Noël
,
Valentina Zantedeschi
,
David Vazquez
,
Nicolas Chapados
,
Christopher Pal
,
Perouz Taslakian
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024.
Paper
Cite
Code
An Ecosystem for Web Agents: WorkArena, BrowserGym, AgentLab and more
The BrowserGym ecosystem addresses the growing need for efficient evaluation and benchmarking of web agents, particularly those …
Alexandre Lacoste
,
Maxime Gasse
,
Thibault Le Sellier De Chezelles
,
Massimo Caccia
,
Léo Boisvert
,
Megh Thakkar
,
Alexandre Drouin
,
Nicolas Chapados
Montreal AI Symposium (MAIS), 2024.
Cite
LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
Large decoder-only language models (LLMs) are the state-of-the-art models on most of today’s NLP tasks and benchmarks. Yet, the …
Parishad BehnamGhader
,
Vaibhav Adlakha
,
Marius Mosbach
,
Dzmitry Bahdanau
,
Nicolas Chapados
,
Siva Reddy
Conference on Language Modeling (COLM), 2024.
Paper
Cite
Code
WorkArena: How Capable are Web Agents at Solving Common Knowledge Work Tasks?
We study the use of large language model-based agents for interacting with software via web browsers. Unlike prior work, we focus on …
Alexandre Drouin
,
Maxime Gasse
,
Massimo Caccia
,
Issam H. Laradji
,
Manuel Del Verme
,
Tom Marty
,
Léo Boisvert
,
Megh Thakkar
,
Quentin Cappart
,
David Vazquez
,
Nicolas Chapados
,
Alexandre Lacoste
International Conference on Machine Learning (ICML), 2024.
Paper
Cite
Code
Video
«
»
Cite
×