ServiceNow Research

TACTiS: Transformer-attentional copulas for time series

Abstract

The estimation of time-varying quantities is a fundamental component of decision making in fields such as healthcare and finance. However, the practical utility of such estimates is limited by how accurately they quantify predictive uncertainty. In this work, we address the problem of estimating the joint predictive distribution of high-dimensional multivariate time series. We propose a versatile method, based on the transformer architecture, that estimates joint distributions using an attention-based decoder that provably learns to mimic the properties of non-parametric copulas. The resulting model has several desirable properties: it can scale to hundreds of time series, supports both forecasting and interpolation, can handle unaligned and non-uniformly sampled data, and can seamlessly adapt to missing data during training. We demonstrate these properties empirically and show that our model produces state-of-the-art predictions on multiple real-world datasets.

Publication
Montreal AI Symposium (MAIS)
Alexandre Drouin
Alexandre Drouin
Research Lead

Research Lead at Human Decision Support located at Montreal, QC, Canada.

Étienne Marcotte
Étienne Marcotte
Applied Research Scientist

Applied Research Scientist at Human Decision Support located at Montreal, QC, Canada.

Nicolas Chapados
Nicolas Chapados
VP of Research

VP of Research at Research Management located at Montreal, QC, Canada.