TACTiS-2: A new model for time series prediction

TACTiS-2: Better, faster, simpler attentional copulas for multivariate probabilistic time series prediction

Authors: Arjun Ashok, Étienne Marcotte, Valentina Zantedeschi, Nicolas Chapados, and Alexandre Drouin

Time series data is ubiquitous, measuring every aspect of our lives, from stock markets and climate patterns to energy use and health conditions. This type of data captures sequences of values that unfold over time, providing a historical record that can be analyzed to uncover patterns, trends, and seasonal variations.

In today's data-driven world, time series prediction stands as a critical tool, enabling businesses, scientists, and policymakers to make informed decisions by predicting future events based on past observations.

Our new work, TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series—a second iteration of our TACTiS model—proposes a flexible model that marks a significant advancement in our ongoing pursuit of developing “push-button” time series prediction models that necessitate minimal input from practitioners.

A general-purpose model

TACTiS is a general - purpose model that can perform a wide range of prediction tasks and support several stylized facts that are ubiquitous in real-world time series.

For instance, TACTiS can gracefully handle settings where groups of time series can have different sampling frequencies or be sampled irregularly. Such properties are commonly found and important to handle in settings such as healthcare, where different patients enter clinical trials at different frequencies, or at irregular moments.

TACTiS can further work in settings where the data exhibits missing values, which is a common problem in many time series due to measurement error or equipment failure.

Where a large number of related time series are available, TACTiS can scalably model the dependencies between all such series and improve prediction (e.g., across 862 time series for the traffic benchmark used in the paper for traffic forecasting on different roads).

TACTiS can also incorporate different types of covariates available in the data. These refer to variables (e.g., day of the week) that may be available in advance and can aid in the prediction of the other quantities.

A flexible architecture

Along with its flexibility, TACTiS can perform a wide range of time series prediction tasks. For instance, it can perform forecasting, the most common task, where future values have to be predicted from the past. It can also perform interpolation, where gaps in recorded time series data need to be filled for the data to be analyzed. It can also perform any combination of forecasting and interpolation, depending on the use case.

For all these scenarios, TACTiS outputs what are known as probabilistic predictions, not just the single best predictive value (as most forecasting techniques in use today do). It also outputs the shape of the probability distribution of possible values. This makes it easy to compute confidence intervals and understand the model uncertainty about the future.

TACTiS gets its flexibility from its architecture. TACTiS uses an encoder-decoder transformer, where the encoder uses a tokenization strategy that allows it to consume all different time series data and perform arbitrary tasks.

The decoder learns a nonparametric copula, which is a statistical object that learns dependencies across different time series. This ability to represent a copula with a neural network, using a technique known as an attentional copula, is a breakthrough introduced in the original TACTiS model (hence, its name: Transformer Attentional Copulas for Time Series).

Top of the game

We test TACTiS on a wide range of problems, such as electricity consumption prediction, traffic forecasting, macroeconomic indicator prediction, solar energy consumption forecasting, and so on. It is state-of-the-art in terms of accuracy of prediction compared to various other models for time series prediction.

TACTiS learns rich dependencies between different time series to use for prediction. The forecasts closely match the actual values when visualized, as shown in the two examples below.

Two graphs showing forecasts closely matching the actual values

Ease of use

We have packaged the model and made it easy to use with your own data and tasks. The hands-on tutorial gives examples of how to use the model.

An example code snippet below, adapted from the tutorial, shows how to create a forecasting model out of TACTiS. It specifies the:

Hyperparameters, such as the number of layers, of the TACTIS model
Task (forecasting, in this case)
Number of time series in the dataset
Frequency of your data
Context and prediction horizon to be used, depending on your use case.

It then provides the training arguments (such as the number of epochs and learning rate), and trains the model on your data, given a training set and a validation set.

forecaster = TACTiSEstimator(
    model_parameters = {
        HYPERPARAMETERS,
        "experiment_mode": "forecasting" 
    },
    num_series = NUMBER_OF_TIME_SERIES_IN_THE_DATA,
    freq = FREQUENCY_OF_THE_DATA,
    history_length = HISTORY_LENGTH,
    prediction_length = PREDICTION_LENGTH,
    trainer = TACTISTrainer(
        TRAINING_ARGUMENTS
    ),
)
trained_forecaster = forecaster.train(training_data, validation_data)

Research: https://arxiv.org/abs/2310.01327

Code: https://github.com/servicenow/tactis

Tutorial: https://colab.research.google.com/drive/1lFoAZAR7HqbXBUqVK3ImpBkRWS9dEK9O?usp=sharing

Find out more about ServiceNow Research.