ServiceNow Research

Efficient Learning

Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts
Merging parameter-efficient task experts has recently gained growing attention as a way to build modular architectures that can be …
Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts
Merging parameter-efficient task experts has recently gained growing attention as a way to build modular architectures that can be …
Unifying Autoregressive and Diffusion-Based Sequence Generation
We take significant steps toward unifying autoregressive and diffusion-based sequence generation by extending the SEDD discrete …
Automatic Data Augmentation Learning using Bilevel Optimization for Histopathological Images
One of the main challenges faced when training a deep learning based model to classify histopathological images is the color and shape …
Workflow discovery in low data regimes
Text-based dialogues are now widely used to solve real-world problems. In cases where solution strategies are already known, they can …
Breadth-First Pipeline Parallelism
We introduce Breadth-First Pipeline Parallelism, a novel training schedule which optimizes the combination of pipeline and data …
Unsupervised Model-based Pre-training for Data-efficient Reinforcement Learning from Pixels
Reinforcement learning (RL) aims at autonomously performing complex tasks. To this end, a reward signal is used to steer the learning …
Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data
Multi-Task Learning (MTL) networks have emerged as a promising method for transferring learned knowledge across different tasks. …