ServiceNow AI Research

Transformers

Systematic Generalization with Edge Transformers
Systematic Generalization with Edge Transformers
Recent research suggests that systematic generalization in natural language understanding remains a challenge for state-of-the-art …
Conference on Neural Information Processing Systems (NeurIPS),  2021.
PDF Cite
Measuring Systematic Generalization in Neural Proof Generation with Transformers
Measuring Systematic Generalization in Neural Proof Generation with Transformers
We are interested in understanding how well Transformer language models (TLMs) can perform reasoning tasks when trained on knowledge …
Conference on Neural Information Processing Systems (NeurIPS),  2020.
PDF Cite Code Slides
On Extractive and Abstractive Neural Document Summarization with Transformer Language Models
On Extractive and Abstractive Neural Document Summarization with Transformer Language Models
We present a method to produce abstractive summaries of long documents that exceed several thousand words via neural abstractive …
Conference on Empirical Methods in Natural Language Processing (EMNLP),  2020.
PDF Cite