ServiceNow AI Research

Large Language Models

Unifying Autoregressive and Diffusion-Based Sequence Generation
Unifying Autoregressive and Diffusion-Based Sequence Generation
We present significant extensions to diffusion-based sequence generation models, blurring the line with autoregressive language models. …
Nima Fathi, Torsten Scholak, Pierre-André Noël
Conference on Language Modeling (COLM),  2025.
Paper Cite Code Video
LLMs can learn self-restraint through iterative self-reflection
LLMs can learn self-restraint through iterative self-reflection
In order to be deployed safely, Large Language Models (LLMs) must be capable of dynamically adapting their behavior based on their …
Alexandre Piche, Aristides Milios, Dzmitry Bahdanau, Christopher Pal
Transactions on Machine Learning Research (TMLR),  2025.
Paper Cite Video
Auto-Cypher: Improving LLMs on Cypher generation via LLM-supervised generation-verification framework
Auto-Cypher: Improving LLMs on Cypher generation via LLM-supervised generation-verification framework
Graph databases like Neo4j are gaining popularity for handling complex, interconnected data, over traditional relational databases in …
Aman Tiwari, Shiva Krishna Reddy Malay, Vikas Yadav, Masoud Hashemi, Sathwik Tejaswi Madhusudhan
North American Chapter of the Association for Computational Linguistics (NAACL),  2025.
Paper Cite
M2Lingual: Enhancing Multilingual, Multi-Turn Instruction Alignment in Large Language Models
M2Lingual: Enhancing Multilingual, Multi-Turn Instruction Alignment in Large Language Models
Instruction finetuning (IFT) is critical for aligning Large Language Models (LLMs) to follow instructions. While many effective IFT …
Rishabh Maheshwary, Vikas Yadav, Hoang Nguyen, Khyati Mahajan, Sathwik Tejaswi Madhusudhan
North American Chapter of the Association for Computational Linguistics (NAACL),  2025.
Paper Cite Code
Prompting with Phonemes: Enhancing LLM Multilinguality for non-Latin Script Languages
Prompting with Phonemes: Enhancing LLM Multilinguality for non-Latin Script Languages
Multilingual LLMs have achieved remarkable benchmark performance, but we find they continue to underperform on non-Latin script …
Hoang Nguyen, Khyati Mahajan, Vikas Yadav, Philip S. Yu, Masoud Hashemi, Rishabh Maheshwary
North American Chapter of the Association for Computational Linguistics (NAACL),  2025.
Paper Cite
Learning to Defer for Causal Discovery with Imperfect Experts
Learning to Defer for Causal Discovery with Imperfect Experts
Integrating expert knowledge, e.g. from large language models, into causal discovery algorithms can be challenging when the knowledge …
Oscar Clivio, Divyat Mahajan, Perouz Taslakian, Sara Magliacane, Ioannis Mitliagkas, Valentina Zantedeschi, Alexandre Drouin
Workshop at the International Conference of Learning Representation (ICLR),  2025.
Paper Cite
Societal Alignment Frameworks Can Improve LLM Alignment
Societal Alignment Frameworks Can Improve LLM Alignment
Recent progress in large language models (LLMs) has focused on producing responses that meet human expectations and align with shared …
Karolina Stanczak, Nicholas Meade, Mehar Bhatia, Hattie Zhou, Konstantin Böttinger, Jeremy Barns, Jason Stanley, Nicolas Papernot, Nicolas Chapados, Denis Therien, Timothy P Lillicrap, Ana Marasovic, Sylvie Delacroix, Gillian K Hadfield, Siva Reddy
Workshop at the International Conference of Learning Representation (ICLR),  2025.
Paper Cite
Unifying Autoregressive and Diffusion-Based Sequence Generation
Unifying Autoregressive and Diffusion-Based Sequence Generation
We take significant steps toward unifying autoregressive and diffusion-based sequence generation by extending the SEDD discrete …
Nima Fathi, Torsten Scholak, Pierre-André Noël
Workshop at the International Conference of Learning Representation (ICLR),  2025.
Paper Cite
Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoning
Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoning
Decoder-only Transformers often struggle with complex reasoning tasks, particularly arithmetic reasoning requiring multiple sequential …
Md Rifat Arefin, Gopeshh Subbaraj, Nicolas Gontier, Yann LeCun, Irina Rish, Ravid Shwartz-Ziv, Christopher Pal
International Conference of Learning Representations (ICLR),  2025.
Paper Cite Code Slides Video
LitLLMs, LLMs for Literature Review: Are We There Yet?
LitLLMs, LLMs for Literature Review: Are We There Yet?

Literature reviews are an essential component of scientific research, but they remain time-intensive and challenging to write, …

Gaurav Sahu, Abhay Puri, Shubham Agarwal, Issam H. Laradji, Krishnamurthy (Dj) Dvijotham, Jason Stanley, Laurent Charlin, Christopher Pal
Transactions on Machine Learning Research (TMLR),  2025.
Paper Cite Code Video