ServiceNow AI Research

Efficient Inference

Auto-Cypher: Improving LLMs on Cypher generation via LLM-supervised generation-verification framework
Auto-Cypher: Improving LLMs on Cypher generation via LLM-supervised generation-verification framework
Graph databases like Neo4j are gaining popularity for handling complex, interconnected data, over traditional relational databases in …
North American Chapter of the Association for Computational Linguistics (NAACL),  2025.
PDF Cite
Unifying Autoregressive and Diffusion-Based Sequence Generation
Unifying Autoregressive and Diffusion-Based Sequence Generation
We take significant steps toward unifying autoregressive and diffusion-based sequence generation by extending the SEDD discrete …
Workshop at the International Conference of Learning Representation (ICLR),  2025.
PDF Cite
XC-Cache: Cross-Attending to Cached Context for Efficient LLM Inference
XC-Cache: Cross-Attending to Cached Context for Efficient LLM Inference
In-context learning (ICL) approaches typically leverage prompting to condition decoder-only language model generation on reference …
Workshop at the Neural Information Processing Systems (NeurIPS),  2024.
PDF Cite Code
Layer-Wise Quantization: A Pragmatic and Effective Method for Quantizing LLMs Beyond Integer Bit-Levels
Layer-Wise Quantization: A Pragmatic and Effective Method for Quantizing LLMs Beyond Integer Bit-Levels
We present a simple meta quantization approach that quantizes different layers of a large language model (LLM) at different bit levels, …
ArXiv,  2024.
PDF Cite Code