Raymond Li

AI Developer

Model Readiness

Raymond is a Research Engineer at ServiceNow AI Research. His research has focused on LLM-related topics such as summarization, text-to-SQL and code generation. He worked on model pretraining, continued-pretraining, SFT, and architectures for efficient inference. Raymond’s background is in Applied Mathematics and Computer Science. He holds a Master’s and engineering degree from École Polytechnique (France) as well as a MSc. in Computer Science from Polytechnique Montreal where he was supervised by Christopher Pal.

Intérêts

Large Language Models
Summarization
Large Code Models

Publications

Using Scaling Laws for Data Source Utility Estimation in Domain-Specific Pre-Training. Oleksiy Ostapenko, Charles Guille-Escuret, Luke Kumar, Max Tian, Denis Kocetkov, Gopeshh Subbaraj, Raymond Li, Joel Lamy Poirier, Sébastien Paquet, Torsten Scholak. At Conference on Language Modeling Workshops, 2025.

Article Citation

StarCoder 2 and The Stack v2: The Next Generation. Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, Tianyang Liu, Max Tian, Denis Kocetkov, Arthur Zucker, Younes Belkada, Zijian Wang, Dmitry Abulkhanov, Indraneil Paul, Zhuang Li, Wen-Ding Li, Megan Risdal, Jia Li, Terry Yue Zhuo, Nii Osae Osae Dade, Lucas Krauß, Naman Jain, Yixuan Su, Xuanli He, Edoardo Abati, Yekun Chai, Xiangru Tang, Christopher Akiki, Chenghao Mou, Binyuan Hui, Nicolas Patry, Canwen Xu, Julian McAuley, Han Hu, Torsten Scholak, Sébastien Paquet, Jennifer Robinson, Carolyn Jane Anderson, Nicolas Chapados, Mostofa Patwary, Nima Tajbakhsh, Yacine Jernite, Carlos Muñoz Ferrandis, Lingming Zhang, Sean Hughes, Thomas Wolf , Arjun Guha, Leandro von Werra, Harm de Vries, Alex Gu, Armel Zebaze, Evgenii Zheltonozhskii, Jian Zhu, Manan Dey, Marc Marone, Mayank Mishra, Muhtasham Oblokulov, Olivier Dehaene, Qian Liu, Tri Dao, Wenhao Yu, Niklas Muennighoff. At ArXiv, 2024.

Article Citation Code Vidéo

StarCoder: may the source be with you!. Raymond Li, Loubna Ben Allal, Yangtian Zi, Denis Kocetkov, Chenghao Mou, Christopher Akiki, Jia Li, Jenny Chim, Terry Yue Zhuo, Thomas Wang, Mishig Davaadorj, João Monteiro, Oleh Shliazhko, Nicolas Gontier, Nicholas Meade, Ming-Ho Yee, Logesh Kumar Umapathi, Benjamin Lipkin, Zhiruo Wang, Rudra Murthy, Jason Stillerman, Siva Sankalp Patel, Dmitry Abulkhanov, Marco Zocca, Zhihan Zhang, Nour Fahmy, Urvashi Bhattacharyya, Swayam Singh, Sasha Luccioni, Paulo Villegas, Maxim Kunakov, Fedor Zhdanov, Manuel Romero, Tony Lee, Nadav Timor, Jennifer Ding, Claire Schlesinger, Hailey Schoelkopf, Jan Ebert, Jennifer Robinson, Carolyn Jane Anderson, Brendan Dolan-Gavitt, Danish Contractor, Siva Reddy, Daniel Fried, Dzmitry Bahdanau, Yacine Jernite, Carlos Muñoz Ferrandis, Sean Hughes, Thomas Wolf , Arjun Guha, Leandro von Werra, Harm de Vries, Joel Lamy Poirier, Alex Gu, Armel Zebaze, Jian Zhu, Manan Dey, Marc Marone, Mayank Mishra, Muhtasham Oblokulov, Olivier Dehaene, Qian Liu, Tri Dao, Wenhao Yu, Niklas Muennighoff. At Transactions on Machine Learning Research (TMLR), 2023.

Article Citation Code

SantaCoder: don't reach for the stars!. Harm de Vries, Raymond Li, Joel Lamy Poirier, Dzmitry Bahdanau, Denis Kocetkov, Sean Hughes. At Workshop at the International Conference on Learning Representations (ICLR), 2023.

Article Citation

The Stack: 3 TB of permissively licensed source code. Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Carlos Muñoz Ferrandis, Yacine Jernite, Margaret Mitchell, Sean Hughes, Thomas Wolf , Dzmitry Bahdanau, Leandro von Werra, Harm de Vries. At Transactions on Machine Learning Research (TMLR), 2022.

Article Citation Code

DuoRAT: Towards Simpler Text-to-SQL Models. Torsten Scholak, Raymond Li, Dzmitry Bahdanau, Harm de Vries, Christopher Pal. At North American Chapter of the Association for Computational Linguistics (NAACL), 2021.

Article Citation Code

On Extractive and Abstractive Neural Document Summarization with Transformer Language Models. Sandeep Subramanian, Raymond Li, Jonathan Pilault, Christopher Pal. At Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.

Article Citation

Towards Deep Conversational Recommendations. Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, Christopher Pal. At Conference on Neural Information Processing Systems (NeurIPS), 2018.

Article Citation Code