Joel Lamy Poirier

Applied Research Scientist

Model Readiness

Joel is an Applied Research Scientist in the Model Readiness squad at ServiceNow AI Research, with experience in various fields of artificial intelligence including computer vision and natural language processing. He focuses on efficient implementations of deep neural networks, particularly on large scale distributed training and large language models. He recently proposed a new form of 3d parallelism, which improves on the state of the art in scalability and computational efficiency.

Prior to joining ServiceNow, he was a graduate researcher at the Perimeter Institute for Theoretical Physics, where he worked on exact computations in supersymmetric quantum field theory and string theory. He has a Ph.D. in physics from the University of Waterloo (2016).

Interests

Large Language Models

Publications

Using Scaling Laws for Data Source Utility Estimation in Domain-Specific Pre-Training. Oleksiy Ostapenko, Charles Guille-Escuret, Luke Kumar, Max Tian, Denis Kocetkov, Gopeshh Subbaraj, Raymond Li, Joel Lamy Poirier, Sébastien Paquet, Torsten Scholak. At Conference on Language Modeling Workshops, 2025.

Paper Cite

Layered gradient accumulation and modular pipeline parallelism: fast and efficient training of large language models. Joel Lamy Poirier. At ArXiv, 2024.

Paper Cite

StarCoder 2 and The Stack v2: The Next Generation. Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, Tianyang Liu, Max Tian, Denis Kocetkov, Arthur Zucker, Younes Belkada, Zijian Wang, Dmitry Abulkhanov, Indraneil Paul, Zhuang Li, Wen-Ding Li, Megan Risdal, Jia Li, Terry Yue Zhuo, Nii Osae Osae Dade, Lucas Krauß, Naman Jain, Yixuan Su, Xuanli He, Edoardo Abati, Yekun Chai, Xiangru Tang, Christopher Akiki, Chenghao Mou, Binyuan Hui, Nicolas Patry, Canwen Xu, Julian McAuley, Han Hu, Torsten Scholak, Sébastien Paquet, Jennifer Robinson, Carolyn Jane Anderson, Nicolas Chapados, Mostofa Patwary, Nima Tajbakhsh, Yacine Jernite, Carlos Muñoz Ferrandis, Lingming Zhang, Sean Hughes, Thomas Wolf , Arjun Guha, Leandro von Werra, Harm de Vries, Alex Gu, Armel Zebaze, Evgenii Zheltonozhskii, Jian Zhu, Manan Dey, Marc Marone, Mayank Mishra, Muhtasham Oblokulov, Olivier Dehaene, Qian Liu, Tri Dao, Wenhao Yu, Niklas Muennighoff. At ArXiv, 2024.

Paper Cite Code Video

Breadth-First Pipeline Parallelism. Joel Lamy Poirier. At Conference on Machine Learning and Systems (MLSYS), 2023.

Paper Cite

StarCoder: may the source be with you!. Raymond Li, Loubna Ben Allal, Yangtian Zi, Denis Kocetkov, Chenghao Mou, Christopher Akiki, Jia Li, Jenny Chim, Terry Yue Zhuo, Thomas Wang, Mishig Davaadorj, João Monteiro, Oleh Shliazhko, Nicolas Gontier, Nicholas Meade, Ming-Ho Yee, Logesh Kumar Umapathi, Benjamin Lipkin, Zhiruo Wang, Rudra Murthy, Jason Stillerman, Siva Sankalp Patel, Dmitry Abulkhanov, Marco Zocca, Zhihan Zhang, Nour Fahmy, Urvashi Bhattacharyya, Swayam Singh, Sasha Luccioni, Paulo Villegas, Maxim Kunakov, Fedor Zhdanov, Manuel Romero, Tony Lee, Nadav Timor, Jennifer Ding, Claire Schlesinger, Hailey Schoelkopf, Jan Ebert, Jennifer Robinson, Carolyn Jane Anderson, Brendan Dolan-Gavitt, Danish Contractor, Siva Reddy, Daniel Fried, Dzmitry Bahdanau, Yacine Jernite, Carlos Muñoz Ferrandis, Sean Hughes, Thomas Wolf , Arjun Guha, Leandro von Werra, Harm de Vries, Joel Lamy Poirier, Alex Gu, Armel Zebaze, Jian Zhu, Manan Dey, Marc Marone, Mayank Mishra, Muhtasham Oblokulov, Olivier Dehaene, Qian Liu, Tri Dao, Wenhao Yu, Niklas Muennighoff. At Transactions on Machine Learning Research (TMLR), 2023.

Paper Cite Code

SantaCoder: don't reach for the stars!. Harm de Vries, Raymond Li, Joel Lamy Poirier, Dzmitry Bahdanau, Denis Kocetkov, Sean Hughes. At Workshop at the International Conference on Learning Representations (ICLR), 2023.

Paper Cite

Breadth-First Pipeline Parallelism. Joel Lamy Poirier. At Workshop at the Neural Information Processing Systems (NeurIPS), 2022.

Paper Cite

Hinted Networks. Joel Lamy Poirier, Anqi Xu. At ArXiv, 2018.

Paper Cite