Joel Lamy Poirier

Applied Research Scientist

Large Language Models Lab

Joel is an Applied Research Scientist in the Emerging Technologies Lab at ServiceNow Research, with experience in various fields of artificial intelligence including computer vision and natural language processing. He focuses on efficient implementations of deep neural networks, particularly on large scale distributed training and large language models. He recently proposed a new form of 3d parallelism, which improves on the state of the art in scalability and computational efficiency.

Prior to joining ServiceNow, he was a graduate researcher at the Perimeter Institute for Theoretical Physics, where he worked on exact computations in supersymmetric quantum field theory and string theory. He has a Ph.D. in physics from the University of Waterloo (2016).

Interests

Large Language Models

Publications

Breadth-First Pipeline Parallelism. Joel Lamy Poirier. At Conference on Machine Learning and Systems, 2023.

PDF Cite Code

StarCoder: may the source be with you!. Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozhskii, Terry Yue Zhuo, Thomas Wang, Olivier Dehaene, Mishig Davaadorj, Joel Lamy Poirier, João Monteiro, Oleh Shliazhko, Nicolas Gontier, Nicholas Meade, Armel Zebaze, Ming-Ho Yee, Logesh Kumar Umapathi, Jian Zhu, Benjamin Lipkin, Muhtasham Oblokulov, Zhiruo Wang, Rudra Murthy, Jason Stillerman, Siva Sankalp Patel, Dmitry Abulkhanov, Marco Zocca, Manan Dey, Zhihan Zhang, Nour Fahmy, Urvashi Bhattacharyya, Wenhao Yu, Swayam Singh, Sasha Luccioni, Paulo Villegas, Maxim Kunakov, Fedor Zhdanov, Manuel Romero, Tony Lee, Nadav Timor, Jennifer Ding, Claire Schlesinger, Hailey Schoelkopf, Jan Ebert, Tri Dao, Mayank Mishra, Alex Gu, Jennifer Robinson, Carolyn Jane Anderson, Brendan Dolan-Gavitt, Danish Contractor, Siva Reddy, Daniel Fried, Dzmitry Bahdanau, Yacine Jernite, Carlos Muñoz Ferrandis, Sean Hughes, Thomas Wolf , Arjun Guha, Leandro von Werra, Harm de Vries. At ArXiv, 2023.

PDF Cite

SantaCoder: don't reach for the stars!. Harm de Vries, Raymond Li, Joel Lamy Poirier, Dzmitry Bahdanau, Denis Kocetkov, Sean Hughes. At Workshop at the International Conference on Learning Representations (ICLR), 2023.

PDF Cite

Breadth-First Pipeline Parallelism. Joel Lamy Poirier. At Workshop at the Neural Information Processing Systems (NeurIPS), 2022.

PDF Cite Code

Layered gradient accumulation and modular pipeline parallelism: fast and efficient training of large language models. Joel Lamy Poirier. At ArXiv, 2022.

PDF Cite

Hinted Networks. Joel Lamy Poirier, Anqi Xu. At ArXiv, 2018.

PDF Cite