Code Generation

GitChameleon: Evaluating AI Code Generation Against Python Library Version Incompatibilities

The rapid evolution of software libraries presents a significant challenge for code generation models, which must adapt to frequent …

Nizar Islah, Justine Gehring, Diganta Misra, Eilif Muller, Irina Rish, Eilif Benjamin Muller, Massimo Caccia

Annual Meeting of the Association for Computational Linguistics (ACL), 2026.

StarFlow: Generating Structured Workflow Outputs From Sketch Images

Workflows are a fundamental component of automation in enterprise platforms, enabling the orchestration of tasks, data processing, and …

Patrice Béchard, Chao Wang, Juan A. Rodriguez, Amirhossein Abaskohi, Christopher Pal, David Vazquez, Spandana Gella, Sai Rajeswar Mudumba, Perouz Taslakian

European Chapter of the Association for Computational Linguistics (EACL), 2026.

GitChameleon: Evaluating AI Code Generation Against Python Library Version Incompatibilities

The rapid evolution of software libraries presents a significant challenge for code generation models, which must adapt to frequent …

Nizar Islah, Justine Gehring, Diganta Misra, Eilif Muller, Irina Rish, Eilif Benjamin Muller, Massimo Caccia

Workshop at the Neural Information Processing Systems (NeurIPS), 2025.

WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation

Understanding diverse web data and automating web development presents an exciting challenge for agentic AI. While existing benchmarks …

Rabiul Awal, Mahsa Massoud, Zichao Li, Aarash Feizi, Suyuchen Wang, Christopher Pal, Aishwarya Agrawal, David Vazquez, Siva Reddy, Juan A. Rodriguez, Perouz Taslakian, Sai Rajeswar Mudumba

Workshop at the Computer Vision and Pattern Recognition Conference (CVPR), 2025.

WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation

Understanding diverse web data and automating web development presents an exciting challenge for agentic AI. While existing benchmarks …

Rabiul Awal, Mahsa Massoud, Zichao Li, Aarash Feizi, Suyuchen Wang, Christopher Pal, Aishwarya Agrawal, David Vazquez, Siva Reddy, Juan A. Rodriguez, Perouz Taslakian, Spandana Gella, Sai Rajeswar Mudumba

Workshop at the International Conference of Learning Representation (ICLR), 2025.

BigDocs: An Open and Permissively-Licensed Dataset for Training Multimodal Models on Document and Code Tasks

Multimodal AI has the potential to significantly enhance document-understanding tasks, such as processing receipts, understanding …

Juan A. Rodriguez, Xiangru Jian, Siba Smarak Panigrahi, Tianyu Zhang, Aarash Feizi, Abhay Puri, Akshay Kalkunte, Francois Savard, Ahmed Masry, Shravan Nayak, Rabiul Awal, Mahsa Massoud, Amirhossein Abaskohi, Zichao Li, Suyuchen Wang, Pierre-André Noël, Mats L. Richter, Saverio Vadacchino, Shubham Agarwal, Sanket Biswas, Sara Shanian, Ying Zhang, Sathwik Tejaswi Madhusudhan, João Monteiro, Krishnamurthy (Dj) Dvijotham, Torsten Scholak, Nicolas Chapados, Sepideh Kharaghani, Sean Hughes, Tamer Özsu, Siva Reddy, Marco Pedersoli, Yoshua Bengio, Christopher Pal, Issam H. Laradji, Spandana Gella, Perouz Taslakian, David Vazquez, Sai Rajeswar Mudumba

International Conference of Learning Representations (ICLR), 2025.

Evaluating In-Context Learning of Libraries for Code Generation

Contemporary Large Language Models (LLMs) exhibit a high degree of code generation and comprehension capability. A particularly …

Arkil Patel, Siva Reddy, Dzmitry Bahdanau, Pradeep Dasigi

North American Chapter of the Association for Computational Linguistics (NAACL), 2024.

InCoRo: In-Context Learning for Robotics Control with Feedback Loops

One of the challenges in robotics is to enable robotic units with the reasoning capability that would be robust enough to execute …

Jiaquiang Ye Zhu, Carla Gomez, David Vazquez, Michal Drozdzal

ArXiv, 2024.

StarCoder 2 and The Stack v2: The Next Generation

The BigCode project, an open-scientific collaboration focused on the responsible development of Large Language Models for Code (Code …

Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, Tianyang Liu, Max Tian, Denis Kocetkov, Arthur Zucker, Younes Belkada, Zijian Wang, Dmitry Abulkhanov, Indraneil Paul, Zhuang Li, Wen-Ding Li, Megan Risdal, Jia Li, Terry Yue Zhuo, Nii Osae Osae Dade, Lucas Krauß, Naman Jain, Yixuan Su, Xuanli He, Edoardo Abati, Yekun Chai, Xiangru Tang, Christopher Akiki, Chenghao Mou, Binyuan Hui, Nicolas Patry, Canwen Xu, Julian McAuley, Han Hu, Torsten Scholak, Sébastien Paquet, Jennifer Robinson, Carolyn Jane Anderson, Nicolas Chapados, Mostofa Patwary, Nima Tajbakhsh, Yacine Jernite, Carlos Muñoz Ferrandis, Lingming Zhang, Sean Hughes, Thomas Wolf , Arjun Guha, Leandro von Werra, Harm de Vries, Alex Gu, Armel Zebaze, Evgenii Zheltonozhskii, Jian Zhu, Manan Dey, Marc Marone, Mayank Mishra, Muhtasham Oblokulov, Olivier Dehaene, Qian Liu, Tri Dao, Wenhao Yu, Niklas Muennighoff

ArXiv, 2024.

StarVector: Generating Scalable Vector Graphics Code from Images and Text

Scalable Vector Graphics (SVGs) have become integral in modern image rendering and graphic design applications due to their infinite …

Juan A. Rodriguez, Shubham Agarwal, Abhay Puri, Issam H. Laradji, Sai Rajeswar Mudumba, Pau Rodriguez, David Vazquez, Christopher Pal, Marco Pedersoli

ArXiv, 2024.