Torsten Scholak

Publications

Breaking the Bottleneck with DiffuApriel: High-Throughput Diffusion LMs with Mamba Backbone. Vaibhav Singh, Oleksiy Ostapenko, Pierre-André Noël, Torsten Scholak. At arXiv, 2025.

ColMate: Contrastive Late Interaction and Masked Text for Multimodal Document Retrieval. Ahmed Masry, Megh Thakkar, Patrice Béchard, Sathwik Madhusudhan, Rabiul Awal, Shambhavi Mishra, Akshay Kalkunte, Enamul Hoque Prince , Spandana Gella, Torsten Scholak, Sai Rajeswar Mudumba. At Conference on Empirical Methods in Natural Language Processing (EMNLP), 2025.

Using Scaling Laws for Data Source Utility Estimation in Domain-Specific Pre-Training. Oleksiy Ostapenko, Charles Guille-Escuret, Luke Kumar, Max Tian, Denis Kocetkov, Gopeshh Subbaraj, Raymond Li, Joel Lamy Poirier, Sébastien Paquet, Torsten Scholak. At Conference on Language Modeling Workshops, 2025.

Unifying Autoregressive and Diffusion-Based Sequence Generation. Nima Fathi, Torsten Scholak, Pierre-André Noël. At Conference on Language Modeling (COLM), 2025.

Paper Cite Code Video

Unifying Autoregressive and Diffusion-Based Sequence Generation. Nima Fathi, Torsten Scholak, Pierre-André Noël. At Workshop at the International Conference of Learning Representation (ICLR), 2025.

BigDocs: An Open and Permissively-Licensed Dataset for Training Multimodal Models on Document and Code Tasks. Juan A. Rodriguez, Xiangru Jian, Siba Smarak Panigrahi, Tianyu Zhang, Aarash Feizi, Abhay Puri, Akshay Kalkunte, Francois Savard, Ahmed Masry, Shravan Nayak, Rabiul Awal, Mahsa Massoud, Amirhossein Abaskohi, Zichao Li, Suyuchen Wang, Pierre-André Noël, Mats L. Richter, Saverio Vadacchino, Shubham Agarwal, Sanket Biswas, Sara Shanian, Ying Zhang, Sathwik Tejaswi Madhusudhan, João Monteiro, Krishnamurthy (Dj) Dvijotham, Torsten Scholak, Nicolas Chapados, Sepideh Kharaghani, Sean Hughes, Tamer Özsu, Siva Reddy, Marco Pedersoli, Yoshua Bengio, Christopher Pal, Issam H. Laradji, Spandana Gella, Perouz Taslakian, David Vazquez, Sai Rajeswar Mudumba. At International Conference of Learning Representations (ICLR), 2025.

Paper Cite Code Video

BigDocs: A Permissively-Licensed Dataset for Training Vision-Language Models on Document and Code Tasks. Juan A. Rodriguez, Xiangru Jian, Siba Smarak Panigrahi, Tianyu Zhang, Aarash Feizi, Abhay Puri, Akshay Kalkunte, Francois Savard, Amirhossein Abaskohi, Ahmed Masry, Shravan Nayak, Mahsa Massoud, Rabiul Awal, Pierre-André Noël, Mats L. Richter, Saverio Vadacchino, Shubham Agarwal, Sanket Biswas, Ying Zhang, Sathwik Tejaswi Madhusudhan, João Monteiro, Krishnamurthy (Dj) Dvijotham, Torsten Scholak, Nicolas Chapados, Sean Hughes, Tamer Özsu, Aishwarya Agrawal, Marco Pedersoli, Christopher Pal, Perouz Taslakian, David Vazquez, Issam H. Laradji, Spandana Gella, Sai Rajeswar Mudumba. At Workshop at the Neural Information Processing Systems (NeurIPS), 2024.

Paper Cite Code Video

StarCoder 2 and The Stack v2: The Next Generation. Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, Tianyang Liu, Max Tian, Denis Kocetkov, Arthur Zucker, Younes Belkada, Zijian Wang, Dmitry Abulkhanov, Indraneil Paul, Zhuang Li, Wen-Ding Li, Megan Risdal, Jia Li, Terry Yue Zhuo, Nii Osae Osae Dade, Lucas Krauß, Naman Jain, Yixuan Su, Xuanli He, Edoardo Abati, Yekun Chai, Xiangru Tang, Christopher Akiki, Chenghao Mou, Binyuan Hui, Nicolas Patry, Canwen Xu, Julian McAuley, Han Hu, Torsten Scholak, Sébastien Paquet, Jennifer Robinson, Carolyn Jane Anderson, Nicolas Chapados, Mostofa Patwary, Nima Tajbakhsh, Yacine Jernite, Carlos Muñoz Ferrandis, Lingming Zhang, Sean Hughes, Thomas Wolf , Arjun Guha, Leandro von Werra, Harm de Vries, Alex Gu, Armel Zebaze, Evgenii Zheltonozhskii, Jian Zhu, Manan Dey, Marc Marone, Mayank Mishra, Muhtasham Oblokulov, Olivier Dehaene, Qian Liu, Tri Dao, Wenhao Yu, Niklas Muennighoff. At ArXiv, 2024.

Paper Cite Code Video

TapeAgents: a Holistic Framework for Agent Development and Optimization. Dzmitry Bahdanau, Nicolas Gontier, Gabriel Huang, Ehsan Kamalloo, Rafael Pardinas, Alexandre Piche, Torsten Scholak, Oleh Shliazhko, Jordan Prince Tremblay, Karam Ghanem, Soham Parikh, Mitul Tiwari, Quaizar Vohra. At ArXiv, 2024.

Paper Cite Code Video

RepoFusion: Training Code Models to Understand Your Repository. Disha Shrivastava, Denis Kocetkov, Harm de Vries, Dzmitry Bahdanau, Torsten Scholak. At ArXiv, 2023.

Paper Cite Code

Multilingual Code Retrieval Without Paired Data: A New Benchmark and Experiments. João Monteiro, Torsten Scholak, Virendra Mehta, David Vazquez, Christopher Pal. At Workshop at the International Conference on Learning Representations (ICLR), 2023.

Paper Cite Slides Video

UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models. Tianbao Xie, Chen Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I. Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, Noah A. Smith, Luke Zettlemoyer, Tao Yu. At Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022.

Paper Cite Code

Towards Neural Functional Program Evaluation. Torsten Scholak, Jonathan Pilault, Joey Velez-Ginorio. At Conference on Neural Information Processing Systems (NeurIPS), 2021.

Picard: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models. Torsten Scholak, Nathan Schucher, Dzmitry Bahdanau. At Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021.

Paper Cite Code Video

DuoRAT: Towards Simpler Text-to-SQL Models. Torsten Scholak, Raymond Li, Dzmitry Bahdanau, Harm de Vries, Christopher Pal. At North American Chapter of the Association for Computational Linguistics (NAACL), 2021.

Paper Cite Code