João Monteiro

Publications

BiXSE: Improving Dense Retrieval via Probabilistic Graded Relevance Distillation. Christos Tsirigotis, Vaibhav Adlakha, João Monteiro, Aaron Courville, Perouz Taslakian. At Conference on Language Modeling (COLM), 2025.

BigDocs: An Open and Permissively-Licensed Dataset for Training Multimodal Models on Document and Code Tasks. Juan A. Rodriguez, Xiangru Jian, Siba Smarak Panigrahi, Tianyu Zhang, Aarash Feizi, Abhay Puri, Akshay Kalkunte, Francois Savard, Ahmed Masry, Shravan Nayak, Rabiul Awal, Mahsa Massoud, Amirhossein Abaskohi, Zichao Li, Suyuchen Wang, Pierre-André Noël, Mats L. Richter, Saverio Vadacchino, Shubham Agarwal, Sanket Biswas, Sara Shanian, Ying Zhang, Sathwik Tejaswi Madhusudhan, João Monteiro, Krishnamurthy (Dj) Dvijotham, Torsten Scholak, Nicolas Chapados, Sepideh Kharaghani, Sean Hughes, Tamer Özsu, Siva Reddy, Marco Pedersoli, Yoshua Bengio, Christopher Pal, Issam H. Laradji, Spandana Gella, Perouz Taslakian, David Vazquez, Sai Rajeswar Mudumba. At International Conference of Learning Representations (ICLR), 2025.

BigDocs: A Permissively-Licensed Dataset for Training Vision-Language Models on Document and Code Tasks. Juan A. Rodriguez, Xiangru Jian, Siba Smarak Panigrahi, Tianyu Zhang, Aarash Feizi, Abhay Puri, Akshay Kalkunte, Francois Savard, Amirhossein Abaskohi, Ahmed Masry, Shravan Nayak, Mahsa Massoud, Rabiul Awal, Pierre-André Noël, Mats L. Richter, Saverio Vadacchino, Shubham Agarwal, Sanket Biswas, Ying Zhang, Sathwik Tejaswi Madhusudhan, João Monteiro, Krishnamurthy (Dj) Dvijotham, Torsten Scholak, Nicolas Chapados, Sean Hughes, Tamer Özsu, Aishwarya Agrawal, Marco Pedersoli, Christopher Pal, Perouz Taslakian, David Vazquez, Issam H. Laradji, Spandana Gella, Sai Rajeswar Mudumba. At Workshop at the Neural Information Processing Systems (NeurIPS), 2024.

Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection. Charles Guille-Escuret, Pierre-André Noël, Ioannis Mitliagkas, David Vazquez, João Monteiro. At NeurIPS Datasets and Benchmarks Track (NeurIPS Datasets), 2024.

RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Content. João Monteiro, Pierre-André Noël, Étienne Marcotte, Sai Rajeswar Mudumba, Valentina Zantedeschi, David Vazquez, Nicolas Chapados, Christopher Pal, Perouz Taslakian. At NeurIPS Datasets and Benchmarks Track (NeurIPS Datasets), 2024.

XC-Cache: Cross-Attending to Cached Context for Efficient LLM Inference. João Monteiro, Étienne Marcotte, Pierre-André Noël, Valentina Zantedeschi, David Vazquez, Nicolas Chapados, Christopher Pal, Perouz Taslakian. At Workshop at the Neural Information Processing Systems (NeurIPS), 2024.

XC-Cache: Cross-Attending to Cached Context for Efficient LLM Inference. João Monteiro, Étienne Marcotte, Pierre-André Noël, Valentina Zantedeschi, David Vazquez, Nicolas Chapados, Christopher Pal, Perouz Taslakian. At Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024.

Performance Control in Early Exiting to Deploy Large Models at the Same Cost of Smaller Ones. Mehrnaz Mofakhami, Reza Bayat, Ioannis Mitliagkas, João Monteiro, Valentina Zantedeschi. At Workshop at the International Conference of Machine Learning (ICML), 2024.

CADet: Fully Self-Supervised Out-Of-Distribution Detection With Contrastive Learning. Charles Guille-Escuret, Pau Rodriguez, David Vazquez, Ioannis Mitliagkas, João Monteiro. At Conference on Neural Information Processing Systems (NeurIPS), 2023.

Group Robust Classification Without Any Group Information. Christos Tsirigotis, João Monteiro, Pau Rodriguez, Aaron Courville. At Conference on Neural Information Processing Systems (NeurIPS), 2023.

StarCoder: may the source be with you!. Raymond Li, Loubna Ben Allal, Yangtian Zi, Denis Kocetkov, Chenghao Mou, Christopher Akiki, Jia Li, Jenny Chim, Terry Yue Zhuo, Thomas Wang, Mishig Davaadorj, João Monteiro, Oleh Shliazhko, Nicolas Gontier, Nicholas Meade, Ming-Ho Yee, Logesh Kumar Umapathi, Benjamin Lipkin, Zhiruo Wang, Rudra Murthy, Jason Stillerman, Siva Sankalp Patel, Dmitry Abulkhanov, Marco Zocca, Zhihan Zhang, Nour Fahmy, Urvashi Bhattacharyya, Swayam Singh, Sasha Luccioni, Paulo Villegas, Maxim Kunakov, Fedor Zhdanov, Manuel Romero, Tony Lee, Nadav Timor, Jennifer Ding, Claire Schlesinger, Hailey Schoelkopf, Jan Ebert, Jennifer Robinson, Carolyn Jane Anderson, Brendan Dolan-Gavitt, Danish Contractor, Siva Reddy, Daniel Fried, Dzmitry Bahdanau, Yacine Jernite, Carlos Muñoz Ferrandis, Sean Hughes, Thomas Wolf , Arjun Guha, Leandro von Werra, Harm de Vries, Joel Lamy Poirier, Alex Gu, Armel Zebaze, Jian Zhu, Manan Dey, Marc Marone, Mayank Mishra, Muhtasham Oblokulov, Olivier Dehaene, Qian Liu, Tri Dao, Wenhao Yu, Niklas Muennighoff. At Transactions on Machine Learning Research (TMLR), 2023.

Constraining Representations Yields Models That Know What They Don't Know. João Monteiro, Pau Rodriguez, Pierre-André Noël, Issam H. Laradji, David Vazquez. At International Conference of Learning Representations (ICLR), 2023.

PDF Cite Model Card Slides Video

Multilingual Code Retrieval Without Paired Data: A New Benchmark and Experiments. João Monteiro, Torsten Scholak, Virendra Mehta, David Vazquez, Christopher Pal. At Workshop at the International Conference on Learning Representations (ICLR), 2023.

PDF Cite Slides Video

Constraining Low-level Representations to Define Effective Confidence Scores. João Monteiro, Pau Rodriguez, Pierre-André Noël, Issam H. Laradji, David Vazquez. At Workshop at the Neural Information Processing Systems (NeurIPS), 2022.

Contrastive Self-supervision Defines General-Purpose Similarity Functions. Charles Guille-Escuret, Pau Rodriguez, David Vazquez, Ioannis Mitliagkas, João Monteiro. At Workshop at the Neural Information Processing Systems (NeurIPS), 2022.

Leveraging Activation Patterns to Define Classifiers Able to Detect and Reject Anomalies. João Monteiro, Pau Rodriguez, Pierre-André Noël, Issam H. Laradji, David Vazquez. At Montreal AI Symposium (MAIS), 2022.

Monotonicity Regularization: Improved Penalties and Novel Applications to Disentangled Representation Learning and Robust Classification. João Monteiro, Mohamed Osama Ahmed, Hossein Hajimirsadeghi, Greg Mori. At Conference on Uncertainty in Artificial Intelligence (UAI), 2022.