ServiceNow recherche

Torsten Scholak

Torsten Scholak

Research Lead

AI Research Deployment​

Torsten stands at the forefront of conversational AI research at ServiceNow Research. As a co-lead on the Large Language Models for Dialogue (LLMD) project alongside Dzmitry Bahdanau, Torsten has dedicated significant efforts to advancing domain-specific language models for enterprise applications. Recent Milestones:

  • Leadership in LLMD Project: Spearheading the LLMD project, Torsten has been instrumental in blending advanced language models with practical enterprise applications.
  • Summer Strategy Demo with CJ: Successfully demonstrated a goal-oriented virtual agent, highlighting the ability to develop domain-specific models that enhance customer experiences significantly.
  • Conversational Catalog with NowLLMs: Led a comprehensive rewrite of the LLMD framework for the Conversational Catalog, integrating chain-of-thought prompting with dialogue actions and ensuring seamless ITSM integration.
  • Framework Development: Played a pivotal role in evolving the LLMD framework into a comprehensive toolset that includes reliable software for enterprise-level virtual agents, a scalable process for data collection and human feedback, strategic approaches to finetuning and reinforcement learning, and innovative metrics for conversational interactions. Key Contributions:
  • Strategic Leadership: Effectively translated executive expectations into actionable research and development tasks, ensuring alignment with organizational goals.
  • Collaborative Synergy: Facilitated vital collaborations between Research and ATG, fostering a synergistic environment that bridges research and practical application.
  • Team Mentorship: Actively mentored team members, emphasizing skill development and ensuring alignment with project objectives.
  • Technical Leadership: Drove critical technical discussions, took charge of scrum master responsibilities, and played a key role in designing the foundational elements of the LLMD conversational agent framework.
  • Strategic Representation: Represented the LLMD project in strategic meetings, advocating for project progress and vision. Looking Ahead: In 2024, Torsten will lead a new Framework Team focused on harmonizing research innovation with practical application in generative AI. This initiative aims to create, expand, and refine key frameworks for conversational agents like LLMD, promoting broad organizational adoption and fostering pivotal collaborations.
Intérêts
  • Large Language Models
  • Text2Code

Publications

Unifying Autoregressive and Diffusion-Based Sequence Generation. Nima Fathi, Torsten Scholak, Pierre-André Noël. At Conference on Language Modeling (COLM),  2025.

PDF Citation

Unifying Autoregressive and Diffusion-Based Sequence Generation. Nima Fathi, Torsten Scholak, Pierre-André Noël. At Workshop at the International Conference of Learning Representation (ICLR),  2025.

PDF Citation

BigDocs: An Open and Permissively-Licensed Dataset for Training Multimodal Models on Document and Code Tasks. Juan A. Rodriguez, Xiangru Jian, Siba Smarak Panigrahi, Tianyu Zhang, Aarash Feizi, Abhay Puri, Akshay Kalkunte, Francois Savard, Ahmed Masry, Shravan Nayak, Rabiul Awal, Mahsa Massoud, Amirhossein Abaskohi, Zichao Li, Suyuchen Wang, Pierre-André Noël, Mats L. Richter, Saverio Vadacchino, Shubham Agarwal, Sanket Biswas, Sara Shanian, Ying Zhang, Sathwik Tejaswi Madhusudhan, João Monteiro, Krishnamurthy (Dj) Dvijotham, Torsten Scholak, Nicolas Chapados, Sepideh Kharaghani, Sean Hughes, Tamer Özsu, Siva Reddy, Marco Pedersoli, Yoshua Bengio, Christopher Pal, Issam H. Laradji, Spandana Gella, Perouz Taslakian, David Vazquez, Sai Rajeswar Mudumba. At International Conference of Learning Representations (ICLR),  2025.

PDF Citation Code Vidéo

BigDocs: A Permissively-Licensed Dataset for Training Vision-Language Models on Document and Code Tasks. Juan A. Rodriguez, Xiangru Jian, Siba Smarak Panigrahi, Tianyu Zhang, Aarash Feizi, Abhay Puri, Akshay Kalkunte, Francois Savard, Amirhossein Abaskohi, Ahmed Masry, Shravan Nayak, Mahsa Massoud, Rabiul Awal, Pierre-André Noël, Mats L. Richter, Saverio Vadacchino, Shubham Agarwal, Sanket Biswas, Ying Zhang, Sathwik Tejaswi Madhusudhan, João Monteiro, Krishnamurthy (Dj) Dvijotham, Torsten Scholak, Nicolas Chapados, Sean Hughes, Tamer Özsu, Aishwarya Agrawal, Marco Pedersoli, Christopher Pal, Perouz Taslakian, David Vazquez, Issam H. Laradji, Spandana Gella, Sai Rajeswar Mudumba. At Workshop at the Neural Information Processing Systems (NeurIPS),  2024.

PDF Citation Code Vidéo

StarCoder 2 and The Stack v2: The Next Generation. Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, Tianyang Liu, Max Tian, Denis Kocetkov, Arthur Zucker, Younes Belkada, Zijian Wang, Dmitry Abulkhanov, Indraneil Paul, Zhuang Li, Wen-Ding Li, Megan Risdal, Jia Li, Terry Yue Zhuo, Nii Osae Osae Dade, Lucas Krauß, Naman Jain, Yixuan Su, Xuanli He, Edoardo Abati, Yekun Chai, Xiangru Tang, Christopher Akiki, Chenghao Mou, Binyuan Hui, Nicolas Patry, Canwen Xu, Julian McAuley, Han Hu, Torsten Scholak, Sébastien Paquet, Jennifer Robinson, Carolyn Jane Anderson, Nicolas Chapados, Mostofa Patwary, Nima Tajbakhsh, Yacine Jernite, Carlos Muñoz Ferrandis, Lingming Zhang, Sean Hughes, Thomas Wolf , Arjun Guha, Leandro von Werra, Harm de Vries, Alex Gu, Armel Zebaze, Evgenii Zheltonozhskii, Jian Zhu, Manan Dey, Marc Marone, Mayank Mishra, Muhtasham Oblokulov, Olivier Dehaene, Qian Liu, Tri Dao, Wenhao Yu, Niklas Muennighoff. At ArXiv,  2024.

PDF Citation Code Vidéo

TapeAgents: a Holistic Framework for Agent Development and Optimization. Dzmitry Bahdanau, Nicolas Gontier, Gabriel Huang, Ehsan Kamalloo, Rafael Pardinas, Alexandre Piche, Torsten Scholak, Oleh Shliazhko, Jordan Prince Tremblay, Karam Ghanem, Soham Parikh, Mitul Tiwari, Quaizar Vohra. At ArXiv,  2024.

PDF Citation Code Vidéo

RepoFusion: Training Code Models to Understand Your Repository. Disha Shrivastava, Denis Kocetkov, Harm de Vries, Dzmitry Bahdanau, Torsten Scholak. At ArXiv,  2023.

PDF Citation Code

Multilingual Code Retrieval Without Paired Data: A New Benchmark and Experiments. João Monteiro, Torsten Scholak, Virendra Mehta, David Vazquez, Christopher Pal. At Workshop at the International Conference on Learning Representations (ICLR),  2023.

PDF Citation Diapositives Vidéo

UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models. Tianbao Xie, Chen Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I. Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, Noah A. Smith, Luke Zettlemoyer, Tao Yu. At Conference on Empirical Methods in Natural Language Processing (EMNLP),  2022.

PDF Citation Code

Towards Neural Functional Program Evaluation. Torsten Scholak, Jonathan Pilault, Joey Velez-Ginorio. At Conference on Neural Information Processing Systems (NeurIPS),  2021.

PDF Citation

Picard: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models. Torsten Scholak, Nathan Schucher, Dzmitry Bahdanau. At Conference on Empirical Methods in Natural Language Processing (EMNLP),  2021.

PDF Citation Code Vidéo

DuoRAT: Towards Simpler Text-to-SQL Models. Torsten Scholak, Raymond Li, Dzmitry Bahdanau, Harm de Vries, Christopher Pal. At North American Chapter of the Association for Computational Linguistics (NAACL),  2021.

PDF Citation Code