Torsten Scholak
Research Lead
AI Research Deployment
Torsten stands at the forefront of conversational AI research at ServiceNow Research. As a co-lead on the Large Language Models for Dialogue (LLMD) project alongside Dzmitry Bahdanau, Torsten has dedicated significant efforts to advancing domain-specific language models for enterprise applications. Recent Milestones:
- Leadership in LLMD Project: Spearheading the LLMD project, Torsten has been instrumental in blending advanced language models with practical enterprise applications.
- Summer Strategy Demo with CJ: Successfully demonstrated a goal-oriented virtual agent, highlighting the ability to develop domain-specific models that enhance customer experiences significantly.
- Conversational Catalog with NowLLMs: Led a comprehensive rewrite of the LLMD framework for the Conversational Catalog, integrating chain-of-thought prompting with dialogue actions and ensuring seamless ITSM integration.
- Framework Development: Played a pivotal role in evolving the LLMD framework into a comprehensive toolset that includes reliable software for enterprise-level virtual agents, a scalable process for data collection and human feedback, strategic approaches to finetuning and reinforcement learning, and innovative metrics for conversational interactions. Key Contributions:
- Strategic Leadership: Effectively translated executive expectations into actionable research and development tasks, ensuring alignment with organizational goals.
- Collaborative Synergy: Facilitated vital collaborations between Research and ATG, fostering a synergistic environment that bridges research and practical application.
- Team Mentorship: Actively mentored team members, emphasizing skill development and ensuring alignment with project objectives.
- Technical Leadership: Drove critical technical discussions, took charge of scrum master responsibilities, and played a key role in designing the foundational elements of the LLMD conversational agent framework.
- Strategic Representation: Represented the LLMD project in strategic meetings, advocating for project progress and vision. Looking Ahead: In 2024, Torsten will lead a new Framework Team focused on harmonizing research innovation with practical application in generative AI. This initiative aims to create, expand, and refine key frameworks for conversational agents like LLMD, promoting broad organizational adoption and fostering pivotal collaborations.
Intérêts
- Large Language Models
- Text2Code
Publications
Unifying Autoregressive and Diffusion-Based Sequence Generation.
Conference on Language Modeling (COLM),
2025.
Unifying Autoregressive and Diffusion-Based Sequence Generation.
Workshop at the International Conference of Learning Representation (ICLR),
2025.
BigDocs: An Open and Permissively-Licensed Dataset for Training Multimodal Models on Document and Code Tasks.
International Conference of Learning Representations (ICLR),
2025.
BigDocs: A Permissively-Licensed Dataset for Training Vision-Language Models on Document and Code Tasks.
Workshop at the Neural Information Processing Systems (NeurIPS),
2024.
StarCoder 2 and The Stack v2: The Next Generation.
ArXiv,
2024.
Multilingual Code Retrieval Without Paired Data: A New Benchmark and Experiments.
Workshop at the International Conference on Learning Representations (ICLR),
2023.
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models.
Conference on Empirical Methods in Natural Language Processing (EMNLP),
2022.
Towards Neural Functional Program Evaluation.
Conference on Neural Information Processing Systems (NeurIPS),
2021.
Picard: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models.
Conference on Empirical Methods in Natural Language Processing (EMNLP),
2021.
DuoRAT: Towards Simpler Text-to-SQL Models.
North American Chapter of the Association for Computational Linguistics (NAACL),
2021.