Oleksiy Ostapenko
Research Scientist
Frontier AI Research
Research Scientist in the Foundation Models Lab. I completed my Ph.D. at MILA, supervised by professor Laurent Charlin, and was a Visiting Researcher at ServiceNow Research. During my Master’s degree, which I obtained at the Humboldt University of Berlin, I interned at SAP AI Research lab, where I also spent some time working as an associate researcher. My main research interest lies in developing systems that can accumulate and transfer knowledge throughout their lifetime.
Intérêts
- Continual Learning
Publications
Breaking the Bottleneck with DiffuApriel: High-Throughput Diffusion LMs with Mamba Backbone.
arXiv,
2025.
Using Scaling Laws for Data Source Utility Estimation in Domain-Specific Pre-Training.
Conference on Language Modeling Workshops,
2025.
Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts.
Conference on Language Modeling (COLM),
2025.
Attention for Compositional Modularity.
Workshop at the Neural Information Processing Systems (NeurIPS),
2022.
Continual Learning with self-selecting specialized modules through expansion and pruning.
Montreal AI Symposium (MAIS),
2022.
Continual Learning with Foundation Models: An Empirical Study of Latent Replay.
Conference on Lifelong Learning Agents (CoLLAs),
2022.
Continual Learning with Foundation Models: An Empirical Study of Latent Replay.
Workshop at the Conference on Computer Vision and Pattern Recognition (CVPR),
2022.
Continual Learning via Local Module Composition.
Conference on Neural Information Processing Systems (NeurIPS),
2021.
Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning.
Conference on Neural Information Processing Systems (NeurIPS),
2020.
Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning.
Workshop at the Conference on Computer Vision and Pattern Recognition (CVPR),
2020.