ServiceNow AI Research

Theory of Machine Learning

Surrogate Minimization: An Optimization Algorithm for Training Large Neural Networks with Model Parallelism
Surrogate Minimization: An Optimization Algorithm for Training Large Neural Networks with Model Parallelism
Optimizing large memory-intensive neural networks requires distributing its layers across multiple GPUs (referred to as model …
Workshop at the Neural Information Processing Systems (NeurIPS),  2023.
PDF Cite
On Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants
On Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants
We investigate the convergence of stochastic mirror descent (SMD) under interpolation in relatively smooth and smooth convex …
Transactions on Machine Learning Research (TMLR),  2023.
PDF Cite
Let's Make Block Coordinate Descent Converge Faster: Faster Greedy Rules, Message-Passing, Active-Set Complexity, and Superlinear Convergence
Let's Make Block Coordinate Descent Converge Faster: Faster Greedy Rules, Message-Passing, Active-Set Complexity, and Superlinear Convergence
Block coordinate descent (BCD) methods are widely used for large-scale numerical optimization because of their cheap iteration costs, …
International Conference on Machine Learning (ICML),  2023.
PDF Cite
DAG Learning on the Permutahedron
DAG Learning on the Permutahedron

We propose a continuous optimization framework for discovering a latent directed acyclic graph (DAG) from observational data. Our …

International Conference of Learning Representations (ICLR),  2023.
PDF Cite Slides
Deep Hyperbolic Reinforcement Learning for Continuous Control
Deep Hyperbolic Reinforcement Learning for Continuous Control
Integrating hyperbolic representations with Deep Reinforcement Learning (DRL) has recently been proposed as a promising approach for …
ICLR, Tiny Papers,  2023.
PDF Cite