OSM: An Open Set Matting Framework with OOD Detection and Few-Shot Learning

Abstract

Natural image matting is the task of precisely estimating alpha mattes to separate foreground objects from background images. Existing matting methods only focus on classical closed-set problems where object categories and data distributions are similar between training and test sets. However, in the open world setup, there exists a situation where testing samples are drawn from a different distribution than the training data. To handle this situation, we present the first open set matting (OSM) framework that contains two networks: (1) an out-of-distribution (OOD) detection network to identify OOD to-be-matted objects; and (2) an incremental few-shot learning matting module to enlarge the existing knowledge base of to-be-matted objects. Our OOD detection network leverages metric-based prototype learning to be aware of unseen objects and increase inter-class separability, utilizing intra-batch connections to enhance intra-class compactness. Compared to other OOD detection methods, our network achieves state-of-the-art performance on SIMD dataset. Further, our incremental few-shot learning matting module improves the performance on unseen to-be-matted objects by gradually incorporating novel classes into the existing knowledge base without catastrophic forgetting and over-fitting.