Analysis of Methods for Few-shot Semantic Segmentation

Rus, Marko (author), Kristan, Matej (mentor)

Few-shot semantic segmentation, which aims at learning new categories from only a few training examples, has progressed substantially in the last decade. The progress was in part driven by datasets derived from the existing datasets for semantic segmentation. However, these datasets have several drawbacks in the context of the few-shot performance evaluation. PASCAL-5 has a low number of classes and objects well separated from the background, COCO-20 has too diverse classes, and FSS-1000 contains objects that are trivial to segment so that our Zero-Shot Segmentation Baseline (ZSSB) model achieves a high mean mIoU of 81.1%. Therefore we construct a new dataset LVIS-1025 from the general semantic segmentation dataset LVIS by applying new criteria for measuring object predictability and expressiveness. We evaluate three state-of-the-art methods (PANet, PPNet, and ASGNet) on this dataset and show that the ranks change compared to those obtained on existing public datasets. ASGNet on the standard datasets outperforms PANet and PPNet by a large margin, but on LVIS-1025 performs worse, indicating that ASGNet is prone to segmenting the most salient object in the image. We believe that future models developed on LVIS-1025 will have better generalization capabilities and will not that heavily rely on the always-present assumption.

• Language: English
• Keywords: computer vision, deep learning, convolutional neural networks, semantic segmentation, few-shot learning, evaluation protocol
• Work type: Master's thesis/paper
• Typology: 2.09 - Master's Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2021
• PID: 20.500.12556/RUL-129658
• COBISS.SI-ID: 75943171
• Publication date in RUL: 06.09.2021

Secondary language

• Language: Slovenian
• Title: Analiza metod za semantično segmentacijo z malo učnimi primeri

Abstract

Semantična segmentacija z malo učnimi primeri, katere cilj je naučiti se novih kategorij z le nekaj učnimi primeri, je v zadnjem desetletju močno napredovala. Napredek so deloma začrtale podatkovne množice, ki izhajajo iz obstoječih podatkovnih množic za semantično segmentacijo. Te podatkovne množice imajo v okviru evalvacije z malo učnimi primeri več pomanjkljivosti, PASCAL-5 ima majhno število razredov in nekatere predmete dobro ločene od ozadja, COCO-20 ima preveč raznolike razrede, FSS-1000 pa vsebuje predmete, ki jih je trivialno segmentirati, tako da naš model ZSSB, ki ne uporablja učne slike, doseže visok povprečni mIoU 81,1%. Zaradi teh pomanjkljivosti zgradimo novo podatkovno množico LVIS-1025, ki jo dobimo iz podatkovne množice LVIS z uporabo novih meril za merjenje predvidljivosti in izraznosti objektov. Na LVIS-1025 evalviramo tri najsodobnejše metode (PANet, PPNet in ASGNet) in pokažemo, da se vrstni red uspešnosti spremeni v primerjavi s pridobljenim na obstoječih podatkovnih množicah. ASGNet na standardnih podatkovnih množicah močno preseže PANet in PPNet, vendar je na LVIS-1025 slabši, kar nakazuje, da je ASGNet nagnjen k segmentiranju najbolj izraznega objekta na sliki. Verjamemo, da bodo prihodnji modeli razviti na LVIS-1025 zmožni boljšega posploševanja in se ne bodo tako močno opirali na predpostavko, da je ciljni predmet vedno prisoten na sliki.

• Keywords: računalniški vid, globoko učenje, konvolucijske nevronske mreže, semantična segmentacija, učenje z malo učnimi primeri, evalvacijski protokol