An improved visual model for tracking by segmentation

Nedanovski, Tilen (author), Kristan, Matej (mentor)

As if in response to the increased focus of the field on visual object tracking and video object segmentation, this work features several trackers escalating the associations between the two disciplines. These trackers, in particular, build upon an existing D3S tracker that has the capacity to produce both highly-reliable localization as well as an accurate segmentation of the target. Furthermore, said products are used in future target state inference to inform the process and achieve excellent tracking performance. In recognition of the benefits reaped by involving segmentation in visual object tracking, this work proposes several trackers in an effort to further both the accuracy and robustness of the D3S, as well as to improve its speed of inference. Novel trackers are compounded from existing components of the D3S implementation along with other constituents giving prominence to the latest advancements in the field. Namely, the two backbones of the original implementation are merged into a single backbone, CARAFE modules are instated to replace the bilinear upsampling stages, Octave convolution is introduced to improve the speed of feature extraction and the attention mechanism is implemented to incorporate contextual information into the tracking process. Alongside this, the lack of dataset diversity inspires a synthetic dataset to be constructed and used in pre-training stages of representation learning. Finally, the suitability of proposed tracking architectures is determined through rigorous evaluation.

• Language: English
• Keywords: computer vision, visual object tracking, segmentation, correlation filters
• Work type: Master's thesis/paper
• Typology: 2.09 - Master's Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2020
• PID: 20.500.12556/RUL-120554
• COBISS.SI-ID: 34078979
• Publication date in RUL: 22.09.2020

NEDANOVSKI, Tilen, 2020, An improved visual model for tracking by segmentation [online]. Master’s thesis. [Accessed 18 March 2025]. Retrieved from: https://repozitorij.uni-lj.si/IzpisGradiva.php?lang=eng&id=120554

Secondary language

• Language: Slovenian
• Title: Izboljšan vizualni model za sledenje s segmentacijo

Abstract

Magistrsko delo obravnava obojestranske koristi med vizualnim sledenjem objektov in segmentacijo objektov v videoposnetkih. Plodovi te obravnave so sledilniki, ki temeljijo na obstoječi metodi sledenja D3S. Poleg visoko zanesljive lokalizacije je sledilnik D3S zmožen tudi natančne segmentacije sledenega objekta, kar dodatno prispeva k uspešnosti metode. To dejstvo tesneje povezuje pričujoči disciplini računalniškega vida. Skozi vsebino dela se koristi, ki jih prinaša segmentacija v sožitju z vizualnim sledenjem objektov, kažejo v več predlaganih sledilniških arhitekturah. Te arhitekture v prizadevanju za izboljšanje natančnosti in robustnosti metode D3S proces sledenja nadgrajujejo ter bogatijo z novimi informacijami. Ena izmed predlaganih arhitektur, na primer, združuje enaki, vendar prvotno ločeni ogrodji omrežja v eno samo v prid hitrosti sledenja. Spet druga vpeljuje operatorje CARAFE na mestih bilinearne interpolacije, in sicer z namenom vključitve informacij širšega konteksta v vzorčenje značilk. Iz enakih razlogov je v tretji arhitekturi dodan mehanizem pozornosti. Poleg novih arhitektur delo obsega tudi konstrukcijo sintetičnega nabora podatkov, navdih čemur so pomanjkljivosti obstoječih zbirk podatkov. Delo se zaključi z eksperimentalno analizo kot merilom uspešnosti in ustreznosti predlaganih metod ter krajšo razpravo.

• Keywords: računalniški vid, vizualno sledenje objektom, segmentacija, korelacijski filtri