Vizualno sledenje poljubnega objekta v videoposnetku z napovedovanjem centralne točke

Fidel, Denis (author), Kristan, Matej (mentor)

V nalogi naslavljamo problem vizualnega sledenja poljubnega objekta v videoposnetku. Pri izbranem problemu uporabnik označi lokacijo objekta na prvi sliki, naloga algoritma pa je slediti izbran objekt čez celoten videoposnetek. Predstavljamo novost sledenja objekta z napovedovanjem centralne točke, katere dodatni atributi so velikost, odmik in premik glede na prejšnjo sliko. Predlagan sledilnik GOCT je razvit iz sledilnika več objektov CenterTrack in je posledično lokalen, saj za napoved uporablja trenutno, prejšnjo in začetno sliko. Lokacijo napovedujemo s pomočjo konvolucijskih nevronskih mrež, pri čemer hrbtenična arhitektura temelji na globokem združevanju nivojev (DLA). Metoda GOCT na zbirki GOT10k pri 9FPS dosega povprečno prekrivanje $AO=0,610$ in delež uspešnosti $SR_{0,50}=0,669$ ter $SR_{0,75}=0,517$. V primerjavi z ostalimi iz članka GOT10k je to 75,29 % izboljšanje AO. O hitrosti razvoja računalniškega vida priča primerjava z aktualno lestvico GOT10k, kjer je metoda GOCT 23,65 % slabša od najboljšega rezultata. Na podatkovni zbirki LaSOT sledilnik GOCT doseže $AO=0,471$, natančnost 0,453 in normalizirano natančnost 0,476, kar potrjuje zmožnost generalizacije na poljuben objekt. Nevronska mreža namreč ni dodatno učena na zbirki LaSOT, kljub temu pa metoda GOCT pripada najboljšim desetim rezultatom iz članka LaSOT. V delu je razvita tudi nekoliko slabša, vendar precej hitrejša mreža, ki je sposobna procesiranja 30FPS. Ključna omejitev za nadaljnje izboljšave je odpoved sledilnika GOCT ob prekrivanju navidez podobnih objektov.

• Language: Slovenian
• Keywords: vizualno sledenje, poljuben objekt, lokalen sledilnik, centralna točka, konvolucijska nevronska mreža, globoko združevanje nivojev
• Work type: Master's thesis/paper
• Organization: FMF - Faculty of Mathematics and Physics
• Year: 2023
• PID: 20.500.12556/RUL-145667
• COBISS.SI-ID: 150959619
• Publication date in RUL: 06.05.2023

Secondary language

• Language: English
• Title: General visual object tracking in video by center point prediction

Abstract

We address the problem of visual object tracking of general object in a video. Problem requires user to input object location in the first frame and the algorithm tracks requested object through the rest of the video. In this work we propose tracking of general object using center point prediction with additional attributes such as size, offset and movement relative to previous frame. Proposed tracker GOCT is local, because it is based on multi object tracker CenterTrack. Similar to CenterTrack, our tracker predicts object location based on current, previous and first frame. We are predicting center point location using convolutional neural networks with backbone architecture based on deep layer aggregation (DLA). On GOT10k database the best version of GOCT runs at 9FPS and achieves average overlap $AO=0.610$ and success rate $SR_{0.50}=0.669$ and $SR_{0.75}=0.517$. Compared to the best result from GOT10k article, this is a 75.29% improvement of AO. Pace of computer vision development is highlighted by current results from GOT10k leaderboard where GOCT method is 23.65% worse than best result at the time of writing. On LaSOT dataset GOCT achieves $AO=0.471$, precision 0.453 and normalized precision 0.476 which confirms trackers ability to generalize to any object. Namely, the neural network is not trained on LaSOT dataset, but the method still achieves top ten results compared to LaSOT article. In this work we also developed a slightly worse, but much faster network capable of processing 30FPS. A key limitation for further improvements of GOCT is the inability to handle overlapping of similar objects in a video.

• Keywords: visual tracking, general object, local tracker, center point, convolutional neural network, deep layer aggregation