A hierarchical adaptive model for robust short-term visual tracking : dissertation

Čehovin Zajc, Luka (author), Leonardis, Aleš (mentor), Kristan, Matej (comentor)

Vizualno sledenje je področje v okviru računalniškega vida, katerega rezultate je mogoče uporabiti na mnogih, tako novih kot tudi že uveljavljenih, področjih, kot so npr. robotika, video-nadzorni sistemi, interakcija med človekom in računalnikom, avtonomna vozila ter analiza športa. Glavno vprašanje vizualnega sledenja je razvoj algoritmov (sledilnikov), ki določajo stanja enega ali več objektov v toku slik ob upoštevanju časovne soslednosti le-teh. V tej doktorski disertaciji naslavljamo dve raziskovalni temi iz področja kratkoročnega vizualnega sledenja. Prvi sklop predstavljenih raziskav naslavlja konstrukcijo vizualnega modela, ki ga sledilnik uporablja za opis izgleda objekta. Vprašanje modeliranja ter osveževanje vizualnega modela je eno izmed ključnih vprašanj vizualnega sledenja. V okviru dela najprej predstavimo hierarhični vizualni model, ki izgled strukturira v več plasti. Najnižja plast vsebuje najbolj specifične informacije o izgledu, višje plasti pa opisujejo izgled v bolj posplošeni obliki. Hierarhična urejenost se odraža tudi v posodabljanju vizualnega modela, kjer višje plasti vodijo posodabljanje nižjih plasti, le-te pa v primeru lastne zanesljivosti služijo kot vir informacij za osveževanje višjih plasti. Koristi hierarhičnega modela sta predstavljeni z dvema implementacijama, ki sta primarno namenjeni sledenju netogih in artikuliranih objektov, kot tiste kategorije objektov, ki predstavlja velik problem za marsikateri vizualni sledilnik. Prvi predlagani model združuje lokalno in globalno predstavitev izgleda v sklopljenem vizualnem modelu. Spodnja plast je sestavljena iz več med seboj povezanih delov, ki so se sposobni prilagajati geometrijskim spremembam netogih objektov, zgornja plast pa vsebuje večmodalno globalno predstavitev izgleda, ki vodi proces posodobitve spodnje plasti. V okviru eksperimentalne analize smo pokazali, da se tak sklopljeni model izgleda izkaže v robustnosti, klub dejstvu, da smo za opis izgleda uporabili sorazmerno preproste opisnike. Analiza razkrije tudi nekaj pomanjkljivosti modela, ki se kažejo v znižani natančnosti sledenja. Zato naš drugi predstavljeni model razširja hierarhijo s tretjo plastjo in konceptom sidrnih predlog. Prvi dve plasti drugega vizualnega modela sta konceptualno zelo podobni osnovnemu sklopljenemu vizualnemu modelu, tretja plast pa vsebuje spominski sistem statičnih predlog, ki vizualnemu modelu nudijo močno informacijo o položaju in velikosti objekta v primeru dobrega ujemanja ene izmed predlog s sliko. Na ta način tretja plast pripomore k hitremu okrevanju celotnega vizualnega modela. Predstavljena eksperimentalna analiza koristi tretje plasti potrdi, saj sledilnik s tem modelom izgleda izboljša natančnost, pa tudi splošno kvaliteto sledenja. Drugo vprašanje, ki ga naslavljamo v tej doktorski disertaciji, je ocenjevanje performans kartkoročnih sledilnikov. V nasprotju s prevladujočimi trendi v zadnjih desetletjih trdimo, da je vizualno sledenje kompleksen proces, katerega lastnosti ni mogoče opisati z eno samo mero uspešnosti, po drugi strani pa tudi ne smemo uporabiti poljubne množice mer, za katere ne poznamo medsebojnih odnosov. V naši raziskavi smo zato pregledali in analizirali pogosto uporabljene mere performans in pokazali, da nekatere izmed njih merijo iste kvalitete ali pa so celo teoretično ekvivalentne. Na temelju te analize smo predlagali par dveh šibko koreliranih mer, ki odražata natančnost in robustnost sledilnega algoritma, ustrezen prikaz takih rezultatov ter analizo celotne metodologije s pomočjo predlaganih teoretičnih sledilnikov, ki izražajo ekstremno obnašanje sledilnih algoritmov. Vse to smo nadgradili še z metodologijo rangiranja večjega števila sledilnikov, ki upošteva morebitno stohastično naravo sledilnikov ter preveri statistično značilnost razlike med njihovimi rezultati. Celotno metodologijo smo implementirali v odprtokodnem programskem orodju, razvili pa smo tudi preprost komunikacijski protokol, ki omogoča preprosto integracijo obstoječih implementacij sledilnikov v sistem. Z uporabo razvitega orodja se predlagana metodologija sedaj uporablja tudi v okviru Visual Object Tracking (VOT) challenge delavnic in tekmovanj.

• Language: English
• Keywords: computer vision, visual tracking, visual model, short-term tracking, articulated object, non-rigid objects, performance measures, performance evaluation, ranking
• Work type: Doctoral dissertation
• Typology: 2.08 - Doctoral Dissertation
• Organization: FRI - Faculty of Computer and Information Science
• Publisher: [L. Čehovin]
• Year: 2015
• Number of pages: 167 str.
• PID: 20.500.12556/RUL-72632
• COBISS.SI-ID: 1536554691
• Publication date in RUL: 29.09.2015

Secondary language

• Language: Slovenian
• Title: Hierarhični adaptivni model za robustno kratkoročno vizualno sledenje

Abstract

Visual tracking is a topic in computer vision with applications in many emerging as well as established technological areas, such as robotics, video surveillance, human-computer interaction, autonomous vehicles, and sport analytics. The main question of visual tracking is how to design an algorithm (visual tracker) that determines the state of one or more objects in a stream of images by accounting for their sequential nature. In this doctoral thesis we address two important topics in single-target short-term visual tracking. The first topic is related to construction of an object appearance model for visual tracking. The modeling and updating of the appearance model is crucial for successful tracking. We introduce a hierarchical appearance model which structures object appearance in multiple layers. The bottom layer contains the most specific information and each higher layer models the appearance information in a more general way. The hierarchical relations are also reflected in the update process where the higher layers guide the lower layers in their update while the lower layers provide a source for adaptation to higher layers if their information is reliable. The benefits of hierarchical appearance models are demonstrated with two implementations, primarily designed to tackle tracking of non-rigid and articulated objects that present a challenge for many existing trackers. The first example of appearance model combines local and global visual information in a coupled-layer appearance model. The bottom layer contains a part-based appearance description that is able to adapt to the geometrical deformations of non-rigid targets and the top layer is a multi-modal global object appearance model that guides the model during object appearance changes. The experimental evaluation shows that the proposed coupled-layer appearance model excels in robustness despite the fact that is uses relatively simple appearance descriptors. Our evaluation also exposed several weaknesses that were reflected in a decreased accuracy. Our second presented appearance model extends the hierarchy by introducing the third layer and a concept of template anchors. The first two layers are conceptually similar to the original two-layer appearance model, while the third layer is a memory system that is composed of static templates that provide a strong spatial cue when one of the templates is matched to the image reliably, thus assisting in quick recovery of the entire appearance model. In the experimental evaluation we show that this addition indeed improves the accuracy, as well as the overall performance of a tracker. The second question that we are addressing is the performance evaluation of single-target short-term visual tracking algorithms. In contrast to the dominant trend in the past decades, we claim that visual tracking is a complex process and that the performance of visual trackers cannot be reduced to a single performance measure, nor should it be described by an arbitrary set of measures where the relationship between measures is not well understood. In our research we investigate performance measures that are traditionally used in performance evaluation of single-target short-term visual trackers, through theoretical and empirical analysis, and show that some of them are measuring the same aspect of tracking performance. Based on our analysis we propose a pair of two weakly correlated measures to measure the accuracy and robustness of a tracker, propose a visualization of the results as well as the analysis of the entire methodology using the theoretical trackers that exhibit extreme tracking behaviors. This is followed by an extension of the methodology on ranking of multiple trackers where we also take into account the potentially stochastic nature of visual trackers and test the statistical significance of performance differences. To support the proposed evaluation methodology we have developed an open-source software tool that implements the methodology and a simple communication protocol that enables a straightforward integration of trackers. The proposed evaluation methodology and the evaluation system have been adopted by several Visual Object Tracking (VOT) challenges.

• Keywords: računalniški vid, vizualno sledenje, vizualni model, kratkoročno sledenje, artikulirani objekti, ne-togi objekti, mere performans, ocenjevanje performans, rangiranje