ANALIZA PROJICIRANIH SLIKOVNIH VZORCEV ZA PRIDOBIVANJE GLOBINSKIH SLIK

KRAVANJA, JAKA (author), Dobrišek, Simon (mentor), Štruc, Vitomir (comentor)

Tema te disertacije je označevanje ujemajočih se parov vzorcev za pridobivanje globinskih slik v zunanjem okolju s pomočjo aktivne strukturirane osvetlitve. Največji problem, s katerim se soočamo pri zajemanju globinskih slik na prostem, so močna osvetlitev ozadja (sonce, drugi svetlobni viri) in motnje drugih podobnih sistemov, ki delujejo na istem območju, ali pa namerne motnje sistema. V delu je predstavljen sistem, ki uporablja poseben, sinhroniziran zajem slike projiciranega vzorca, s katerim odpravi večino motenj in neželene osvetlitve ozadja, s čimer pridobi želeno sliko projiciranega vzorca na prizoru. Pri metodah pridobivanja slike s strukturirano osvetlitvijo je ključni problem določitev ujemajočih se parov točk projiciranega vzorca in točk na zajeti sliki, ki pripadajo projiciranemu vzorcu. Podobni sistemi uporabljajo posebne projekcijske vzorce ali celo barve in s tem olajšajo problem iskanja parov. V tem delu se osredotočimo na iskanje parov z uporabo preprostega enobarvnega vzorca (ena valovna dolžina projicirane svetlobe), ki je sestavljen iz vzporednih premic. Uporaba takega vzorca omogoča uporabo zelo preprostega projektorja vzorcev (laser s posebno lečo, ki žarek razprši v vzporedne premice), uporaba samo ene valovne dolžine svetlobe pa omogoča uporabo ozkopasovnih optičnih filtrov na kameri, ki zajema sliko. S tem se že izloči večina motenj. Pridobljena slika vsebuje projekcijski vzorec na opazovanem prizoru, ki ga je treba izluščiti iz slike in točkam na vzorcu dodeliti oznake oziroma pare, ki so pripadajoče točke na projiciranem vzorcu. V tem delu je predstavljena rešitev, ki ta problem rešuje z uporabo verjetnostnih grafičnih modelov. Predlagamo novo metodo, ki upošteva večje število zaporednih slik, in predstavimo vpliv števila slik na točnost označevanja projiciranih vzorcev. Zaznani projicirani vzorec na sliki razdelimo na večje število rojev slikovnih točk in vsakemu roju dodelimo naključno spremenljivko. Vsi roji slikovnih točk (naključnih spremenljivk) povežemo v verjetnostni grafični model. Naključne spremenljivke, ki pripadajo sosednjim rojem slikovnih točk, uporabimo za izgradnjo faktorjev, s katerimi modeliramo soodvisnost sosednjih naključnih spremenljivk. Upoštevamo soodvisnosti v vodoravni in navpični smeri, vključimo pa še informacijo zaporednih slik, tako da določimo faktorje med prekrivajočimi se roji slikovnih točk v zaporednih slikah na istem položaju na sliki. Izkaže se, da upoštevanje te informacije iz zaporednih slik pomaga pri določevanju oznak rojev slikovnih točk. Kljub specializiranemu senzorju, ki odpravi večino motenj, se lahko v sliki pojavijo slikovni elementi (roji slikovnih točk), motnje, ki niso posledica projiciranega vzorca. Take motnje poskušamo odpraviti s posebno prilagoditvijo grafičnega modela. Naključnim spremenljivkam pri gradnji verjetnostnega grafičnega modela dodamo še eno stanje, ki predstavlja oznako »šum«. Zaželeno je, da model rojem točk, ki ne pripadajo projiciranemu vzorcu, dodeli to oznako. V zadnjem delu predstavimo rešitev z uporabo vrste povratnih nevronskih mrež (ang. recurrent neural network), nevronskega omrežja z dolgim kratkoročnim spominom (ang. Long short-term memory, LSTM). Vhodno sliko razdelimo na večje število manjših področij, ki predstavljajo vhodno zaporedje v nevronsko mrežo. Vsakemu področju dodelimo ustrezno oznako v odvisnosti od dela projekcijskega vzorca na danem področju slike. Nevronsko mrežo učimo tako, da hkrati na sliki zaznava in označuje projicirane vzorce iz surovih podatkov slike brez njene predobdelave. S to metodo je mogoče z veliko natančnostjo na sliki pravilno zaznati in označiti projicirani vzorec.

• Language: Slovenian
• Keywords: Zajemanje globinskih slik, triangulacija, strukturirana osvetlitev, verjetnostni grafični modeli, povratne nevronske mreže, LSTM
• Work type: Doctoral dissertation
• Organization: FE - Faculty of Electrical Engineering
• Year: 2016
• PID: 20.500.12556/RUL-82696
• COBISS.SI-ID: 11364692
• Publication date in RUL: 16.05.2016

Secondary language

• Language: English
• Title: ANALYSIS OF PROJECTED PATTERNS FOR DEPTH IMAGE ACQUISITION

Abstract

The subject of this dissertation is matching correspondence points for depth image acquisition in outdoor environments utilizing active illumination techniques. The biggest challenge that we are facing when acquiring depth image data outdoor is the presence of strong background illumination (e.g. sunlight) and disturbances caused by other similar systems operating in the same environment or intentional disturbances of the system. In this work a synchronized image acquisition system is presented, which eliminates most of the disturbances and unwanted background illumination, yielding the desired image of illumination pattern on the object. With structured light techniques, the key problem which needs to be solved before obtaining a three dimensional reconstruction of the image is finding pairs of points in the image which correspond to the same part of the projected pattern. Similar active triangulation systems utilize special projection patterns or even color which make the correspondence problem easier. In this work we focus on correspondence problem using a simple monochromatic projection pattern (one wavelength) composed of parallel lines (light planes). Utilizing such a projection pattern enables the use of very simple pattern projectors (laser with a special lens which splits the beam in several parallel lines). The use of monochromatic light also enables the use of band-pass optical filters on the camera, which already filters out most of the disturbances. The acquired frame contains the deformed projected pattern on the observed scene which needs to be segmented out and points on the pattern need to be assigned a pair from the original projected pattern, which comes down to assigning a corresponding label of a projected pattern light plane. In this work a solution utilizing probabilistic graphical models is presented. We propose a method that takes into account a larger number of subsequent frames and test the effect on accuracy of labeling projected patterns. The detected (and segmented out) pattern is divided on a larger number of small pixel-clusters which are also assigned a random variable. All the pixelclusters (random variables) are connected to form a probabilistic graphical model. The random variables which belong to neighboring pixel-clusters are used to form factors through which co-dependencies of neighboring pixel-clusters are modeled. Codependency in vertical and horizontal direction is considered, we also include the information of subsequent images, so that we assign factors between overlapping pixelclusters in subsequent frames on the same position in the image. It turns out that including this information helps with labeling the pixel-clusters. Despite the use of a special imaging sensor which suppresses most of the background and disturbances, unwanted pixel-clusters can appear in the image which are not caused by the projected pattern. We try to remove such pixel-clusters with a special modification of factors in graphical models. When constructing the probabilistic graphical model we assign an additional state to the random variables, which represents the label “noise”. It is desired that the model assigns this label to the pixel-clusters which do not belong to the projected pattern. In the last part of this work we present a solution using a kind of recurrent neural network, long short-term memory (LSTM). The input image is partitioned into a larger number of smaller sub-images, which represent the input sequence for the neural network. Each sub-image is assigned a label depending on the part of the projected pattern which is occupying that particular part of the image. We train the neural network end-to-end from raw image data to label projected patterns in the corresponding parts of the image. The image thus does not need to be preprocessed (thresholding, pixel-cluster detection). Using this method it is possible to detect and assign the correct labels to the projected pattern with high accuracy.

• Keywords: Depth image acquisition, triangulation, structured light, probabilistic graphical models, recurrent neural networks, LSTM