Vodena super-resolucija termalnih slik z RGB referencami

ŠINKOVEC, ROK (avtor), Štruc, Vitomir (mentor), Perš, Janez (komentor)

V našem delu predstavljamo metodo s katero rešujemo problematiko vodene super-resolucije termalnih slik. Metoda temelji na pristopih strojnega učenja z uporabo konvolucijskih nevronskih mrež, ki so učene v enovitnem načinu. Model si za pomoč pri rekonstrukciji visokofrekvenčnih detajlov izhodnih visokoresolucijskih termalnih slik pomaga z visokofrekvenčnimi RGB referencami. Kot izvorne slike pa prejme realne termalne slike, zajete z nizko resolucijsko termalno kamero. Model ima preprosto pet-slojno strukturo, z dvema ločenima vhodoma konvolucijskih slojev. Za zajem zbirke slik smo sestavili senzorski sistem treh kamer, s katerim smo zajemali slike v LWIR in RGB spektru. Za evaluacijo modela sta bili zbrani dve zbirki slik, od katerih prva predstavlja omejeno svetlobno okolje, druga pa predstavlja nenadzorovano svetlobno okolje. Zbirki je bilo potrebno pred učenjem urediti z oceno homografije. Modelom smo v fazi testiranja spreminjali velikosti in število filtrov v konvolucijskih slojih, ter učne podatke. Rezultate učenja pa smo ovrednotili z merami kakovosti slik SSIM in PNSR, ki sta danes široko uporabljeni. Model je po koncu učenja prve zbirke rekonstruiral slike z vrednostjo povprečne SSIM 0.84 in vrednosti povprečnega PNSR 22.67 dB. Pri vrednotenju rekonstrukcij pri drugi podatkovni zbirki pa je model dosegel povprečno vrednosti SSIM 0.62, vrednost povprečnega PNSR pa je dosegala 17.713 dB.

• Jezik: Slovenski jezik
• Ključne besede: vodena super-resolucija, konvolucijske nevronske mreže, strojno učenje, senzorski sistem, LWIR, mere kakovosti, SSIM, PNSR
• Vrsta gradiva: Magistrsko delo/naloga
• Organizacija: FE - Fakulteta za elektrotehniko
• Leto izida: 2020
• PID: 20.500.12556/RUL-121072
• Datum objave v RUL: 29.09.2020

Sekundarni jezik

• Jezik: Angleški jezik
• Naslov: Guided super-resolution of thermal images using RGB references

Izvleček

In our work, we present a method by which we solve the problem of guided super-resolution of thermal images. The method is based on machine learning approaches using convolutional networks that are learned in a end-to-end manner. The model uses high-frequency RGB references to help reconstruct the high-frequency details of the output high-resolution thermal images. As source images, it receives realistic thermal images captured by a low-resolution thermal camera. The model has a simple five-layer structure with two separate entrances of convolutional layers. To capture the image collection, we assembled a sensor system of three cameras with which we captured images in the LWIR and RGB spectrum. Two collections of images were collected to evaluate the model, the first representing a limited light environment and the second representing an uncontrolled light environment. The collection had to be edited with the homography assessment method before learning. In the testing phase, we changed the sizes and number of filters in the convolution layers, as well as the learning data. Learning outcomes were evaluated with SSIM and PNSR image quality measures, which are widely used today. After learning the first collection, the model reconstructed images with an average SSIM value of 0.84 and an average PNSR value of 22.67 dB. When evaluating the reconstructions in the second database, the model achieved an average SSIM value of 0.62, and the average PNSR value reached 17,713 dB

• Ključne besede: guided super-resolution, convolutional neural networks, machine learning, sensor system, LWIR, quality measures, SSIM, PNSR