Določanje osvetlitve scene v obogateni resničnosti

MODIC, LEON (author), Čehovin Zajc, Luka (mentor)

Eden izmed glavnih načinov uporabe obogatene resničnosti je dodajanje objektov in označb v 3D prostor zajet s kamero mobilne naprave. Novi objekti v sceni morajo biti osvetljeni na način, ki odraža resnično osvetlitev, saj se na ta način zdijo bolj resnični in se zlijejo z okolico. V tem diplomskem delu smo razvili metodo za hitro in robustno določanje osvetlitve scene z uporabo konvolucijskih nevronskih mrež, ki bo lahko uporabljena v kontekstu obogatene resničnosti. Izdelali smo tudi zbirko sintetičnih podatkov, ki je bila uporabljena za učenje razvite mreže. Naučili smo več modelov z različnimi arhitekturami hrbtenice in primerjali njihovo natančnost ter hitrost na zbirki zajetih fotografij iz resničnega sveta. Rezultati eksperimentov kažejo uspešnost pri določanju smeri glavnega svetlobnega vira s pomočjo konvolucijskih nevronskih mrež tudi na podatkih, ki jih mreža med učenjem ni videla. Na koncu smo rezultate tudi vizualizirali na nekaterih izmed zajetih fotografij.

• Language: Slovenian
• Keywords: računalniški vid, obogatena resničnost, osvetlitev, konvolucijske nevronske mreže
• Work type: Bachelor thesis/paper
• Typology: 2.11 - Undergraduate Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2022
• PID: 20.500.12556/RUL-135481
• COBISS.SI-ID: 101461251
• Publication date in RUL: 16.03.2022

Secondary language

• Language: English
• Title: Determining scene illumination in augmented reality

Abstract

One of the main use cases of augmented reality is to add objects and markings to a 3D space captured by a mobile device camera. New objects in a scene need proper lighting that reflects real lighting, as this way they appear more realistic and blend in better with the surroundings. In this dissertation, we developed a method for fast and robust detection of scene lighting using convolutional neural networks, which could be used in the context of augmented reality. We also created a dataset consisting of synthetic images used for training the convolutional neural network. We trained multiple models with different backbone architectures, and we compared their accuracy and speed on a dataset consisting of captured photos from the real world. The results of experiments demonstrate that convolutional neural networks can successfully determine the direction of the main light source on data not seen by the network during training. In the end, we visualized the results on some of the captured real world photos.

• Keywords: computer vision, augmented reality, lighting, convolutional neural networks