Nevronska mreža za monokularno semantično segmentacijo in napovedovanje toka scene v vodnem okolju

Trček, Žiga (author), Kristan, Matej (mentor), Žust, Lojze (comentor)

Metode za napoved disparitete in toka scene imajo veliko vlogo pri zavedanju okolice avtonomnih vozil in plovil. Za napovedovanje pogosto potrebujejo drag in nepraktičen sistem dveh kamer. Na področju avtonomnih vozil je bilo na problemu napovedovanja toka scene z eno kamero izvedenih več raziskav, ustreznost metod na vodnem okolju, ki je zaradi odbleskov in zrcaljenja bistveno drugačno od cestnega, pa ni bila raziskana. Ena najuspešnejših metod za napoved toka scene in disparitete z eno kamero v realnem času je Self-Mono-SF. Na podlagi te metode razvijemo metodo SceneFlowSegmentation, ki v realnem času z uporabo ene kamere napoveduje dispariteto in tok scene, nato pa vmesne informacije uporabi za semantično segmentacijo. Metodo učimo na podatkovnih zbirkah MODD2 in MaSTr1325 ter jo evalviramo na podatkovni zbirki MODS. Napoved disparitete in toka scene primerjamo z napovedjo izhodiščnega modela, učenega na mestni domeni in ugotovimo, da učenje na morski domeni izboljša rezultate. Rezultate segmentacije primerjamo z rezultati modela WaSR. Ocena F1 razvite metode je za 2 odstotni točki slabša od referenčne, a metoda z enako hitrostjo (10 slik na sekundo) hkrati napoveduje še dispariteto in tok scene.

• Language: Slovenian
• Keywords: tok scene, dispariteta, semantična segmentacija, monokularna kamera, avtonomno plovilo
• Work type: Bachelor thesis/paper
• Typology: 2.11 - Undergraduate Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2023
• PID: 20.500.12556/RUL-149213
• COBISS.SI-ID: 166800387
• Publication date in RUL: 05.09.2023

Secondary language

• Language: English
• Title: A neural network for monocular semantic segmentation and scene flow estimation in an aquatic environment

Abstract

Disparity and scene flow estimation methods play a large role in the perception of the environment of autonomous vehicles and vessels. They often require expensive and unpractical stereo camera systems to ensure accuracy. Monocular approaches have been well researched in city environments, but there has been no research about the viability of such approaches in maritime environments that pose challenges such as object mirroring and glitter. We use Self-Mono-SF, one of the best real-time monocular disparity and scene flow estimation methods, as a basis for SceneFlowSegmentation, a novel method that predicts scene flow and disparity and semantic segmentation in real-time. We use the MODD2 and MaSTr1325 datasets for training and the MODS dataset for evaluation. We compare the disparity and scene flow estimation with the predictions of Self-Mono-SF that is trained on a city domain and observe improvements in accuracy. Segmentation results are compared to the current state-of-the-art method, WaSR. The F1 score is decreased by 2 percentage points, however the newly developed method is able to accurately predict disparity and scene flow at the same time while operating at the same speed (10 frames per second).

• Keywords: scene flow, disparity, semantic segmentation, monocular camera, autonomous vessel