Mobilna aplikacija z uporabo strojnega vida za podporo slepim in slabovidnim

Urankar, Eva (author), Pogačnik, Matevž (mentor), Pečnik, Klemen (comentor)

Diplomsko delo obravnava razvoj mobilne aplikacije z imenom »VIDA« (ang. Visual Interpretation and Direction Assistant), namenjene slepim in slabovidnim osebam kot podporno orodje za izboljšanje vsakodnevne samostojnosti ter orientacije v prostoru. Glavni cilj naloge je bil izdelati uporabniško prilagojeno in tehnično zanesljivo aplikacijo, ki deluje tudi brez internetne povezave, podpira slovenski jezik in vključuje ključne funkcionalnosti zaznavanja okolja v realnem času. V teoretičnem delu naloge so analizirane obstoječe podporne rešitve ter njihove pomanjkljivosti, zlasti glede dostopnosti, funkcionalnosti in jezikovne podpore. Na podlagi teh ugotovitev je bila načrtovana in implementirana aplikacija, ki vključuje module za zaznavanje in razpoznavo objektov (YOLOv8), klasifikacijo prizorov (ResNet), optično prepoznavanje besedil (OCR) ter sintezo govora (TTS), vse v realnem času. Poseben poudarek je bil namenjen razvoju in učenju lastnega modela strojnega vida, ki omogoča zanesljivo delovanje v raznolikih svetlobnih pogojih in kompleksnih prizorih in dosega srednjo povprečno natančnost (mAP50) 0,48. Aplikacijo smo implementirali v programskem okolju Android z uporabo arhitekture Model–Pogled–Model pogleda (MVVM) in knjižnice Jetpack Compose, kar omogoča visoko modularnost, odzivnost ter prilagodljivost uporabniškega vmesnika za uporabnike z okvarami vida. Aplikacijo smo testirali v sodelovanju s slepo uporabnico, pri čemer so bile potrjene uporabnost, intuitivnost ter stabilnost rešitve v realnem okolju. Rezultati diplomskega dela dokazujejo, da je s pomočjo sodobnih pristopov na področju računalniškega vida in umetne inteligence mogoče razviti kakovostno in učinkovito mobilno aplikacijo, ki konkretno pripomore k večji vključenosti ter samostojnosti slepih in slabovidnih oseb v družbi. V zaključku so predstavljeni predlogi za nadgradnjo, zlasti na področju integracije dodatnih funkcij.

• Language: Slovenian
• Keywords: računalniški vid, umetna inteligenca, mobilna aplikacija, dostopnost YOLOv8, OCR, TTS, VIDA
• Work type: Undergraduate thesis
• Typology: 2.11 - Undergraduate Thesis
• Organization: FE - Faculty of Electrical Engineering
• Year: 2025
• PID: 20.500.12556/RUL-170069
• COBISS.SI-ID: 242790915
• Publication date in RUL: 02.07.2025

Secondary language

• Language: English
• Title: A mobile application using computer vision to assist blind and visually impaired persons

Abstract

This thesis presents the development of a mobile application entitled »VIDA« (Visual Interpretation and Direction Assistant), designed as an assistive tool for blind and visually impaired individuals to enhance their independence and spatial orientation in daily life. The primary objective was to create a user-friendly and technically robust application that operates offline, supports the Slovenian language, and incorporates key real-time environment recognition functionalities. The theoretical part of the thesis provides an in-depth analysis of existing assistive technologies, identifying critical limitations in terms of accessibility, functionality, and linguistic support. Based on these insights, »VIDA« was designed and implemented with integrated modules for object detection (YOLOv8), scene classification (ResNet), optical character recognition (OCR), and text-to-speech synthesis (TTS), all functioning in real time. We trained a custom computer vision model capable of reliable performance under diverse conditions, achieving a mean Average Precision (mAP50) of 0.48. The application was implemented for the Android platform using the Model–View–ViewModel (MVVM) architecture and Jetpack Compose UI framework, providing high modularity, responsiveness, and accessibility for users with visual impairments. The solution was tested in collaboration with a blind user, confirming its usability, intuitiveness, and stability in real-world scenarios. The results of this work demonstrate that modern computer vision and artificial intelligence approaches can be effectively applied to develop high-quality mobile solutions that significantly contribute to the inclusion and autonomy of blind and visually impaired individuals. The conclusion outlines future development directions, particularly in the integration of additional assistive functionalities.

• Keywords: computer vision, artificial intelligence, mobile application, accessibility, YOLOv8, OCR, TTS, VIDA