Razpoznavanje hrane na podlagi slik z nevronskimi mrežami

Sedevcic, Kevin (author), Kristan, Matej (mentor)

V magistrski nalogi naslavljamo problem klasifikacije slik in sestave opisov slik s tremi implementiranimi metodami z nevronskimi mrežami (klasifikacija hrane, sestava opisov hrane in sestava opisov hrane z regijami), ki so bile učene in testirane na dveh podatkovnih zbirkah. Prva razdeljena na 21 kategorij hrane z 1470 slikami in druga, podatkovna zbirka opisov (2 kategoriji in 5 opisnih stavkov na sliko). Prva implementirana metoda—metoda klasifikacije hrane—uporablja arhitekturo GoogLeNet-Inception-v3 mreže (že učena na zbirki ILSVRC), ki je bila dodatno učena na naši podatkovni zbirki hrane na kateri dosega 82.4% top-1 in 98% top-5 točnosti. Druga metoda—metoda sestave opisov—uporablja arhitekturo Show and Tell mreže, ki je inicializirana z našim modelom klasifikacije hrane in doseže 23.3 točk perpleksnosti. Metoda ne sestavi popolnih opisov slik, ko sta prisotna dva ali več objekta na sliki, zato smo implementirali še metodo, ki bi izpisala vsebovanost objektov v slikah. Tretja metoda—metoda za sestavo opisov slik z regijami—uporablja isti vizualni model, ki je uporabljen v prejšnjih dveh metodah a z razliko, da klasificira regije vhodne slike. Rezultat evaluacije nad isto podatkovno zbirko je 86.5% top-1 točnosti. Dodatna evaluacija, ki testira količino razpoznanih objektov v slikah z več različnimi objekti hrane je dokazala da metoda razpozna 64% objektov na slikah.

• Language: English
• Keywords: računalniški vid, računalništvo, strojno učenje
• Work type: Master's thesis/paper
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2017
• PID: 20.500.12556/RUL-98939
• Publication date in RUL: 14.12.2017

Secondary language

• Language: Slovenian
• Title: Image-based food recognition using neural networks

Abstract

In this thesis we address the problems of image classification and image captioning with three implemented methods with neural networks (food classification, food captioning and food captioning by region-proposal). The methods were trained and tested on a 21-category food image dataset with 1470 images and a 2-category food caption dataset with 750 caption sentences. The first method—food classification method—uses the architecture of the GoogLeNet-Inception-v3 model trained on our food dataset, achieving a top-1 prediction accuracy of 82.4% and top-5 prediction accuracy of 98%. The second method—food captioning method—uses the Show and Tell architecture trained on our food caption dataset, achieving a perplexity score of 23.3. Our food visual model was used to classify the input images, but the overall results did not meet expectations, as the model does not correctly caption images containing multiple foods. The third method—food captioning with region proposal—uses our food classification method to classify images and performs better than the food-classification method alone, achieving a prediction accuracy of 86.5%. Additionally, this third method summarizes the contents of images containing different types of food with an accuracy score of 64%.

• Keywords: computer vision, computer science, machine learning