Analiza katastrofalnega pozabljanja pri inkrementalnem učenju klasifikacijske konvolucijske nevronske mreže

Božič, Jakob (author), Skočaj, Danijel (mentor)

Katastrofalno pozabljanje je pojav, ko umetna nevronska mreža ob inkrementalnem učenju novih nalog nemudoma in skoraj v celoti pozabi prejšnje. Problem je dobro znan in obstajajo različni pristopi k odpravljanju oz. omejevanju le-tega, vendar ga noben izmed pristopov ne reši v celoti. V delu eksperimentalno preverimo, kateri so glavni dejavniki, ki privedejo do katastrofalnega pozabljanja. Analizo opravimo na globoki konvolucijski nevronski mreži, na problemu klasifikacije slik. Rezultate interpretiramo z matrikami zamenjav in grafi klasifikacijskih točnosti, spremembe uteži in odmikov tudi vizualiziramo. Dognanja iz analize uporabimo za zasnovo različnih pristopov k osveževanju parametrov, s katerimi želimo preprečiti oz. omiliti katastrofalno pozabljanje. Preverimo tudi scenarij, kjer imamo ob uporabi nevronske mreže na voljo oraklja, ki določi podmnožico razredov, v katere lahko klasificiramo primer. Implementiramo eno izmed obstoječih metod za odpravljanje katastrofalnega pozabljanja in jo prilagodimo, da deluje tudi brez oraklja. Ugotovitve, predstavljene v delu, služijo kot izhodišče za zasnovo novih metod za odpravo katastrofalnega pozabljanja.

• Language: Slovenian
• Keywords: katastrofalno pozabljanje, inkrementalno učenje, konvolucijske nevronske mreže, klasifikacija
• Work type: Bachelor thesis/paper
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2019
• PID: 20.500.12556/RUL-107841
• Publication date in RUL: 30.05.2019

Secondary language

• Language: English
• Title: Analysis of catastrophic forgetting during incremental learning of classificational convolutional neural network of classificational convolutional neural network

Abstract

Catastrophic forgetting is phenomenon when an artificial neural network immediately and almost completely forgets previously learned tasks when trained incrementally on new ones. It is a well-known problem and although there are many approaches to alleviating it, none of them solves it completely. We experimentally check for main causes of catastrophic forgetting. Analysis is performed on a deep convolutional neural network for image classification. Results are interpreted by confusion matrices and classification accuracy graphs, we also visualize changes of weights and biases of network. Analytical findings serve as a basis for designing different approaches to updating network parameters, aiming to prevent or alleviate catastrophic forgetting. We also evaluate effects of availability of Oracle, capable of determining subset of all possible classes for classification, when using the network. We implement one of existing approaches to preventing catastrophic forgetting and adapt it to work without Oracle. Findings, presented in thesis serve as a starting point for design of new approaches aimed at preventing catastrophic forgetting.

• Keywords: catastrophic forgetting, incremental learning, convolutional neural networks, classification