Superposition and compression of deep neutral networksZeman, Marko (Avtor) Bosnić, Zoran (Mentor) Osipov, Evgeny (Komentor) Artificial IntelligenceMachine LearningDeep LearningConvolutional Neural NetworksModel CompressionSuperposition of ModelsIn this work we investigate a combination of the two recently proposed techniques: superposition of multiple neural networks into one and neural network compression. We show that these two techniques can be successfully combined to deliver a great potential for trimming down deep (convolutional) neural networks. We study the trade-offs between the model compression rate and the accuracy of the superimposed tasks and present a new approach, where the fully connected layers are isolated from the convolutional layers and serve as a general purpose processing unit for several CNN models. We evaluate our techniques on adapted MNIST and CIFAR-100 dataset, calculating classification accuracy and comparing baseline to the superposition method. Our experiments confirm the usability of superposition in terms of avoiding the catastrophic forgetting effect. The work has a significant importance in the context of implementing deep learning on low-end computing devices as it enables neural networks to fit edge devices with constrained computational resources (e.g. sensors, mobile devices, controllers).20202020-09-03 14:45:02Magistrsko delo/naloga119135sl