Approximate computing for resource efficient on-device neural network training

Ciglarič, Timotej (author), Pejović, Veljko (mentor), Machidon, Octavian Mihai (comentor)

Since most neural network approximation techniques are limited to inference, we work to adapt it for practical usage during training, establishing options for training on weaker edge devices. We build on previous work to expand upon perforated convolution as a neural network approximation technique as well as adapt it for use during training. We speed up convolutional layers by skipping their evaluation and gradient calculation in some spatial positions, then interpolating the missing values. We analyze the effect of using perforated convolution during training. We show that on weak edge devices, using perforated convolution training with segmentation networks such as AgriAdapt U-Net reduces memory usage by up to 30% with up to 5x speedup, with a significant reduction in training energy usage.

• Language: English
• Keywords: convolutional neural networks, perforated convolution, approximated training, sustainable computing, AI on the edge
• Work type: Master's thesis/paper
• Typology: 2.09 - Master's Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2026
• PID: 20.500.12556/RUL-178939
• COBISS.SI-ID: 283318531
• Publication date in RUL: 02.02.2026

Secondary language

• Language: Slovenian
• Title: Približno učenje za učinkovito učenje nevronskih mrež na mobilnih napravah

Abstract

Ker je večino postopkov za aproksimacijo nevronskih mrež omejenih na inferenco, smo jih v tem delu prilagodili za praktično uporabo med učenjem. S tem gradimo nove možnosti za učenje mrež na šibkejših robnih napravah. Nadgradili smo prejšnje delo in s prilagoditvijo za uporabo med učenjem razširili uporabo perforirane konvolucije kot tehnike za aproksimacijo nevronskih mrež. Konvolucijske nivoje smo pospešili z izpuščanjem njihove evalvacije na določenih mestih in z nadaljnjo interpolacijo manjkajočih vrednosti. Analizirali smo učinek uporabe perforirane konvolucije med učenjem. Pokazali smo, da na šibkih robnih napravah učenje segmentacijskih nevronskih mrež kot je AgriAdapt U-Net porabi do 30\% manj spomina in se izvaja do 5x hitreje, z velikim zmanjšanjem skupne porabe energije.

• Keywords: konvolucijske nevronske mreže, perforirana konvolucija, približno učenje, trajnostno računanje, UI na robnih napravah