Client Clustering for Improved Federated Learning on Heterogeneous Data

Radovič, Boris (author), Pejović, Veljko (mentor)

Federated learning (FL) is a distributed machine learning paradigm in which a model is collectively trained by using the data available on multiple devices without such devices exposing their data. This concept marks a significant stride towards decentralized AI. However, the challenge arises when dealing with non-independently and identically distributed (non-IID) data, as any kind of data heterogeneity among devices' datasets can hinder training convergence and worsen the predictive quality of the model being trained. Among the many techniques recently proposed for addressing such difficulties, there is clustering. Established clustering methods require the devices to possess a labelled dataset in order to assign the devices to a cluster, and this limits the applicability of such clustering approaches. In this thesis, we introduce a comprehensive framework and a suite of algorithms designed to cluster devices that lack a labelled dataset. Through experimentation, we demonstrate that our proposed algorithms yield results comparable to current state-of-the-art methods. An advantage of our approach is its capability to cluster devices that did not participate in the training stage. This includes cases where devices lack a labelled dataset or the devices' computational capabilities are limited.

• Language: English
• Keywords: federated learning, deep learning
• Work type: Master's thesis/paper
• Typology: 2.09 - Master's Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2023
• PID: 20.500.12556/RUL-149119
• COBISS.SI-ID: 164990467
• Publication date in RUL: 04.09.2023

Secondary language

• Language: Slovenian
• Title: Gručenje odjemalcev za izboljšanje zveznega učenja na heterogenih podatkih

Abstract

Zvezno učenje (ZU) je pristop, v katerem množica naprav sodeluje z namenom treniranja modela strojnega učenja. Pri tem si sodelujoče naprave ne izmenjujejo surovih podatkov, tako da ohranja postopek varnost in zasebnost uporabnikovih podatkov. Med težavami, s katerimi se ZU trenutno sooča, je učenje modelov v primerih, ko so podatki na sodelujočih napravah porazdeljeni neenakomerno. V primeru prisotnosti heterogenosti podatkov se kakovost končnih predikcij treniranega modela zmanjša in v najhujših primerih lahko model celo divergira. Med uveljavljenimi pristopi, ki skušajo omiliti negativne posledice heterogenosti podatkov, je gručenje naprav. Sodobne metode gručenja v ZU-ju zahtevajo, da imajo naprave označeno podatkovno množico, ta predpostavka pa omejuje uporabnost takšnih pristopov. V magistrski nalogi predstavimo torej celovito ogrodje in nabor algoritmov, ki omogočijo gručenje naprav, ki nimajo označene podatkovne množice. Poizkusi, ki jih izvedemo v magistrski, kažejo na dejstvo, da predlagani algoritmi dajejo rezultate, ki so primerljivi s tistimi, ki jih dosegajo uveljavljene metode gručenja v ZU-ju. V primerjavi z obstoječimi metodami pa razviti algoritmi omogočijo gručenje naprav, ki niso sodelovale med učenjem zaradi pomanjkanja označenih podatkov oz. zaradi omejenih računskih sposobnosti.

• Keywords: zvezno učenje, globoko učenje