Strojno učenje v porazdeljenem okolju z uporabo paradigme MapReduce : magistrsko delo

ORAČ, ROMAN (author), Robnik Šikonja, Marko (mentor), Lavrač, Nada (comentor)

Implementacija algoritmov strojnega učenja v porazdeljenem okolju prinaša več prednosti, kot sta zmožnost obdelave velikih množic podatkov in linearna pospešitev izvajanja z dodatnimi računskimi enotami. V magistrski nalogi opišemo paradigmo MapReduce, ki omogoča porazdeljeno računanje na računalniški gruči, in ogrodje Disco, ki ga implementira. Predstavimo sumarno obliko, ki je pogoj za učinkovito implementacijo algoritmov strojnega učenja s paradigmo MapReduce in opišemo implementacije izbranih algoritmov. Poleg tega predstavimo nove različice porazdeljenih naključnih gozdov, ki gradijo model na podmnožicah podatkov. Implementirane algoritme ovrednotimo s primerjavo z uveljavljenimi programi strojnega učenja. Magistrsko delo zaključimo z opisom vključitve implementiranih algoritmov v platformo ClowdFlows, ki omogoča sestavljanje, izvajanje in deljenje interaktivnih delotokov podatkovnega rudarjenja. S tem omogočimo obdelavo velikih paketnih podatkov z vizualnim programiranjem.

• Language: Slovenian
• Keywords: MapReduce, porazdeljeno računanje, Disco, strojno učenje, sumarna oblika, DiscoMLL, porazdeljeni naključni gozdovi, ClowdFlows, računalništvo, računalništvo in informatika, magisteriji
• Work type: Master's thesis/paper
• Typology: 2.09 - Master's Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Publisher: R. Orač
• Year: 2014
• Number of pages: 123 str.
• PID: 20.500.12556/RUL-29969
• COBISS.SI-ID: 1536017347
• Publication date in RUL: 22.10.2014

ORAČ, ROMAN, 2014, Strojno učenje v porazdeljenem okolju z uporabo paradigme MapReduce : magistrsko delo [online]. Master’s thesis. R. Orač. [Accessed 11 April 2025]. Retrieved from: https://repozitorij.uni-lj.si/IzpisGradiva.php?lang=eng&id=29969

Secondary language

• Language: English
• Title: Machine learning algorithms in distributed environment with MapReduce paradigm

Abstract

Implementation of machine learning algorithms in a distributed environment ensures us multiple advantages, like processing of large datasets and linear speedup with additional processing units. We describe the MapReduce paradigm, which enables distributed computing, and the Disco framework, which implements it. We present the summation form, which is a condition for efficient implementation of algorithms with the MapReduce paradigm, and describe the implementations of the selected algorithms. We propose novel distributed random forest algorithms that build models on subsets of the dataset. We compare time and accuracy of the algorithms with the well recognized data analytics tools. We end our master thesis by describing the integration of the implemented algorithms into the ClowdFlows platform, which is a web platform for construction, execution and sharing of interactive workflows for data mining. With this integration, we enabled processing of big batch data with visual programming.

• Keywords: MapReduce, distributed computing, Disco, machine learning, summation form, DiscoMLL, distributed random forest, ClowdFlows, computer science, computer and information science, master's degree