Prepoznavanje globokih ponaredkov s konvolucijskimi nevronskimi mrežami

Kronovšek, Andrej (avtor), Peer, Peter (mentor), Batagelj, Borut (komentor)

V okviru diplomskega dela smo si zadali nalogo, narediti model, ki bi učinkovito prepoznaval globoke ponaredke. Prijavili smo se na tekmovanje DeepFake Game Competition, na katerem smo prepoznavali ponaredke, ki so jih oddali naši sotekmovalci. Skozi proces razvoja modelov za detekcijo globokih ponaredkov smo preizkušali različne modele in ideje. Slike obrazov smo delili na več manjših, takšnih, ki bi zajemale del obraza, za katerega smo predpostavili, da ob ustvarjanju ponaredka postane popačen. V času poteka tekmovanja smo bili omejeni z zahtevami postavljenimi z njihove strani, naš najboljši model je bila fuzija modelov Xception in Efficient\-Net. Da so bili rezultati primerljivi, smo modele, celoten proces, tudi po končanem tekmovanju, učili na primerljivi množici podatkov. Po objavi baz, ki so jih sotekmovalci generirali in na katerih so naše modele testirali, smo ugotovili, da vsebujejo visoko mero šuma na obrazu. Usmerili smo se v izboljšanje rezultata na bazi tekmovanja in v cevovod dodali avtokodirnik, ki bi zaznal tovrsten šum. Končni model je sestavljen iz modelov Xception, EfficientNet, dodan pa je Skip-GANomaly, učen na dvakrat valjčno transformiranih vhodnih slikah. S tem modelom na omenjeni bazi dosežemo velikost območja pod ROC krivuljo enako 0,645902, kar bi nas na tekmovanju uvrstilo na 11. mesto od 28.

• Jezik: Slovenski jezik
• Ključne besede: strojno učenje, nevronske mreže, globoki ponaredki
• Vrsta gradiva: Diplomsko delo/naloga
• Tipologija: 2.11 - Diplomsko delo
• Organizacija: FRI - Fakulteta za računalništvo in informatiko
• Leto izida: 2021
• PID: 20.500.12556/RUL-130300
• COBISS.SI-ID: 77493251
• Datum objave v RUL: 13.09.2021

KRONOVŠEK, Andrej, 2021, Prepoznavanje globokih ponaredkov s konvolucijskimi nevronskimi mrežami [na spletu]. Diplomsko delo. [Dostopano 26 marec 2025]. Pridobljeno s: https://repozitorij.uni-lj.si/IzpisGradiva.php?lang=slv&id=130300

Sekundarni jezik

• Jezik: Angleški jezik
• Naslov: Deepfake detection using convolutional neural networks

Izvleček

As part of the thesis, we set ourselves the task of creating a model that would effectively identify deepfakes. We entered the DeepFake Game Competition, where we recognized the fakes submitted by our teammates. Through the process of developing models for the detection of deepfakes, we tested different models and ideas. We divided images of faces into several smaller ones. Ones that would cover the part of the face which we assumed would become distorted when the fake was created. During the competition, we were limited by the requirements set by the organizers. To make the results comparable, we taught the models on a comparable set of data through the whole process, even after the end of the competition. After the publication of the databases generated by the competitors on which our models were tested, we found that they contained a high amount of noise. We focused on improving the result on the competition dataset and added an autoencoder to the pipeline that would detect the noise. The final model consists of the Xception, EfficientNet, and Skip-GANomaly models learned from 2-D second level wavelet transformed input images. With this model, on the mentioned dataset, we achieve the area under the ROC curve equal to 0.645902, which would place us in 11th place out of 28 competitors in the competition.

• Ključne besede: machine learning, neural networks, deepfakes