Differences between the perception of AI-generated and human-composed music in audiovisual content: a psychophysiological study : master's thesis

Fišer, Nikolaj (author), Tomc, Gregor (mentor), Andreu-Sánchez, Celia (comentor)

This research examines which are the possible differences in perceiving audiovisual content, in which the music is generated either by artificial intelligence (AI) using two different prompt strategies or composed by humans. The topic of generative AI and its impact on various aspects of humanity is currently being extensively debated, with its effects not yet fully understood, which results in numerous research gaps. In the study, we examine two levels of audiovisual perception: the conscious level and the physiological level. For the conscious level, we use self-assessment methodology, in our case a questionnaire, and for the physiological level we use instruments of measuring the bodily responses to emotional and cognitive processing, namely galvanic skin response (GSR), pupil dilation and blink rate. The fields of perception we are mostly interested in are emotional valence, arousal, assessments of congruence, familiarity and prevailing emotion in the presented stimuli. For the visual stimuli, we randomly gathered 14 short videos from the streaming service Vimeo. To choose the soundtracks with congruent properties, we performed a silent pre-test, in which the participants provided us with their assessment of the previously mentioned fields. Based on the responses, we chose the human-composed music from a database of previously existing film music with emotional ratings, and the AI-generated soundtracks based on the participants’ ratings and associations using Stable Audio. We found that AI mostly outperforms human-composed music when a more detailed and creative prompt is written. The results bring important implications for the consumption, production and development of AI-generated audiovisual content.

• Language: English
• Keywords: artificial intelligence, media perception, audiovisual content, music, creativity
• Work type: Master's thesis/paper
• Typology: 2.09 - Master's Thesis
• Organization: FDV - Faculty of Social Sciences
• Place of publishing: Ljubljana
• Publisher: N. Fišer
• Year: 2024
• Number of pages: 1 spletni vir (1 datoteka PDF (99 str.))
• PID: 20.500.12556/RUL-159424
• UDC: 78:159.91(043.2)
• COBISS.SI-ID: 202633475
• Publication date in RUL: 10.07.2024

Secondary language

• Language: Slovenian
• Title: Razlika med zaznavanjem UI generirane glasbe in človeško ustvarjene glasbe v avdiovizualnih vsebinah: psihofiziološka študija : magistrsko delo

Abstract

To je raziskava, ki preučuje morebitne razlike v dojemanjem avdiovizualne vsebine, v kateri je glasba generirana s pomočjo umetne inteligence z dvema različnima strategijama za izdelavo »prompt-a« in vsebine, v kateri je človeško komponirana glasba. Tema UI in njenega učinka na številne aspekte človštva je trenutno ena izmed osrednjih tem v znanstvenih razpravah, njeni vplivi pa niso popolnoma razumljeni, kar nudi številne vrzeli v literaturi. V tej študiji preučujemo dve ravni dojemanja avdiovizualnih dražljajev: zavedno in fiziološko raven. Pri zavedni ravni uporabljamo metodološke pristope za vrednotenje dražljajev, kot je anketni vprašalnik, pri fiziološki ravni pa uporabljamo naprave za merjenje telesnih odzivov na čustveno in kognitivno procesiranje, bolj natančno napravo za merjenje prevodnosti kože, razširjenosti zenice in hitrosti mežikanja. Področja dojemanja, ki nas zanimajo so predvsem čustvena valenca, vzburjenost, vrednotenje skladnosti, poznavanja in prevladujočega čustva v predstavljenem dražljaju. Pri vizualnih dražljajih smo izbrali 14 naključnih videov iz platforme Vimeo. Za določanje ustreznih zvočnih podlag smo izvedli predhodni eksperiment, v katerem so udeleženci podali svojo oceno videov brez zvočnih podlag. Glede na njihove odzive smo izbrali človeško narejeno glasbo iz podatkovne baze obstoječih filmskih podlag s podanimi ocenami, UI generirano glasbo pa glede na ocene udeležencev in njihove asociacije s pomočjo programa Stable Audio. Ugotovili smo, da UI prekosi človeško ustvarjeno glasbo, kadar je prompt bolj podroben in ustvarjalen. Rezultati prinašajo pomembne implikacije za razumevanje porabe, produkcije in razvoja UI generiranih avdiovizualnih vsebin.

• Keywords: umetna inteligenca, dojemanje medijev, avdiovizualna vsebina, glasba, ustvarjalnost