Details

Računalniška sinteza zvoka na osnovi gibanja telesa
ID PERŠIN, MAJ ŽAN (Author), ID Kos, Andrej (Mentor) More about this mentor... This link opens in a new window

.pdfPDF - Presentation file, Download (886,32 KB)
MD5: DB7E97C6C3564A180B39E2111CCB7203

Abstract
V diplomskem delu sem si zamislil program, ki združuje moji dve vzporedni karieri: elektrotehniko in ples. Osnovna ideja je izhajala iz vprašanja, kako bi bilo mogoče gibanje pretvoriti v zvok. Vedno, ko se plesalec giblje v tišini, si v glavi ustvarja glasbo. Cilj je bil razviti prototip programa, v katerem glasba sledi gibanju in ne obratno. V programu je bilo treba združiti tehnologije za zajem gibanja in sintezo zvoka. Za razvoj programa sem uporabil programski jezik Python zaradi velike razpoložljivosti knjižnic, ki omogočajo povezovanje obeh področij. Za računalniški vid in zajemanje položajev telesa sem uporabil knjižnici OpenCV in MediaPipe, knjižnici NumPy in SoundDevice pa za sintezo in predvajanje zvoka. S kamero zajeti položaji telesa se shranijo v matrike, nad katerimi lahko izvajamo matematične operacije in jih pretvorimo v zvočne valove. S položajem glave so povezane frekvence akorda, pri čemer frekvence ohranjajo medsebojna razmerja, njihova absolutna višina pa se spreminja glede na višino glave. Razdalja med rokama vpliva na zadušenost akorda, hitrost gibanja rok pa določa glasnost najvišjih frekvenc. Zvok s takim preslikovanjem gibanja je bil premalo razgiban, zato sem v program vključil tudi sprožilce, vezane na položaje rok in nog. Za to sem potreboval funkcijo, ki posnema zvok klavirja. Ko noge presežejo višino bokov, se sproži naključen nižji ton z daljšim pojemanjem zvoka iz pentatonične lestvice C-dur, ko pa roke presežejo višino glave, se sproži naključen višji ton s krajšim zvočnim pojemanjem. Ti dogodki prispevajo k večji izraznosti in glasbeni raznolikosti. Rezultati kažejo, da razvit prototip omogoča smiselno pretvorbo gibanja v zvok, ki zadovoljivo odraža značilnost gibanja. S tem je dosežen osnovni cilj diplomskega dela, kjer telo postane inštrument za ustvarjanje glasbe in predstavlja primer povezovanja računalniškega vida, sinteze zvoka in umetniškega izražanja ter odpira možnost za nadaljnji razvoj.

Language:Slovenian
Keywords:ples, glasba, sinusno valovanje, Python, računalniški vid, sinteza zvoka, zvočenje podatkov, interpolacija, amplitudna modulacija
Work type:Bachelor thesis/paper
Typology:2.11 - Undergraduate Thesis
Organization:FE - Faculty of Electrical Engineering
Year:2026
PID:20.500.12556/RUL-186343 This link opens in a new window
COBISS.SI-ID:290046979 This link opens in a new window
Publication date in RUL:31.08.2026
Views:202
Downloads:63
Metadata:XML DC-XML DC-RDF
:
Copy citation
Share:Bookmark and Share

Secondary language

Language:English
Title:Computer-aided Sound Synthesis from Body Motion
Abstract:
In this bachelor's thesis, I conceived a program that combines my two parallel careers: electrical engineering and dance. The basic idea originated from the question of how movement could be transformed into sound. Whenever a dancer moves in silence, they create their own music in their mind. The goal was to develop a prototype program in which music follows movement rather than the other way around. The program had to combine motion capture technologies and sound synthesis. For the development of the program, I used the Python programming language because of the wide availability of libraries that enable the integration of both fields. I used the OpenCV and MediaPipe libraries for computer vision and body pose estimation, and the NumPy and SoundDevice libraries for sound synthesis and playback. The body positions captured by the camera are stored as arrays of numbers, on which mathematical operations can be performed and which can be transformed into sound waves. The frequencies of a chord are linked to the position of the head, with the frequency ratios remaining constant while their absolute pitch changes according to the height of the head. The distance between the hands affects the dampening of the chord, while the speed of hand movement determines the volume of the highest notes. The sound produced by this mapping of movement was not sufficiently dynamic, so I also incorporated triggers linked to the positions of the hands and feet into the program. For this purpose, I required a function that emulates the sound of a piano. When the feet rise above hip level, a random lower note from the pentatonic C major scale with a longer sound decay is triggered, whereas when the hands rise above head level, a random higher note with a shorter sound decay is triggered. These events contribute to greater expressiveness and musical diversity. The results show that the developed prototype enables a meaningful transformation of movement into sound that satisfactorily reflects the characteristics of the movement. In this way, the primary objective of the bachelor's thesis is achieved, whereby the body becomes an instrument for creating music and serves as an example of integrating computer vision, sound synthesis, and artistic expression, while opening possibilities for further development.

Keywords:dance, music, sine wave, Python, computer vision, sound synthesis, data sonification, interpolation, amplitude modulation

Similar documents

Similar works from RUL:
Similar works from other Slovenian collections:

Back