Drone control using gestures : master's thesis

Cikač, Jaka (author), Skočaj, Danijel (mentor), Fraundorfer, Friedrich (comentor)

Quadcopters are becoming more popular and integrated into modern society. From high resolution video recording to autonomous navigation at high speed, quadcopters even shine as an everyday toy. We are now familiar with controlling quadcopters via our mobile phones. In this work we set out to develop a quadcopter gesture control system. We aspired to develop a system that can be used on a low-cost quadcopter equipped with a simple RGB camera and a powerful embedded computer. We also assembled such a quadcopter. The system is split into three modules - action detection with optical flow, human pose estimation with convolutional neural networks and gesture classification with relational features computed on the human pose. The integrated system is developed with the help of OpenCV and meta operating system ROS. For the purpose of development and evaluation we also assembled our own dataset called DS2017, in which 640 gestures are performed by 20 people. We show that action detection can detect actions sufficiently well, the human pose estimation works very well at high speed and gesture classification achieves high accuracy.

• Language: English
• Keywords: quadcopter, gesture recognition, human pose estimation, optical flow, human-robot interaction, convolutional neural networks, UAV, computer science, computer and information science, master's degree
• Work type: Master's thesis
• Typology: 2.09 - Master's Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Publisher: J. Cikač
• Year: 2017
• Number of pages: 113 str.
• PID: 20.500.12556/RUL-96744
• COBISS.SI-ID: 1537613507
• Publication date in RUL: 13.10.2017

Secondary language

• Language: Slovenian
• Title: Upravljanje kvadrokopterja z gestami

Abstract

Kvadrokopterji postajajo vse bolj priljubljeni in integrirani v današnjo družbo. Zmožni so visoko resolucijskega snemanja in avtonomnega navigiranja pri vrtoglavih hitrostih, navdušijo pa tudi kot vsakodnevna igrača. Kvadrokopterje je potrebno tudi nadzirati, za kar večinoma uporabljamo mobilne telefone. V tem delu smo razvili sistem za nadzor kvadrokopterja z gestami. Naš cilj je bil razviti sistem, ki bi se lahko izvajal na nizkocenovnem kvadrokopterju, ki je opremljen le z barvno kamero in zmogljivim vgrajenim računalnikom. Tak kvadrokopter smo tudi sestavili. Sistem je razdeljen v tri module - detekcija akcije z optičnim tokom, ocena človeške poze s konvolucijskimi nevronskimi mrežami ter klasifikacija geste z relacijskimi značilkami osnovanimi na človeški pozi. Integrirani sistem za nadzor kvadrokopterja z gestami smo implementirali s pomočjo knjižnice OpenCV in meta operacijskega sistema ROS. V namen razvoja in evalvacije sistema smo sestavili svojo bazo slik DS2017, v kateri je skupno 640 gest, ki jih je izvedlo 20 ljudi. V evalvaciji pokažemo, da sistem doseže zadovoljivo točnost pri detekciji akcij ter da hitro in natančno detektira človeško pozo in odlično klasificira detektirane geste.

• Keywords: kvadrokopter, razpoznavanje gest, ocena človeške poze, optični tok, interakcija človek-robot, konvolucijske nevronske mreže, avtonomni letalnik, računalništvo, računalništvo in informatika, magisteriji