Vodenje robota za pobiranje stebelne zelenjave na osnovi robotskega vida

Gimpelj, Jakob (author), Mihelj, Matjaž (mentor), Šlajpah, Sebastjan (comentor)

Kmetijski sektor se sooča s problemi, kot sta upadanje razpoložljive delovne sile in naraščajoče povpraševanje po pridelkih, kar povečuje potrebo po tehnoloških inovacijah, na primer po robotih, ki lahko izboljšajo pridelovalne procese. Pobiranje špargljev je eno izmed področij, kjer ročno delo zahteva veliko truda in kjer občutljiva narava ter nepredvidljiva rast špargljev otežujeta avtomatizacijo. V magistrskem delu smo se posvetili izboljšavi obstoječega mobilnega robotskega sistema za pobiranje špargljev, s poudarkom na zaznavi in lokalizaciji špargljev za bolj natančno pobiranje. Pripravili smo mobilno platformo za delo na polju in izboljšali vodenje paralelnega robota. Podrobneje smo se posvetili procesu zaznave špargljev z robotskim vidom. Prilagodili smo nosilec senzorjev, tako da je bilo poleg laserskega skenerja na mobilno platformo mogoče namestiti tudi globinski kameri. Nadgradili smo obstoječi algoritem za prepoznavanje špargljev na podlagi podatkov iz laserskega skenerja in tako omogočili bolj robustno zaznavo. Dodali smo tudi proces za prepoznavanje in sledenje špargljev na osnovi barvnih in globinskih slik iz globinske kamere z uporabo algoritma YOLO. Ta na barvnih slikah detektira in segmentira šparglje, nato pa jim med premikanjem sledi z algoritmom ByteTrack. Proces smo optimizirali za izvajanje z zadostno hitrostjo za delovanje v realnem času. Podatke iz zaznave smo uporabili za določanje višine špargljev in točke pobiranja. Rezultati dela obsegajo validacijo pobiranja špargljev na polju ter primerjavo zmožnosti zaznavanja na podlagi laserskega skenerja in globinske kamere.

• Language: Slovenian
• Keywords: pobiranje špargljev, paralelni robotski sistem, mobilni robot, robotski vid, YOLO, ROS
• Work type: Master's thesis/paper
• Organization: FE - Faculty of Electrical Engineering
• Year: 2023
• PID: 20.500.12556/RUL-152585
• COBISS.SI-ID: 180917251
• Publication date in RUL: 29.11.2023

Secondary language

• Language: English
• Title: Vision-based robot control for harvesting stem vegetables

Abstract

The agricultural sector is facing issues such as a decline in available workforce and increasing demand for crops, which heightens the need for technological innovations, for instance, robots that can improve production processes. Asparagus harvesting is one area where manual labor is highly demanding, and the delicate nature and unpredictable growth of asparagus make automation challenging. In this master's thesis, we focused on improving an existing mobile robotic system for asparagus picking, with an emphasis on the detection and localization of asparagus. We prepared the mobile platform for fieldwork and improved the guidance of its parallel robot. We focused on the process of detecting asparagus using robotic vision. We modified the sensor holder to allow for the installation of depth cameras on the mobile platform, in addition to the laser scanner. We enhanced the existing algorithm for identifying asparagus using data from the laser scanner, to enhance detection reliability. We also added a process for recognizing and tracking asparagus based on color and depth frames from the depth camera using the YOLO algorithm. The algorithm detects and segments asparagus in color frames and then tracks them using the ByteTrack algorithm as they move. We optimized this process to run with sufficient speed for real-time operation. We used the detection data to determine the height of the asparagus and the picking point. This work's results include the validation of asparagus picking in the field and a comparison of the detection capabilities of laser scanner and depth camera methods.

• Keywords: asparagus harvesting, mobile robot, delta robot, detection, YOLO, ROS