Joint calibration of a multimodal sensor system for autonomous vehicles

Muhovič, Jon Natanael (author), Perš, Janez (author)

Multimodal sensor systems require precise calibration if they are to be used in the field. Due to the difficulty of obtaining the corresponding features from different modalities, the calibration of such systems is an open problem. We present a systematic approach for calibrating a set of cameras with different modalities (RGB, thermal, polarization, and dual-spectrum near infrared) with regard to a LiDAR sensor using a planar calibration target. Firstly, a method for calibrating a single camera with regard to the LiDAR sensor is proposed. The method is usable with any modality, as long as the calibration pattern is detected. A methodology for establishing a parallax-aware pixel mapping between different camera modalities is then presented. Such a mapping can then be used to transfer annotations, features, and results between highly differing camera modalities to facilitate feature extraction and deep detection and segmentation methods.

• Language: English
• Keywords: USV, calibration, multimodal system, annotation, autonomous vehicle
• Work type: Article
• Typology: 1.01 - Original Scientific Article
• Organization: FE - Faculty of Electrical Engineering
• Publication status: Published
• Publication version: Version of Record
• Year: 2023
• Number of pages: Str. 1-22
• Numbering: Vol. 23, iss. 12, [article no.] 5676
• PID: 20.500.12556/RUL-148255
• UDC: 681.586: 621.713.3
• ISSN on article: 1424-8220
• DOI: 10.3390/s23125676
• COBISS.SI-ID: 160796163
• Publication date in RUL: 07.08.2023

Record is a part of a journal

• Title: Sensors
• Shortened title: Sensors
• Publisher: MDPI
• ISSN: 1424-8220
• COBISS.SI-ID: 10176278

Licences

• License: CC BY 4.0, Creative Commons Attribution 4.0 International
• Link: http://creativecommons.org/licenses/by/4.0/
• Description: This is the standard Creative Commons license that gives others maximum freedom to do what they want with the work as long as they credit the author.

Secondary language

• Language: Slovenian
• Keywords: USV, kalibracija, večmodalni sistem, anotacije, avtonomno plovilo

Projects

• Funder: ARRS - Slovenian Research Agency
• Project number: J2-2506-2020
• Name: Adaptivne globoke metode zaznavanja za avtonomna plovila

• Funder: ARRS - Slovenian Research Agency
• Project number: P2-0095-2020
• Name: Vzporedni in porazdeljeni sistemi