Exploring the effects of multimodal user interfaces in autonomous vehicles

Stojmenova Pečečnik, Kristina (author), Gruden, Timotej (author), Jakus, Grega (author), Tomažič, Sašo (author), Sodnik, Jaka (author)

This short paper presents a study design created to explore how multimodal user interfaces (UIs) influence comfort, user experience, and overall well-being in fully autonomous vehicles. The study uses a motion-based driving simulator equipped with real vehicle components to provide realistic driving conditions. Participants are asked to complete two trials: a baseline trial and one with either a simple or extended multimodal UI combining auditory, tactile, and speech-based feedback. Driving scenarios include lane changes, emergency braking, V2X and V2V communication, and road surface variations. Data collection integrates physiological measures (EGG, EDA, PPG), self-assessment questionnaires on motion sickness (MSAQ) and user experience (UEQ), and performance on a reading comprehension task simulating non-driving activities. The paper provides some initial data about the participants’ demographics and their user experience with the UI, and gives an insight into the planned nest steps in terms of analysis and results application. The findings from this study will provide an insight into the potential of multimodal HMI design to enhance safety, comfort, and user acceptance in autonomous vehicles.

• Language: English
• Keywords: user interface, autonomous vehicle, multimodal interaction, non-driving related task
• Typology: 1.08 - Published Scientific Conference Contribution
• Organization: FE - Faculty of Electrical Engineering
• Publication status: Published
• Publication version: Version of Record
• Year: 2025
• Number of pages: Str. 181-186
• PID: 20.500.12556/RUL-179025
• UDC: 004.5:629.33.052
• DOI: 10.26493/978-961-293-559-7.17
• COBISS.SI-ID: 267101187
• Publication date in RUL: 03.02.2026

Record is a part of a monograph

• Title: HCI SI Koper 2025 : proceedings of the 10th Human-Computer Interaction Slovenia Conference
• Place of publishing: Koper
• Publisher: University of Primorska Press
• Year: 2025
• ISBN: 978-961-293-559-7
• COBISS.SI-ID: 266380291
• Conference organizer: University of Primorska, Faculty of Mathematics, Natural Sciences and Information Technologies, Koper, Slovenia

Licences

• License: CC BY 4.0, Creative Commons Attribution 4.0 International
• Link: http://creativecommons.org/licenses/by/4.0/
• Description: This is the standard Creative Commons license that gives others maximum freedom to do what they want with the work as long as they credit the author.

Secondary language

• Language: Slovenian
• Keywords: uporabniški vmesnik, avtonomno vozilo, multimodalna interakcija, naloge, ki niso povezane z vožnjo

Projects

• Funder: ARIS - Slovenian Research and Innovation Agency
• Project number: P2-0246
• Name: ICT4QoL - Informacijsko komunikacijske tehnologije za kakovostno življenje

• Funder: EC - European Commission
• Project number: 101147819
• Name: Federated cybeR-physical infrastructure for ODD cOntinuity
• Acronym: FRODDO