Transfer-learning enabled micro-expression recognition using dense connections and mixed attention

Gan, Chenquan (author), Xiao, Junhao (author), Zhu, Qingyi (author), Jain, Deepak Kumar (author), Štruc, Vitomir (author)

Micro-expression recognition (MER) is a challenging computer vision problem, where the limited amount of available training data and insufficient intensity of the facial expressions are among the main issues adversely affecting the performance of existing recognition models. To address these challenges, this paper explores a transfer–learning enabled MER model using a densely connected feature extraction module with mixed attention. Unlike previous works that utilize transfer learning to facilitate MER and extract local facialexpression information, our model relies on pretraining with three diverse macro-expression datasets and, as a result, can: ▫$(i)$▫ overcome the problem of insufficient sample size and limited training data availability, ▫$(ii)$▫ leverage (related) domain-specific information from multiple datasets with diverse characteristics, and ▫$(iii)$▫ improve the model adaptability to complex scenes. Furthermore, to enhance the intensity of the microexpressions and improve the discriminability of the extracted features, the Euler video magnification (EVM) method is adopted in the preprocessing stage and then used jointly with a densely connected feature extraction module and a mixed attention mechanism to derive expressive feature representations for the classification procedure. The proposed feature extraction mechanism not only guarantees the integrity of the extracted features but also efficiently captures local texture cues by aggregating the most salient information from the generated feature maps, which is key for the MER task. The experimental results on multiple datasets demonstrate the robustness and effectiveness of our model compared to the state-of-the-art.

• Language: English
• Keywords: computer vision, deep learning, expression recognition, microexpressions
• Work type: Article
• Typology: 1.01 - Original Scientific Article
• Organization: FE - Faculty of Electrical Engineering
• Publication status: Published
• Publication version: Version of Record
• Year: 2024
• Number of pages: 14 str.
• Numbering: Vol. 305, art. 112640
• PID: 20.500.12556/RUL-164607
• UDC: 004.93
• ISSN on article: 1872-7409
• DOI: 10.1016/j.knosys.2024.112640
• COBISS.SI-ID: 213732355
• Publication date in RUL: 05.11.2024

Record is a part of a journal

• Title: Knowledge-based systems
• Publisher: Elsevier BV
• ISSN: 1872-7409
• COBISS.SI-ID: 152275459

Licences

• License: CC BY 4.0, Creative Commons Attribution 4.0 International
• Link: http://creativecommons.org/licenses/by/4.0/
• Description: This is the standard Creative Commons license that gives others maximum freedom to do what they want with the work as long as they credit the author.

Secondary language

• Language: Slovenian
• Keywords: računalniški vid, globoko učenje, razpoznavanje mimike, mikro izrazi

Projects

• Funder: Other - Other funder or multiple funders
• Project number: AB24010317
• Name: Guangxi Science and Technology Project

• Funder: ARIS - Slovenian Research and Innovation Agency
• Project number: P2-0250
• Name: Metrologija in biometrični sistemi