Attention-Block Deep Learning Based Features Fusion in Wearable Social Sensor for Mental Wellbeing Evaluations

Jikun Jin, Bin Gao, Sihao Yang, Bingmei Zhao, Lizhu Luo, Wai Lok Woo

Research output: Contribution to journalArticlepeer-review

28 Citations (Scopus)
21 Downloads (Pure)


With the progressive increase of stress, anxiety and depression in working and living environment, mental health assessment becomes an important social interaction research topic. Generally, clinicians evaluate the psychology of participants through an effective psychological evaluation and questionnaires. However, these methods suffer from subjectivity and memory effects. In this paper, a new multi- sensing wearable device has been developed and applied in self-designed psychological tests. Speech under different emotions as well as behavior signals are captured and analyzed. The mental state of the participants is objectively assessed through a group of psychological questionnaires. In particular, we propose an attention-based block deep learning architecture within the device for multi-feature classification and fusion analysis. This enables the deep learning architecture to autonomously train to obtain the optimum fusion weights of different domain features. The proposed attention-based architecture has led to improving performance compared with direct connecting fusion method. Experimental studies have been carried out in order to verify the effectiveness and robustness of the proposed architecture. The obtained results have shown that the wearable multi-sensing devices equipped with the attention-based block deep learning architecture can effectively classify mental state with better performance.
Original languageEnglish
Article number9091521
Pages (from-to)89258-89268
Number of pages11
JournalIEEE Access
Early online date12 May 2020
Publication statusPublished - 22 May 2020


Dive into the research topics of 'Attention-Block Deep Learning Based Features Fusion in Wearable Social Sensor for Mental Wellbeing Evaluations'. Together they form a unique fingerprint.

Cite this