Wearable Structured Mental-Sensing-Graph Measurement

Daili Yang, Bin Gao*, W. L. Woo, Houlai Wen, Yihong Zhao, Zhao Gao

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

3 Downloads (Pure)

Abstract

The emotional assessment under internet of things (IoT) architecture can support researchers to establish the relationships between human social and physiological signals and emotions. In this paper, a wearable emotion sensing system is developed under narrow band internet of things (NB-IoT) wireless communication technology. The wearable sensing device integrates social linked sensors including voice, activity, and heart rate. Using this system, a dating experiment is set up to investigate multimodal factors of male’s attractiveness perception. In particular, the multimodal data are fused in a graph structure, and this further leads to a graph convolutional neural networks model for emotion evaluation and a mental-sensing-graph intelligent interpreter. Different types of mental-sensing-graphs are fused during the training stage, and the model achieves a verification accuracy of 0.93. The intrinsic relationships among the multimodal data have been captured by the subgraphs which have star-shaped structures, and the center of the subgraphs are mostly audio node. The obtained results show that the attractiveness perception of the male participants in dating is more aligned to language communication. The results also reveal that when the male participants date highly attractive women during the experiments, a significant correlation is observed between the multimodal features and the attractiveness perception levels.

Original languageEnglish
Article number2528112
Pages (from-to)1-12
Number of pages12
JournalIEEE Transactions on Instrumentation and Measurement
Volume72
Early online date26 Dec 2022
DOIs
Publication statusPublished - 5 Oct 2023

Fingerprint

Dive into the research topics of 'Wearable Structured Mental-Sensing-Graph Measurement'. Together they form a unique fingerprint.

Cite this