Using Wearable and Structured Emotion-Sensing-Graphs for Assessment of Depressive Symptoms in Patients Undergoing Treatment

Depression, as a common mental illness, has become a significant public health issue, and the recurrence rate for patients with depression who have been treated is relatively high. In this study, a mental health monitoring system based on wearable sensing wristbands with sensors for voice, activity, and heart rate has been developed. Using this system, we perform a therapeutic monitoring study for hospitalized patients with depression and healthy controls to investigate multimodal changes before, during, and after a course of treatment. The obtained results demonstrate that there are significant changes in multimodal features such as audio short-time energy and angular velocity shape skewness with the remission of depressive symptoms. According to Mikels’ emotion wheel, a day’s data for subjects is defined as three types of emotional units and the emotional state of each emotional unit is recognized as positive or negative emotions. With this, emotion-sensing-graphs guided by Mikels’ emotion wheel theory are constructed. The analysis of emotion-sensing-graphs reveals that the same emotions are more closely linked to each other and the average degree and proportion of positive emotion nodes after a course of treatment have increased significantly. Finally, an emotion-sensing-graph graph convolutional network (ESG-GCN) model fused three types of emotion-sensing-graphs with emotion labels has been developed to assess the levels of depression, thereby monitoring the changes in depressive symptoms. Compared with classical machine learning models, the accuracy, F1 score, and recall rate of the model perform best and the model achieves a verification accuracy of 0.83.