TY - JOUR
T1 - Automatic Detection of Reflective Thinking in Mathematical Problem Solving based on Unconstrained Bodily Exploration
AU - Olugbade, Temitayo
AU - Newbold, Joseph
AU - Johnson, Rose
AU - Volta, Erica
AU - Alborno, Paolo
AU - Niewiadomski, Radoslaw
AU - Dillon, Max
AU - Volpe, Gualtiero
AU - Bianchi-Berthouze, Nadia
N1 - Funding information: Research funded by Horizon 2020 Framework Programme (732391)
PY - 2022/4/1
Y1 - 2022/4/1
N2 - For technology (like serious games) that aims to deliver interactive learning, it is important to address relevant mental experiences such as reflective thinking during problem solving. To facilitate research in this direction, we present the weDraw-1 Movement Dataset of body movement sensor data and reflective thinking labels for 26 children solving mathematical problems in unconstrained settings where the body (full or parts) was required to explore these problems. Further, we provide qualitative analysis of behaviours that observers used in identifying reflective thinking moments in these sessions. The body movement cues from our compilation informed features that lead to average F1 score of 0.73 for binary classification of problem-solving episodes by reflective thinking based on Long Short-Term Memory neural networks. We further obtained 0.79 average F1 score for end-to-end classification, i.e. based on raw sensor data. Finally, the algorithms resulted in 0.64 average F1 score for subsegments of these episodes as short as 4 seconds. Overall, our results show the possibility of detecting reflective thinking moments from body movement behaviours of a child exploring mathematical concepts bodily, such as within serious game play.
AB - For technology (like serious games) that aims to deliver interactive learning, it is important to address relevant mental experiences such as reflective thinking during problem solving. To facilitate research in this direction, we present the weDraw-1 Movement Dataset of body movement sensor data and reflective thinking labels for 26 children solving mathematical problems in unconstrained settings where the body (full or parts) was required to explore these problems. Further, we provide qualitative analysis of behaviours that observers used in identifying reflective thinking moments in these sessions. The body movement cues from our compilation informed features that lead to average F1 score of 0.73 for binary classification of problem-solving episodes by reflective thinking based on Long Short-Term Memory neural networks. We further obtained 0.79 average F1 score for end-to-end classification, i.e. based on raw sensor data. Finally, the algorithms resulted in 0.64 average F1 score for subsegments of these episodes as short as 4 seconds. Overall, our results show the possibility of detecting reflective thinking moments from body movement behaviours of a child exploring mathematical concepts bodily, such as within serious game play.
KW - Affect sensing and analysis
KW - Annotations
KW - Education
KW - Emotional corpora
KW - Games
KW - Neural nets
KW - Neural networks
KW - Observers
KW - Problem-solving
KW - Task analysis
UR - http://www.scopus.com/inward/record.url?scp=85085980194&partnerID=8YFLogxK
U2 - 10.1109/TAFFC.2020.2978069
DO - 10.1109/TAFFC.2020.2978069
M3 - Article
AN - SCOPUS:85085980194
VL - 13
SP - 944
EP - 957
JO - IEEE Transactions on Affective Computing
JF - IEEE Transactions on Affective Computing
SN - 1949-3045
IS - 2
ER -