TY - JOUR
T1 - Movement sonification expectancy model
T2 - leveraging musical expectancy theory to create movement-altering sonifications
AU - Newbold, Joseph
AU - Gold, Nicolas E.
AU - Bianchi-Berthouze, Nadia
PY - 2020/6/1
Y1 - 2020/6/1
N2 - When designing movement sonifications, their effect on people’s movement must be considered. Recent work has shown how real-time sonification can be designed to alter the way people move. However, the mechanisms through which these sonifications alter people’s expectations of their movement is not well explained. This is especially important when considering musical sonifications, to which people bring their own associations and musical expectation, and which can, in turn, alter their perception of the sonification. This paper presents a Movement Expectation Sonification Model, based on theories of motor-feedback and expectation, to explore how musical sonification can impact the way people perceive their movement. Secondly, we present a study that validates the predictions of this model by exploring how harmonic stability within sonification interacts with contextual cues in the environment to impact movement behaviour and perceptions. We show how musical expectancy can be built to either reward or encourage movement, and how such an effect is mediated through the presence of additional cues. This model offers a way for sonification designers to create movement sonifications that not only inform movement but can be used to encourage progress and reward successes.
AB - When designing movement sonifications, their effect on people’s movement must be considered. Recent work has shown how real-time sonification can be designed to alter the way people move. However, the mechanisms through which these sonifications alter people’s expectations of their movement is not well explained. This is especially important when considering musical sonifications, to which people bring their own associations and musical expectation, and which can, in turn, alter their perception of the sonification. This paper presents a Movement Expectation Sonification Model, based on theories of motor-feedback and expectation, to explore how musical sonification can impact the way people perceive their movement. Secondly, we present a study that validates the predictions of this model by exploring how harmonic stability within sonification interacts with contextual cues in the environment to impact movement behaviour and perceptions. We show how musical expectancy can be built to either reward or encourage movement, and how such an effect is mediated through the presence of additional cues. This model offers a way for sonification designers to create movement sonifications that not only inform movement but can be used to encourage progress and reward successes.
KW - Movement perception
KW - Musical expectancy
KW - Sonification design
UR - http://www.scopus.com/inward/record.url?scp=85083115066&partnerID=8YFLogxK
U2 - 10.1007/s12193-020-00322-2
DO - 10.1007/s12193-020-00322-2
M3 - Article
AN - SCOPUS:85083115066
VL - 14
SP - 153
EP - 166
JO - Journal on Multimodal User Interfaces
JF - Journal on Multimodal User Interfaces
SN - 1783-7677
IS - 2
ER -