Movement sonification expectancy model: leveraging musical expectancy theory to create movement-altering sonifications

Joseph Newbold*, Nicolas E. Gold, Nadia Bianchi-Berthouze

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)
7 Downloads (Pure)

Abstract

When designing movement sonifications, their effect on people’s movement must be considered. Recent work has shown how real-time sonification can be designed to alter the way people move. However, the mechanisms through which these sonifications alter people’s expectations of their movement is not well explained. This is especially important when considering musical sonifications, to which people bring their own associations and musical expectation, and which can, in turn, alter their perception of the sonification. This paper presents a Movement Expectation Sonification Model, based on theories of motor-feedback and expectation, to explore how musical sonification can impact the way people perceive their movement. Secondly, we present a study that validates the predictions of this model by exploring how harmonic stability within sonification interacts with contextual cues in the environment to impact movement behaviour and perceptions. We show how musical expectancy can be built to either reward or encourage movement, and how such an effect is mediated through the presence of additional cues. This model offers a way for sonification designers to create movement sonifications that not only inform movement but can be used to encourage progress and reward successes.

Original languageEnglish
Pages (from-to)153-166
Number of pages14
JournalJournal on Multimodal User Interfaces
Volume14
Issue number2
Early online date30 Mar 2020
DOIs
Publication statusPublished - 1 Jun 2020
Externally publishedYes

Fingerprint

Dive into the research topics of 'Movement sonification expectancy model: leveraging musical expectancy theory to create movement-altering sonifications'. Together they form a unique fingerprint.

Cite this