Kinect based interactive music application for disabled children

This work is part of a project for imitation based turn-taking and interaction games for children with communication impairments. The paper focuses on a music based interaction game mainly for children with autism. Autism Spectrum Disorder (ASD) involves communication impairments, limited social interaction, and limited imagination. Music can facilitate social interaction between the child and teacher/parent. In this work, the child is expected to imitate the predefined drumming actions in order, and a Kinect camera based system is used to recognize the signs and convert the actions to drum beats. The system simulates an e-drum which maps 6 different arm actions to different drum beats. A visual interface uses the input from the Kinect based system and gives visual and audio feedback to the participants based on the performance. In the pilot studies with adult participants, positive feedback about the system interface is taken, as well as the high success rate in the recognition of the actions and transfering these actions to music by the Kinect based system.