A Dual-Stream Recurrent Neural Network for Student Feedback Prediction using Kinect

Shanfeng Hu, Hindol Bhattacharya, Matangini Chattopadhyay, Nauman Aslam, Hubert P. H. Shum

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Citation (Scopus)
13 Downloads (Pure)

Abstract

Convenience internet access and ubiquitous computing have opened up new avenues for learning and teaching. They are now no longer confined to the classroom walls, but are available to anyone connected to the internet. E-learning has opened massive opportunities for learners who otherwise would have been constrained due to geographical distances, time and/or cost factors. It has revolutionized the learning methods and represents a paradigm shift from traditional learning methods. However, despite all its advantages, e-learning is not without its own shortcomings. Understanding the effectiveness of a teaching strategy through learner feedback has been a key performance measure and decision making criteria to fine tune the teaching strategy. However, traditional methods of collecting learner feedback are inadequate in a geographically distributed, virtual setup of the e-learning environment. Innovative and novel learner feedback collection mechanism is hence the need of the hour. In this work, we design and develop a deep learning based student feedback prediction system by recognizing the subtle facial motions during a student’s learning activity. This allows the system to infer the needs of the learners as if it is a real human teacher in order to provide the appropriate feedback. We propose a recurrent convolutional neural network structure to understand the color and depth streams of video taken by an RGB-D camera. Experimental results have shown that our system achieve high accuracy in estimating the feedback labels. While we demonstrate the proposed framework in an e-learning setup, it can be adapted to other applications such as in-house patient monitoring and rehabilitation training.
Original languageEnglish
Title of host publication2018 12th International Conference on Software, Knowledge, Information Management & Applications (SKIMA): Phnom Penh, Cambodia 3 – 5 December 2018.
Place of PublicationPiscataway, NJ
PublisherIEEE
Pages1-8
Number of pages8
ISBN (Electronic)9781538691410
ISBN (Print)9781538691427
DOIs
Publication statusPublished - 4 Feb 2019
Event12th International Conference on Software, Knowledge, Information Management and Applications - Institute of Technology of Cambodia, Phnom Penh, Cambodia
Duration: 3 Dec 20185 Dec 2018
http://skimanetwork.info/

Publication series

NameInternational Conference on Software, Knowledge, Information Management and Applications (SKIMA), Proceedings
PublisherIEEE
ISSN (Print)2373-082X
ISSN (Electronic)2573-3214

Conference

Conference12th International Conference on Software, Knowledge, Information Management and Applications
Abbreviated titleSKIMA 2018
Country/TerritoryCambodia
CityPhnom Penh
Period3/12/185/12/18
Internet address

Fingerprint

Dive into the research topics of 'A Dual-Stream Recurrent Neural Network for Student Feedback Prediction using Kinect'. Together they form a unique fingerprint.

Cite this