A Mixed Reality Human Robot Interaction Interface Using Hand Tracking and Stereo Vision for Construction Applications

Damith Tennakoon; Mojgan Jadidi; Seyedreza Razavialavi

doi:10.35490/EC3.2024.275

A Mixed Reality Human Robot Interaction Interface Using Hand Tracking and Stereo Vision for Construction Applications

Damith Tennakoon, Mojgan Jadidi, Seyedreza Razavialavi

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

37 Downloads (Pure)

Abstract

The modern day construction site is able to complete complex tasks using highly skilled on-site workers and robust machinery. On-site workers face severe physical injuries from unforeseen accidents due to construction site dynamics. This project proposes and implements a mixed reality human robot interaction system, utilizing stereo vision systems and hand tracking algorithms, to teleoperate a robot arm within a simulation. Results demonstrate enhanced depth and situational awareness in the robot workspace versus traditional teleoperation. This research allows operators to safely perform complex construction tasks remotely with a robot arm using a head-mounted display.

Original language	English
Title of host publication	Proceedings of the 2024 European Conference on Computing in Construction
Editors	Marijana Srećković, Mohamad Kassem, Ranjith Soman, Athanasios Chassiakos
Publisher	European Council on Computing in Construction (EC3)
Pages	1015-1022
Number of pages	8
ISBN (Print)	9789083451305
DOIs	https://doi.org/10.35490/EC3.2024.275
Publication status	Published - 17 Jul 2024
Event	European Conference on Computing in Construction, EC3 2024 - Chania, Greece Duration: 14 Jul 2024 → 17 Jul 2024

Publication series

Name	Proceedings of the European Conference on Computing in Construction
Volume	2024
ISSN (Electronic)	2684-1150

Conference

Conference	European Conference on Computing in Construction, EC3 2024
Country/Territory	Greece
City	Chania
Period	14/07/24 → 17/07/24

Access to Document

10.35490/EC3.2024.275Licence: CC BY-NC-ND

EC32024_275Final published version, 2.61 MBLicence: CC BY-NC-ND

Cite this

Tennakoon, D., Jadidi, M., & Razavialavi, S. (2024). A Mixed Reality Human Robot Interaction Interface Using Hand Tracking and Stereo Vision for Construction Applications. In M. Srećković, M. Kassem, R. Soman, & A. Chassiakos (Eds.), Proceedings of the 2024 European Conference on Computing in Construction (pp. 1015-1022). (Proceedings of the European Conference on Computing in Construction; Vol. 2024). European Council on Computing in Construction (EC3). https://doi.org/10.35490/EC3.2024.275

Tennakoon, Damith ; Jadidi, Mojgan ; Razavialavi, Seyedreza. / A Mixed Reality Human Robot Interaction Interface Using Hand Tracking and Stereo Vision for Construction Applications. Proceedings of the 2024 European Conference on Computing in Construction. editor / Marijana Srećković ; Mohamad Kassem ; Ranjith Soman ; Athanasios Chassiakos. European Council on Computing in Construction (EC3), 2024. pp. 1015-1022 (Proceedings of the European Conference on Computing in Construction).

@inproceedings{d0bc125621164f1f9bcc3330bfb5fd45,

title = "A Mixed Reality Human Robot Interaction Interface Using Hand Tracking and Stereo Vision for Construction Applications",

abstract = "The modern day construction site is able to complete complex tasks using highly skilled on-site workers and robust machinery. On-site workers face severe physical injuries from unforeseen accidents due to construction site dynamics. This project proposes and implements a mixed reality human robot interaction system, utilizing stereo vision systems and hand tracking algorithms, to teleoperate a robot arm within a simulation. Results demonstrate enhanced depth and situational awareness in the robot workspace versus traditional teleoperation. This research allows operators to safely perform complex construction tasks remotely with a robot arm using a head-mounted display.",

author = "Damith Tennakoon and Mojgan Jadidi and Seyedreza Razavialavi",

year = "2024",

month = jul,

day = "17",

doi = "10.35490/EC3.2024.275",

language = "English",

isbn = "9789083451305",

series = "Proceedings of the European Conference on Computing in Construction",

publisher = "European Council on Computing in Construction (EC3)",

pages = "1015--1022",

editor = "Marijana Sre{\'c}kovi{\'c} and Mohamad Kassem and Ranjith Soman and Athanasios Chassiakos",

booktitle = "Proceedings of the 2024 European Conference on Computing in Construction",

note = "European Conference on Computing in Construction, EC3 2024 ; Conference date: 14-07-2024 Through 17-07-2024",

}

Tennakoon, D, Jadidi, M & Razavialavi, S 2024, A Mixed Reality Human Robot Interaction Interface Using Hand Tracking and Stereo Vision for Construction Applications. in M Srećković, M Kassem, R Soman & A Chassiakos (eds), Proceedings of the 2024 European Conference on Computing in Construction. Proceedings of the European Conference on Computing in Construction, vol. 2024, European Council on Computing in Construction (EC3), pp. 1015-1022, European Conference on Computing in Construction, EC3 2024, Chania, Greece, 14/07/24. https://doi.org/10.35490/EC3.2024.275

A Mixed Reality Human Robot Interaction Interface Using Hand Tracking and Stereo Vision for Construction Applications. / Tennakoon, Damith; Jadidi, Mojgan; Razavialavi, Seyedreza.
Proceedings of the 2024 European Conference on Computing in Construction. ed. / Marijana Srećković; Mohamad Kassem; Ranjith Soman; Athanasios Chassiakos. European Council on Computing in Construction (EC3), 2024. p. 1015-1022 (Proceedings of the European Conference on Computing in Construction; Vol. 2024).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Mixed Reality Human Robot Interaction Interface Using Hand Tracking and Stereo Vision for Construction Applications

AU - Tennakoon, Damith

AU - Jadidi, Mojgan

AU - Razavialavi, Seyedreza

PY - 2024/7/17

Y1 - 2024/7/17

N2 - The modern day construction site is able to complete complex tasks using highly skilled on-site workers and robust machinery. On-site workers face severe physical injuries from unforeseen accidents due to construction site dynamics. This project proposes and implements a mixed reality human robot interaction system, utilizing stereo vision systems and hand tracking algorithms, to teleoperate a robot arm within a simulation. Results demonstrate enhanced depth and situational awareness in the robot workspace versus traditional teleoperation. This research allows operators to safely perform complex construction tasks remotely with a robot arm using a head-mounted display.

AB - The modern day construction site is able to complete complex tasks using highly skilled on-site workers and robust machinery. On-site workers face severe physical injuries from unforeseen accidents due to construction site dynamics. This project proposes and implements a mixed reality human robot interaction system, utilizing stereo vision systems and hand tracking algorithms, to teleoperate a robot arm within a simulation. Results demonstrate enhanced depth and situational awareness in the robot workspace versus traditional teleoperation. This research allows operators to safely perform complex construction tasks remotely with a robot arm using a head-mounted display.

UR - https://www.scopus.com/pages/publications/85203445934

U2 - 10.35490/EC3.2024.275

DO - 10.35490/EC3.2024.275

M3 - Conference contribution

AN - SCOPUS:85203445934

SN - 9789083451305

T3 - Proceedings of the European Conference on Computing in Construction

SP - 1015

EP - 1022

BT - Proceedings of the 2024 European Conference on Computing in Construction

A2 - Srećković, Marijana

A2 - Kassem, Mohamad

A2 - Soman, Ranjith

A2 - Chassiakos, Athanasios

PB - European Council on Computing in Construction (EC3)

T2 - European Conference on Computing in Construction, EC3 2024

Y2 - 14 July 2024 through 17 July 2024

ER -

Tennakoon D, Jadidi M, Razavialavi S. A Mixed Reality Human Robot Interaction Interface Using Hand Tracking and Stereo Vision for Construction Applications. In Srećković M, Kassem M, Soman R, Chassiakos A, editors, Proceedings of the 2024 European Conference on Computing in Construction. European Council on Computing in Construction (EC3). 2024. p. 1015-1022. (Proceedings of the European Conference on Computing in Construction). doi: 10.35490/EC3.2024.275

A Mixed Reality Human Robot Interaction Interface Using Hand Tracking and Stereo Vision for Construction Applications

Abstract

Publication series

Conference

Access to Document

Other files and links

Cite this