Intelligent vision-based online inspection system of screw-fastening operations in light-gauge steel frame manufacturing

Offsite construction shifts construction work to the factory environment, enabling automation of traditionally manual operations. While this approach has proven to be advantageous for the construction industry in Canada, changes in the process of manufacturing panelized walls have brought new challenges and opportunities. With the aim of providing alternatives to current manual quality control activities, an automated vision-based online inspection system for screw-fastening operations in light-gauge steel frame manufacturing is proposed in this paper. Targeting two common defects in the framing process, squareness and the quality of the screw-fastening operation itself, real-time quality assessment of the frame is achieved. The proposed system uses novel machine vision algorithms to generate the inspection data: a squareness estimation algorithm is developed, based on edge detection and the Hough transform, that can determine the squareness of a steel connection with a margin of error less than two degrees, whereas a R-CNN is designed based on a pre-trained ResNet-50 for detection and classification of screw-fastening operations, achieving an overall accuracy of 91.67%. All the inspection results are made accessible through a Python-based interface in order to direct operators towards either identifying potential quality issues or confirming the quality of the product leaving the station. The inspection system is validated over a real frame in a semi-automated machine environment. In future work, the inspection system will be deployed in continuous production setups in order to further test its accuracy and applicability.