Building recognition in urban environments: A survey of state-of-the-art and future challenges

Building recognition in urban environments aims to identify different buildings in a large-scale image dataset. This identification facilitates the annotation of any visual object to a building’s façade and is an essential step in a variety of applications, such as automatic target detection in surveillance, real-time robot localization and visual navigation, architectural design, and 3D city reconstruction. Because of its importance, a significant number of building recognition systems have been proposed in recent years. Nevertheless, there is no systematic survey of building recognition in urban environments yet. To this end, we present a comprehensive review of the dominant building recognition systems by first grouping them into two categories: (i) effectiveness approaches that mainly focus on the improvement of recognition performance and (ii) efficiency methods that attempt to enhance the recognition speed. Effectiveness approaches are further categorized into two different groups: (i) feature representation-based algorithms and (ii) wide baseline matching-based methods. Efficiency methods are divided into: (i) dimensionality reduction-based methods and (ii) clustering-based algorithms. We provide analysis and discussions on each type of method and summarize their advantages and weaknesses in depth. Furthermore, we outline future research directions and associated challenges in this promising area. This survey can serve as a starting point for new researchers in building recognition to generate new ideas according to their specific requirements.