Using multi-scale and hierarchical deep convolutional features for 3D semantic classification of TLS point clouds

ABSTRACT

Point cloud classification, which provides meaningful semantic labels to the points in a point cloud, is essential for generating three-dimensional (3D) models. Its automation, however, remains challenging due to varying point densities and irregular point distributions. Adapting existing deep-learning approaches for two-dimensional (2D) image classification to point cloud classification is inefficient and results in the loss of information valuable for point cloud classification. In this article, a new approach that classifies point cloud directly in 3D is proposed. The approach uses multi-scale features generated by deep learning. It comprises three steps: (1) extract single-scale deep features using 3D convolutional neural network (CNN); (2) subsample the input point cloud at multiple scales, with the point cloud at each scale being an input to the 3D CNN, and combine deep features at multiple scales to form multi-scale and hierarchical features; and (3) retrieve the probabilities that each point belongs to the intended semantic category using a softmax regression classifier. The proposed approach was tested against two publicly available point cloud datasets to demonstrate its performance and compared to the results produced by other existing approaches. The experiment results achieved 96.89% overall accuracy on the Oakland dataset and 91.89% overall accuracy on the Europe dataset, which are the highest among the considered methods.