Three-dimensional point cloud object detection using scene appearance consistency among multi-view projection directions

Three-dimensional (3D) object detection in point clouds is an important technique for various high-level computer vision tasks. In this study, we propose a method for point-wise detection of regions of objects in a scene. We regard the 3D object detection problem as a series of optimal matching problems between object and scene images, which are obtained by projecting point clouds into multiple viewpoints. The main novelty of this study is treating the 3D object detection problem as the determination of optimal correspondence among image sets. Unlike the existing methods that directly employ individual correspondences between projected image pairs, the simultaneous matching of projected image sets allows the evaluation of the appearance consistency of the target object in multi-viewpoint scene images. The other novelty of the proposed method is using principal component analysis to estimate effective image-projection directions for object point clouds. By projecting object point clouds in directions orthogonal to the first principal component basis, the projected images can include plenty of point clouds information, thus providing highly discriminative features for image matching. We back-project reliable matching results retrieved from the image-set correspondence into 3D space to achieve point-wise object detection. Experiments using public datasets demonstrate the effectiveness and performance of the proposed method.