四元组标靶辅助下的RGB-D室内点云配准

面向室内弱纹理三维重建需求，本文以RGB-D摄影测量技术获取室内点云为基础，提出了四元组标靶辅助的点云配准方法。该方法首先通过阈值筛选大曲率点，自动识别邻接点云中的辅助标靶，然后采用随机采样一致性表达方法，拟合标靶参数及其中心坐标，并根据拟合参数匹配同名标靶中心，通过刚性转换完成邻接点云粗配准。在此基础上，迭代估算邻接点云间的重叠区域，优化点云间的配准参数，从而实现点云精配准。利用Kinect相机获取两类室内场景各12站点云对本文方法进行测试，试验结果表明，配准后的多站点云间距最大均方根误差优于一个采样间隔，证明了该方法在弱纹理室内点云配准中的可靠性。     

RGB-D SLAM在室内高精度三维测图中的应用

高精度三维测图是室内三维制图的重要支撑,基于三维激光雷达扫描技术的三维测图成本高,需要提前布置标靶,在室内复杂环境中易导致数据不完整;基于图像序列的三维重建建模时间长,易受多种因素影响。针对以上问题,本文将RGB-D SLAM技术应用于室内高精度三维测图中。通过将深度相机与SLAM技术相结合,计算相机位姿并恢复三维空间信息,获取室内三维点云模型,并以目标物实际量测为基准评价密集点云精度。试验结果表明,该方法可快速获取精度较高的三维点云模型,成本低且效率高,能够较好地满足应用需求。     

低成本大视场深度相机阵列系统

在测距传感器不断轻量化、小型化以及室内外地图一体化导航应用的驱动下,三维（3D）室内移动测量成为当今研究和应用的热点,在室内建模、室内定位等新兴领域中的应用越来越广泛。3D室内移动测量系统通常配备激光扫描仪、全景相机、惯性测量单元（inertial measurement unit,IMU）系统和里程计等传感器,虽能实现3D室内点云数据的采集,但其距离传感器-激光扫描仪价格昂贵且便携性较差。彩色深度（RGB depth,RGB-D）相机为低成本3D室内移动测量系统构建提供了新的距离成像传感器选择,但主流型号RGB-D相机视场角小,继而导致数据采集效率远低于传统激光扫描仪,难以做到点云数据的完整覆盖与稳健采集,且易造成同时定位与制图（simultaneous localization and mapping,SLAM）过程中跟踪失败。针对以上问题,构建了一种低成本室内3D移动测量系统采集设备,通过组合多台消费级RGB-D相机构成大视场RGB-D相机阵列,提出了一种阵列RGB-D相机内外参数标定方法,并通过实验检验了设计系统采集的点云数据的精度。     

Cartographic Design and the Space–Time Cube

This paper reports on the results of an empirical evaluation that aimed to define the effectiveness and efficiency of different visual variables in depicting the Space–Time Cube’s (STC) content. Existing STC applications demonstrate that the most used visual variables are size and colour hue. Less is known, however, about their usability metrics. The research sets design criteria for STC contents, such as space–time paths, based on the cartographic design theory. The visual variables colour hue, colour value, colour saturation, size and orientation have been applied in two different use case studies. Besides, to support the three-dimensional visual environment, depth cues such as shading and transparency were considered too. User tests have been executed based on real-world problems with particular attention for the visualization strategy and data complexity. The outcomes revealed the most efficient and effective visual variables to represent data of various complexities in the STC.     

Semantic enrichment of octree structured point clouds for multi‐story 3D pathfinding

3D indoor navigation in multi‐story buildings and under changing environments is still difficult to perform. 3D models of buildings are commonly not available or outdated. 3D point clouds turned out to be a very practical way to capture 3D interior spaces and provide a notion of an empty space. Therefore, pathfinding in point clouds is rapidly emerging. However, processing of raw point clouds can be very expensive, as these are semantically poor and unstructured data. In this article we present an innovative octree‐based approach for processing of 3D indoor point clouds for the purpose of multi‐story pathfinding. We semantically identify the construction elements, which are of importance for the indoor navigation of humans (i.e., floors, walls, stairs, and obstacles), and use these to delineate the available navigable space. To illustrate the usability of this approach, we applied it to real‐world data sets and computed paths considering user constraints. The structuring of the point cloud into an octree approximation improves the point cloud processing and provides a structure for the empty space of the point cloud. It is also helpful to compute paths sufficiently accurate in their consideration of the spatial complexity. The entire process is automatic and able to deal with a large number of multi‐story indoor environments.     

多像位姿估计的全景纹理映射算法

纹理映射技术作为获取具有丰富纹理信息的真彩色点云的有效手段，正以其独特的优势广泛地应用于众多行业领域。研究了一种利用三维激光扫描仪与外置数码相机联合标定解算多张影像位姿并获取全景真彩色点云的方法。其基本思想是利用摄像机与激光扫描仪固有的相对位置姿态，通过对首张影像进行标定得到其位置姿态后，利用摄像机空间旋转的几何特性，根据首张影像的位姿获取其余影像的位姿，继而完成多张影像的纹理映射，获取全景彩色点云。对比目前主流的全景影像纹理映射算法，该算法在精度与效率上均有一定提高。对多种点云数据进行纹理映射实验，结果表明，该方法能够快速准确地获取真三维全景彩色点云，为三维精细化建模提供了数据基础。     

3D change detection at street level using mobile laser scanning point clouds and terrestrial images

Automatic change detection and geo-database updating in the urban environment are difficult tasks. There has been much research on detecting changes with satellite and aerial images, but studies have rarely been performed at the street level, which is complex in its 3D geometry. Contemporary geo-databases include 3D street-level objects, which demand frequent data updating. Terrestrial images provides rich texture information for change detection, but the change detection with terrestrial images from different epochs sometimes faces problems with illumination changes, perspective distortions and unreliable 3D geometry caused by the lack of performance of automatic image matchers, while mobile laser scanning (MLS) data acquired from different epochs provides accurate 3D geometry for change detection, but is very expensive for periodical acquisition. This paper proposes a new method for change detection at street level by using combination of MLS point clouds and terrestrial images: the accurate but expensive MLS data acquired from an early epoch serves as the reference, and terrestrial images or photogrammetric images captured from an image-based mobile mapping system (MMS) at a later epoch are used to detect the geometrical changes between different epochs. The method will automatically mark the possible changes in each view, which provides a cost-efficient method for frequent data updating. The methodology is divided into several steps. In the first step, the point clouds are recorded by the MLS system and processed, with data cleaned and classified by semi-automatic means. In the second step, terrestrial images or mobile mapping images at a later epoch are taken and registered to the point cloud, and then point clouds are projected on each image by a weighted window based z-buffering method for view dependent 2D triangulation. In the next step, stereo pairs of the terrestrial images are rectified and re-projected between each other to check the geometrical consistency between point clouds and stereo images. Finally, an over-segmentation based graph cut optimization is carried out, taking into account the color, depth and class information to compute the changed area in the image space. The proposed method is invariant to light changes, robust to small co-registration errors between images and point clouds, and can be applied straightforwardly to 3D polyhedral models. This method can be used for 3D street data updating, city infrastructure management and damage monitoring in complex urban scenes.     

基于2D-3D语义传递的室内三维点云模型语义分割

针对现有三维点云模型重建对象化和结构化信息缺失的问题,提出一种基于图模型的二维图像语义到三维点云语义传递的算法。该算法利用扩展全卷积神经网络提取2D图像的室内空间布局和对象语义,基于以2D图像超像素和3D点云为结点构建融合图像间一致性和图像内一致性的图模型,实现2D语义到3D语义的传递。基于点云分类实验的结果表明,该方法能够得到精度较高的室内三维点云语义分类结果,点云分类的精度可达到73.875 2%,且分类效果较好。     

A multi-sensor approach to creating accurate virtual environments

Creating virtual environment models often requires geometric data from range sensors as well as photometric data from CCD cameras. The model must be geometrically correct, visually realistic, and small enough in size to allow real-time rendering. We present an approach based on 3D range sensor data, multiple CCD cameras, and a colour high-resolution digital still camera. The multiple CCD cameras provide images for a photogrammetric bundle adjustment with constraints. The results of the bundle adjustments are used to register the 3D images from the range sensor in one coordinate system. The images from the high-resolution still camera provide the texture for the final model. The paper describes the system, the techniques for the registration of the 3D images, the building of the efficient geometric model, and the registration and integration of the texture with a simplified geometric model.     

基于密集匹配点云的DEM生产可行性研究

无人机倾斜摄影直接生产的成果通常包括三维模型、TDOM、DSM等,然而规划设计通常不能直接利用倾斜数据输出的DEM,需要辅以人工编辑。作为倾斜摄影影像处理的过程成果,密集匹配点云未得到充分利用。其与激光雷达点云具备相似的结构,且点云密度可自由选择,在不考虑数据量的情况下,密集匹配点云的点密度可数倍于激光雷达点云。此外,密集匹配点云无需单独赋色,即具有纹理信息, 对人工目视编辑自动分类后的地面点具有一定的辅助作用。本文对比分析了同一测区的密集匹配点云与激光雷达点云,验证了密集匹配点云用于房屋建筑区及稀疏林区地面点滤波并生产DEM的可行性。     

Multi-view dense matching supported by triangular meshes

We present a new procedure to compute dense 3D point clouds from a sequential set of images. This procedure is considered as a second step of a three-step algorithm for 3D reconstruction from image sequences, whose first step consists of image orientation and the last step is shape reconstruction. We assume that the camera matrices as well as a sparse set of 3D points are available and we strive for obtaining a dense and reliable 3D point cloud. Three novel ideas are presented: (1) for sparse tracking and triangulation, the search space for correspondences is reduced to a line segment by means of known camera matrices and disparity ranges are provided by triangular meshes from the already available points; (2) triangular meshes from extended sets of points are used for dense matching, because these meshes help to reconstruct points in weakly textured areas and present a natural way to obtain subpixel accuracy; (3) two non-local optimization methods, namely, 1D dynamic programming along horizontal lines and semi-global optimization were employed for refinement of local results obtained from an arbitrary number of images. All methods were extensively tested on a benchmark data set and an infrared video sequence. Both visual and quantitative results demonstrate the effectiveness of our algorithm.     

面向室内场景点云的对象重建

针对当前逆向工程中对象提取及模型重建效率较低的问题,提出了一种面向室内场景点云的对象重建方法。首先构建直通滤波器,采用改进的RANSAC算法剔除非对象点云,然后根据欧氏聚类提取算法分割出各个对象点云,最后基于α-shape理论批量重建出对象模型。试验结果表明,该方法能够从散乱的室内场景点云中快速、自动地重建出代表真实对象的三维模型,具有较高的实用价值。     

Image classification-based ground filtering of point clouds extracted from UAV-based aerial photos

With the advent of unmanned aerial vehicles (UAVs) for mapping applications, it is possible to generate 3D dense point clouds using stereo images. This technology, however, has some disadvantages when compared to Light Detection and Ranging (LiDAR) system. Unlike LiDAR, digital cameras mounted on UAVs are incapable of viewing beneath the canopy, which leads to sparse points on the bare earth surface. In such cases, it is more challenging to remove points belonging to above-ground objects using ground filtering algorithms generated especially for LiDAR data. To tackle this problem, a methodology employing supervised image classification for filtering 3D point clouds is proposed in this study. A classified image is overlapped with the point cloud to determine the ground points to be used for digital elevation model (DEM) generation. Quantitative evaluation results showed that filtering the point cloud with this methodology has a good potential for high-resolution DEM generation.     

Camera localization for augmented reality and indoor positioning: a vision-based 3D feature database approach

ABSTRACT

The recent fast development in computer vision and mobile sensor technology such as mobile LiDAR and RGB-D cameras is pushing the boundary of the technology to suit the need of real-life applications in the fields of Augmented Reality (AR), robotics, indoor GIS and self-driving. Camera localization is often a key and enabling technology among these applications. In this paper, we developed a novel camera localization workflow based on a highly accurate 3D prior map optimized by our RGB-D SLAM method in conjunction with a deep learning routine trained using consecutive video frames labeled with high precision camera pose. Furthermore, an AR registration method tightly coupled with a game engine is proposed, which incorporates the proposed localization algorithm and aligns the real Kinetic camera with a virtual camera of the game engine to facilitate AR application development in an integrated manner. The experimental results show that the localization accuracy can achieve an average error of 35?cm based on a fine-tuned prior 3D feature database at 3?cm accuracy compared against the ground-truth 3D LiDAR map. The influence of the localization accuracy on the visual effect of AR overlay is also demonstrated and the alignment of the real and virtual camera streamlines the implementation of AR fire emergency response demo in a Virtual Geographic Environment.     

基于激光点云的3维可视化方法

介绍了利用3维激光点云与数字影像,生成云冈石窟正射影像的原理与方法。采用激光扫描与数码相机同步获取石窟、石佛的3维点云与数字影像,建立点云与数字影像映射关系模型,将影像的纹理信息赋予3维点云模型,实现点云模型真彩色3维可视化,并在此基础上生成正射影像图。研究成果对于历史遗迹、文物保护与修复具有重要意义。     

Indoor scene texturing based on single mobile phone images and 3D model fusion

Realistic texture mapping and coherent up-to-date rendering is one of the most important issues in indoor 3-D modelling. However, existing texturing approaches are usually performed manually during the modelling process, and cannot accommodate changes in indoor environments occurring after the model was created, resulting in out-dated and misleading texture rendering. In this study, a structured learning-based texture mapping method is proposed for automatic mapping a single still photo from a mobile phone onto an already-constructed indoor 3-D model. The up-to-date texture is captured using a smart phone, and the indoor structural layout is extracted by incorporating per-pixel segmentation in the FCN algorithm and the line constraints into a structured learning algorithm. This enables real-time texture mapping according to parts of the model, based on the structural layout. Furthermore, the rough camera pose is estimated by pedestrian dead reckoning (PDR) and map information to determine where to map the texture. The experimental results presented in this paper demonstrate that our approach can achieve accurate fusion of 3-D triangular meshes with 2-D single images, achieving low-cost and automatic indoor texture updating. Based on this fusion approach, users can have a better experience in virtual indoor3-D applications.     

Indoor and outdoor depth imaging of leaves with time-of-flight and stereo vision sensors: Analysis and comparison

In this article we analyze the response of Time-of-Flight (ToF) cameras (active sensors) for close range imaging under three different illumination conditions and compare the results with stereo vision (passive) sensors. ToF cameras are sensitive to ambient light and have low resolution but deliver high frame rate accurate depth data under suitable conditions. We introduce metrics for performance evaluation over a small region of interest. Based on these metrics, we analyze and compare depth imaging of leaf under indoor (room) and outdoor (shadow and sunlight) conditions by varying exposure times of the sensors. Performance of three different ToF cameras (PMD CamBoard, PMD CamCube and SwissRanger SR4000) is compared against selected stereo-correspondence algorithms (local correlation and graph cuts). PMD CamCube has better cancelation of sunlight, followed by CamBoard, while SwissRanger SR4000 performs poorly under sunlight. Stereo vision is comparatively more robust to ambient illumination and provides high resolution depth data but is constrained by texture of the object along with computational efficiency. Graph cut based stereo correspondence algorithm can better retrieve the shape of the leaves but is computationally much more expensive as compared to local correlation. Finally, we propose a method to increase the dynamic range of ToF cameras for a scene involving both shadow and sunlight exposures at the same time by taking advantage of camera flags (PMD) or confidence matrix (SwissRanger).     

Keypoint-based 4-Points Congruent Sets – Automated marker-less registration of laser scans

We propose a method to automatically register two point clouds acquired with a terrestrial laser scanner without placing any markers in the scene. What makes this task challenging are the strongly varying point densities caused by the line-of-sight measurement principle, and the huge amount of data. The first property leads to low point densities in potential overlap areas with scans taken from different viewpoints while the latter calls for highly efficient methods in terms of runtime and memory requirements.A crucial yet largely unsolved step is the initial coarse alignment of two scans without any simplifying assumptions, that is, point clouds are given in arbitrary local coordinates and no knowledge about their relative orientation is available. Once coarse alignment has been solved, scans can easily be fine-registered with standard methods like least-squares surface or Iterative Closest Point matching. In order to drastically thin out the original point clouds while retaining characteristic features, we resort to extracting 3D keypoints. Such clouds of keypoints, which can be viewed as a sparse but nevertheless discriminative representation of the original scans, are then used as input to a very efficient matching method originally developed in computer graphics, called 4-Points Congruent Sets (4PCS) algorithm. We adapt the 4PCS matching approach to better suit the characteristics of laser scans.The resulting Keypoint-based 4-Points Congruent Sets (K-4PCS) method is extensively evaluated on challenging indoor and outdoor scans. Beyond the evaluation on real terrestrial laser scans, we also perform experiments with simulated indoor scenes, paying particular attention to the sensitivity of the approach with respect to highly symmetric scenes.     

主被动多传感器组合的宽角成像系统设计与实验

按照国家航空应急测绘的需求提出了一种多传感器组合成像系统。在此系统中,将组合数码相机、视频和红外相机、激光雷达（light detection and ranging,LiDAR）和小型合成孔径雷达（miniature synthetic aperture radar,MiniSAR）集成到统一的时空基准,分别且同时地实现大像幅光学成像,白天和夜间视频影像获取与传输,三维激光点云获取,以及全天候的微波遥感。详细阐述了组合宽角相机在改善影像质量和作业效率方面的优势。根据上述设计,将实验系统安装在中航时无人机上进行了飞行检验,给出了所获取的白天和夜间数据成果。     

融合RGB-D深度图像和LiDAR点云的石油管线BIM建模

建筑信息模型(BIM)已经被广泛用于提高复杂管线系统的管理效率,但已有石油管线的BIM建模,往往依靠设计图纸或现场测绘手工完成,耗时费力。为此本文提出了一种融合RGB-D深度图像和LiDAR点云数据的石油管线BIM重建方法。首先利用RGB-D图像提供的丰富语义信息和LiDAR点云精确几何信息,对深度相机采集的RGB图像进行分割,生成三维语义地图;然后通过点云粗匹配和精确匹配实现数据融合;最后给出了不同结构管线构件的BIM模型制作方法。试验结果表明,与以往的管线BIM重建方法相比,该方法更准确、高效,有助于石油企业对含有复杂管线的计转站等实施信息化管理。