无人机实时视频与三维地理场景融合

针对无人机视频巡查时无法获取视频要素与周围场景地理要素的空间关系等问题,本文提出了一种无人机实时视频与三维地理场景融合方法。该方法通过设置路线站点获取无人机实时巡查视频,利用高分辨率倾斜摄影模型构建三维地理场景,将视频在三维地理场景中投影,构建视频投影参数规范和无人机数据传输模型,实现无人机实时视频与三维地理场景融合展示;对无人机实景融合的应用流程进行设计,并通过具体案例进行试验验证。试验结果表明,无人机实景融合具有良好的表达效果,视频与三维地理场景融合度高且视频画面变形较小。该方法将无人机视频应用扩展至三维空间,为视频要素赋予了空间参考。     

Real-time fusion of multiple videos and 3D real scenes based on optimal viewpoint selection

3D real scenes are a digital virtual space that represents and portrays the real world in a photorealistic, three-dimensional, and sequential manner. Existing methods for constructing and updating 3D models, such as oblique photography and laser scanning, are difficult to meet the demand of perceiving the real world intuitively, dynamically, and in real time. In recent years, the method of integrating rapidly rising video data and 3D models has become increasingly popular. Compared with existing methods, it enhances the real-time perception of 3D scenes by taking advantage of the real-time character of videos and the intuitive character of 3D models. In this article, we propose a real-time fusion method of multiple videos and 3D real scenes based on optimal viewpoint selection. To begin, 3D reconstruction and video camera calibration were used to prepare the basic data for the fusion of videos and 3D model. Second, a visible-surface detection-based video space restoration method was provided, and the overlapping region between multiple videos was determined. Third, to split the overlapping region into the corresponding camera spaces, a segmentation method based on optimal viewpoint selection was given. Finally, the 2D videos were dynamically fitted to the 3D model using the dynamic texture mapping method, while accomplishing the fusing and rendering of the 3D real scene. After experimental verification, the overall effect of the multiple videos and 3D real scene fusion system implemented using the method proposed in this article is better, while the algorithm is less time-consuming and efficient in rendering.     

三维GIS场景与多路视频融合的对象快速确定法

虚实融合技术是视频增强三维虚拟地理场景的关键环节之一,对提高真实沉浸式视觉体验具有重要作用。现有基于视频投影的虚实融合方法,在三维场景动态更新时,通常需要对用户视角下新场景内所有渲染对象进行遍历,来筛选出摄像头范围内需融合的对象,导致融合过程耗时长、效率不高。本文提出一种顾及拓扑关系的多路视频与三维GIS场景融合的对象快速确定方法。首先对三维场景视锥体内视频对象集渲染对象动态建立拓扑网络模型;然后基于拓扑网络模型建立视频与渲染对象的拓扑关系,完成融合对象的快速确定及融合渲染;最后利用滕州市三维场景进行了验证。试验结果表明,本文提出的方法在视频与三维GIS场景融合时,整体耗时少、效率高,极大地降低了融合的计算代价,具有良好的可行性及稳健性。     

三维GIS支持下的实景融合系统设计与实现

视频监控在安防领域的应用日益广泛,但传统的分镜头监控模式具有画面相互孤立、缺乏关联性的应用局限。针对这一问题本文分析了实时监控视频在三维场景中融合展示的难点问题,设计了基于三维GIS的实景融合系统。系统通过视觉特征匹配计算视频投射视角,针对球机设备研究了虚实场景视频联动投射的方法,并通过服务化配置的方式进行数据的加载。试验结果表明该系统在实景融合展示上效果优异,一定程度上提升了多路实时视频集成在三维场景中的展示性能,可以广泛应用于强视频、强GIS业务的公安和智慧城市领域。     

三维虚拟地球中移动对象的时空数据组织方法

针对三维虚拟地球中实时GIS数据组织、管理与动态可视化的要求,设计了三维虚拟地球中移动对象的时空数据组织方法。提出了基于HR-tree扩展的时空索引方法,实现了对虚拟地球中移动对象的检索与查询,并基于三维虚拟地球平台对本文方法进行了实验,结果验证了本文方法的有效性和可行性。     

顾及轨迹地理方向的监控视频浓缩方法

针对视频浓缩研究未引入运动目标轨迹地理方向进行分析,所实现的可视化效果不能区分表达不同轨迹方向的运动目标的问题,提出了顾及轨迹地理方向的监控视频浓缩方法。该方法使用单应模型将运动目标轨迹由图像空间映射至地理空间,利用属性阈值聚类（quality threshold clustering,QT）算法对轨迹出入口进行聚类,实现轨迹地理方向分析。在对轨迹进行映射聚类处理的基础上,通过选取虚拟场景视点实现视频浓缩背景构建,生成轨迹类拟合中心线作为运动目标显示轨迹,并通过构建轨迹类表达模型实现运动目标显示排序,最终实现视频浓缩构建。实验结果表明,该方法能依据轨迹地理方向对运动目标进行区分表达,并能灵活调整不同轨迹类的表达策略。     

基于游戏引擎技术的地铁3维虚拟演练系统的开发与实现

根据地铁反恐实战模拟演练的需要,以地铁火灾事件处置为例,采用游戏引擎技术,开发了地铁3维虚拟演练系统,对地铁内部环境进行了逼真模拟,在3维环境下,实现了信息查询、多媒体信息关联展示、动态视频调用等功能,并可以动态操作控制3维虚拟对象,进行事件的动态处置,在基于3维的虚拟演练方面,做了有益的探索与尝试。     

实时GIS时空数据模型

为满足动态目标与传感器等实时观测数据获取、存储、管理、分析与可视化的要求,需要发展一种新型地理信息系统—实时GIS。本文根据实时GIS中各种地理要素的特点以及存贮管理要求,提出了一种面向动态地理对象与动态过程模拟的实时GIS时空数据模型,它将时空过程、地理对象、事件、事件类型、状态、观测等相关要素整合在一个时空数据模型中。基于该模型研发了新一代实时GIS,并以四种动态地理对象（包括移动对象、原位传感器对象、视频对象和过程模拟对象）的时空数据的实时接入、存储与可视化为例,验证的模型的可行性。     

利用摄像机构建数码城市模型

介绍了将多张数码影像镶嵌拼接成“虚拟单像”的方法,解决了摄像机数码影像分辨率不高的问题,实现了城市建筑物、主要装饰物和地面景物的三维模型重建,对已有地形图而无合适航空影像的地区构成了数码城市模型,取得了有益的实验结果。     

基于GPU和UNITY的嵌入式图像实时传输方法

智能化无人矿山对作业现场环境的可视化要求较高,现有的可视化方法仍存在诸多问题:数据采集方式单一,存在监控盲区;数据传输线缆布设困难且易被损坏,传输延时较高;表现形式不够全面立体,并且不能用于VR/AR、SLAM、机器人定位避障等应用场景。为了满足智能矿山建设的可视化需求,本文结合当前传感技术、矿用机器人以及5G技术的发展,探讨了从数据采集、服务器部署到接收显示的详细步骤。针对全景及深度影像这类新型三维数据,提出一种基于GPU和UNITY的嵌入式视频实时传输方法,包括实时编码、异步传输、轻量级的嵌入式流媒体系统、利用UNITY实时处理以及元数据的同步传输。借助UNITY平台,将三维可视化任务从CPU转移至GPU,仿真实验表明,最高渲染帧率为60 fps时,GPU占用率在35%以下。最后,以全景和深度传感器为例进行了测试,对数据编码、位移贴图、纹理纠正进行有效性验证,并从延迟、帧率、CPU占用率3个方面评估性能。结果表明,所提关键技术均可有效提高运行效率、减少资源占用,相比FFplay延时更低。全景影像的可视化代替了视角固定的传统监控,深度数据为智能矿山巡检机器人定位及避障提供实时数据源,传输方法整体向下兼容。不仅解决传统方法视角单一、布线困难的问题,而且考虑到了智能矿山建设过程中的新需求。     

YOLO v4框架下Multi-Patch多帧增量式交通视频目标检测

提升目标检测模型的泛化能力是计算机视觉领域的研究热点和关键难点。本文提出了一种Multi-Patch方法和多帧增量式预测策略,提升了不同场景下交通视频目标检测的稳健性,有效解决了目标尺度多变导致的视频中目标召回率低的问题。根据视频分辨率和目标尺寸,基于Multi-Patch方法自动将视频帧分割成最佳输入尺寸,使用YOLO v4神经网络并关联连续帧的上下文信息,采用增量式预测策略降低视频目标检测的漏检率,提升不同场景下视频目标的检测置信度得分和召回率。采集不同拍摄条件下的交通视频,验证该方法的有效性。试验结果表明,本文提出的目标检测方法召回率在80%以上,置信度平均得分在0.84以上。     

地形TIN中地理实体表达与LOD技术

现有的LOD技术确保了3维地表的实时绘制,但其不能解决以地理实体为结构进行组织、具有实体表达能力的高精度、大规模地形TIN的3维可视化实时响应问题,仍经常发生场景滞涩、跳跃等现象。为此,本文从面向对象的角度出发,对地形TIN按实体进行组织,建立了OTIN,设计出实体包围球、高程误差队列等结构,可以确保以实体为单位的大规模地形TIN的实时绘制。     

利用遥感数据对单株木和森林场景进行建模

针对遥感在三维可视化方面的不足,提出了用SRTM(航天飞机雷达地形测图任务)和TM(陆地资源卫星专题绘图仪)数据建立森林场景的方法。利用L系统语言和L-studio软件,通过提取的分枝规则建立了针叶树和不同叶面积指数(LAI)落叶树的几何模型,并将其作为森林场景内树木的纹理贴图。实验区的三维地形用四叉树结构建立,综合应用数据分块、动态载入、双缓存数组等措施提高大范围场景的漫游速度。从TM影像获取目视判读的遥感分类图和LAI反演图,结合控制树木的栽种位置和纹理选择,建立了单幅TM影像大小的森林场景。实验为其他遥感数据的三维显示提供了通用的建模方法。     

带有地理编码的光学视频卫星物方稳像方法

发现运动目标并获得运动目标的地理位置与运动速度,是光学视频卫星非常重要的一个高精度应用。现有光学卫星视频稳像方法虽然可以获得稳定流畅的视频数据,但视频数据中缺少几何信息,难以满足这一应用需求。针对该问题,提出一种带有地理编码的光学视频卫星物方稳像方法。该方法充分利用卫星视频帧的定向参数信息,构建基于视频帧定向模型的帧间运动模型,并通过辅帧定向参数的运动补偿,实现视频序列帧间几何一致性,再在物方空间对视频序列进行地理编码,生成带有地理编码的流畅视频数据。经对珠海一号卫星视频数据的试验结果表明,该方法可以有效消除卫星平台抖动和卫星定轨测姿等误差对卫星视频帧间几何精度的影响,获得的帧间几何精度与视频稳像精度均优于0.3像素。     

Crowd sensing and spatiotemporal analysis in urban open space using multi-viewpoint geotagged videos

Increasing concern for urban public safety has motivated the deployment of a large number of surveillance cameras in open spaces such as city squares, stations, and shopping malls. The efficient detection of crowd dynamics in urban open spaces using multi-viewpoint surveillance videos continues to be a fundamental problem in the field of urban security. The use of existing methods for extracting features from video images has resulted in significant progress in single-camera image space. However, surveillance videos are geotagged videos with location information, and few studies have fully exploited the spatial semantics of these videos. In this study, multi-viewpoint videos in geographic space are used to fuse object trajectories for crowd sensing and spatiotemporal analysis. The YOLOv3-DeepSORT model is used to detect a pedestrian and extract the corresponding image coordinates, combine spatial semantics (such as the positions of the pedestrian in the field of view of the camera) to build a projection transformation matrix and map the object recorded by a single camera to geographic space. Trajectories from multi-viewpoint videos are fused based on the features of location, time, and directions to generate a complete pedestrian trajectory. Then, crowd spatial pattern analysis, density estimation, and motion trend analysis are performed. Experimental results demonstrate that the proposed method can be used to identify crowd dynamics and analyze the corresponding spatiotemporal pattern in an urban open space from a global perspective, providing a means of intelligent spatiotemporal analysis of geotagged videos.     

交通监控视频图像语义分割及其拼接方法

视频拼接是图像拼接的外延,在场景监控、目标识别等应用中发挥着重要作用。传统视频拼接算法多要求视频间具有较大重叠区域且特征点匹配过程中只顾及图像几何特征,当处理交通监控视频时,会因不同摄像头之间重叠区域极小或主光轴之间夹角较大而导致无法拼接或图像变形较大。为此,本文提出一种交通监控视频图像语义分割及其拼接方法。首先,利用边缘角度二阶差分直方图算法自动识别多视频交汇区域的正射影像,并将其作为拼接背景图像;然后,基于全卷积神经网络对正射影像和视频图像进行语义分割,提取图像中的交通专题语义;最后,以交通专题语义作为约束进行特征点匹配,将各个交通监控视频匹配至背景正射影像,实现监控区域视频拼接。采用山东省某市实际视频数据进行试验验证,结果表明对于重叠区域较小的监控视频,本文方法可获得较好地拼接图像,同时可有效提高特征点匹配的准确度。     

网络二维地图与三维虚拟场景集成探讨

文章以网络二维地图和三维虚拟场景集成为主要线索,从常用空间数据三维模型建模方法出发,探讨了集成的总体思路。并从系统的角度讨论了网络环境下二三维集成的体系结构及集成后各模块之间的通信方法。在实例研究部分,设计并实现了一个二三维集成的原型系统。最后对网络二维地图与三维虚拟场景集成进行了展望。     

Object tracking in surveillance video based on Siamese network considering geographical spatiotemporal constraints

Identifying and tracking objects in surveillance videos is an important means of mining information during surveillance. Currently, most object-tracking methods rely only on image features, which cannot accurately express the motion of the tracked object in real geographical scenes and are easily influenced by occlusion and surrounding objects having similar features. However, tracked objects, such as pedestrians and vehicles, usually move in geographical space with fixed patterns of motion, and the motion in a short time is constrained by geographical space and time, the motion trajectory is predictable, and the range of motion is limited. Therefore, based on the SiamFC object tracking framework, this study introduces geographical spatiotemporal constraints into the tracking framework and proposes the GeoSiamFC method. The objective of this is to: (1) construct the mapping relationship between geographical space and image space to solve the problem that the pixel movement within the image after perspective imaging cannot accurately express the motion of the tracked object in a real geographical scene; (2) add candidate search areas according to the predicted trajectory location to correct the tracking errors caused by the occlusion of the object; and (3) focus on the search for the range of motion of the mapped pixel within the image space according to the limited geographical range of motion of the tracked objects in a certain time to reduce the interference of similar objects within the search area. In this study, separate experiments were conducted on a common test dataset using multiple methods to deal with challenges such as occlusion and illumination changes. In addition, a robust test dataset with noise addition and luminance adjustment based on the common test dataset was used. The results show that GeoSiamFC outperforms other object-tracking methods in the common test dataset with a precision score of 0.995 and a success score of 0.756 compared with most other object-tracking algorithms under the base condition of using only shallow networks. Moreover, GeoSiamFC maintained the highest precision score (0.970) and high success score (0.734) in the more challenging robust test dataset as well. The tracking speed of 59 frames per second far exceeds the real-time requirement of 25 FPS. Geographical spatiotemporal constraints were considered to improve tracker performance while providing real-time feedback on the motion trajectory of the target in geographical space. Thus, the proposed method is suitable for real-time tracking of the motion trajectory of a target in real geographical scenes in various surveillance videos.     

移动轨迹的尺度特征

移动轨迹是移动对象在地理空间环境中移动而留下的痕迹。移动轨迹数据是一类特殊的地理时空数据，它具有时间尺度、空间尺度和语义尺度特征。本文首先分析了轨迹数据的空间尺度特征与时间尺度特征，建立了轨迹空间尺度与时间尺度转换关系式；然后论述了轨迹的语义内涵和语义尺度特征，将轨迹语义分为移动对象语义、地理空间环境语义、采集设备语义、移动方式语义，并从地理空间环境语义的角度分析了轨迹的语义多尺度；最后探讨了轨迹语义尺度与时空尺度之间的一致性关系。