基于CNN和RNN的像素级视频目标跟踪算法

影像目标跟踪定位技术是当前计算机视觉领域的研究热点,目标跟踪算法也是现阶段将视频结果用于定位的薄弱环节之一.本文分析了像素级目标跟踪存在的问题,根据深度学习在图像领域的最新研究成果与视频跟踪需求,结合最新的图像分割、卷积神经网络(CNN)、循环神经网络(RNN)和加密解码结构等方法提出了一种像素级视频目标跟踪算法.使用公开数据集实现算法并设计了定量评价指标.实验结果表明该算法具有较强的像素级视频目标跟踪定位能力.     

Crowd sensing and spatiotemporal analysis in urban open space using multi-viewpoint geotagged videos

Increasing concern for urban public safety has motivated the deployment of a large number of surveillance cameras in open spaces such as city squares, stations, and shopping malls. The efficient detection of crowd dynamics in urban open spaces using multi-viewpoint surveillance videos continues to be a fundamental problem in the field of urban security. The use of existing methods for extracting features from video images has resulted in significant progress in single-camera image space. However, surveillance videos are geotagged videos with location information, and few studies have fully exploited the spatial semantics of these videos. In this study, multi-viewpoint videos in geographic space are used to fuse object trajectories for crowd sensing and spatiotemporal analysis. The YOLOv3-DeepSORT model is used to detect a pedestrian and extract the corresponding image coordinates, combine spatial semantics (such as the positions of the pedestrian in the field of view of the camera) to build a projection transformation matrix and map the object recorded by a single camera to geographic space. Trajectories from multi-viewpoint videos are fused based on the features of location, time, and directions to generate a complete pedestrian trajectory. Then, crowd spatial pattern analysis, density estimation, and motion trend analysis are performed. Experimental results demonstrate that the proposed method can be used to identify crowd dynamics and analyze the corresponding spatiotemporal pattern in an urban open space from a global perspective, providing a means of intelligent spatiotemporal analysis of geotagged videos.     

利用天文观测图像对空间碎片目标进行自动识别与追踪

提出了一种利用天文观测手段获取的CCD图像序列对空间碎片进行自动识别和追踪的方法。该方法采用计算机图像处理、图像识别与分析和计算机视觉等相关技术,自动识别出每幅CCD图像中的空间碎片以及背景恒星等空间目标,并定量计算其有关特征;然后根据空间碎片移动较快的特点,在CCD图像序列中结合基于Snake模型的主动轮廓追踪和特征相似性比较两种方法,对其中出现的空间碎片目标进行自动识别和追踪。实验结果显示,该方法能准确地对空间碎片目标进行自动识别和追踪。     

Augmented virtual environment: fusion of real-time video and 3D models in the digital earth system

ABSTRACT

An Augmented virtual environment (AVE) is concerned with the fusion of real-time video with 3D models or scenes so as to augment the virtual environment. In this paper, a new approach to establish an AVE with a wide field of view is proposed, including real-time video projection, multiple video texture fusion and 3D visualization of moving objects. A new diagonally weighted algorithm is proposed to smooth the apparent gaps within the overlapping area between the two adjacent videos. A visualization method for the location and trajectory of a moving virtual object is proposed to display the moving object and its trajectory in the 3D virtual environment. The experimental results showed that the proposed set of algorithms are able to fuse multiple real-time videos with 3D models efficiently, and the experiment runs a 3D scene containing two million triangles and six real-time videos at around 55 frames per second on a laptop with 1GB of graphics card memory. In addition, a realistic AVE with a wide field of view was created based on the Digital Earth Science Platform by fusing three videos with a complex indoor virtual scene, visualizing a moving object and drawing its trajectory in the real time.     

Tracking of facial deformations in multi-image sequences with elimination of rigid motion of the head

The paper deals with measurement of human facial deformations from synchronized image sequences taken with multiple calibrated cameras from different viewpoints. SIFT (Scale Invariant Feature Transform) keypoints are utilized as image feature points in the first place to determine spatial and temporal correspondences between images. If no temporal match is found for an image point by keypoint matching, then the tracking of the point is switched to least squares matching provided the point has one or more spatial corresponding points in the other views of the previous frame. For this purpose, a new method based on affine multi-image least squares matching is proposed where multiple spatial and temporal template images are simultaneously matched against each search image and part of the spatial template images also change during adjustment. A new method based on analyzing temporal changes in the image coordinates of the tracked points in multiple views is then presented for detecting the 3-D points which move only rigidly between consecutive frames. These points are used to eliminate the effect of rigid motion of the head and to obtain the changes in the 3-D points and in the corresponding image points due to pure deformation of the face. The methods are thoroughly tested with three multi-image sequences of four cameras including also quite large changes of facial deformations. The test results prove that the proposed affine multi-image least squares matching yields better results than another method using only fixed templates of the previous frame. The elimination of the effect of rigid motion works well and the points where the face is deforming can be correctly detected and the true deformation estimated. A method based on a novel adaptive threshold is also proposed for automated extraction and tracking of circular targets on a moving calibration object.     

顾及道路约束条件的动态目标检测与跟踪方法

智能交通是智慧城市的重要组成部分,面对复杂多变的道路背景,如何能够快速检测、跟踪道路监控影像中的动态目标,是智能交通建设的关键技术难点。本文根据道路监控视频特点,提出了采用道路约束条件与颜色特征集相结合的动态目标跟踪方法,以道路约束条件确定运动目标搜索区域,利用HSV颜色特征集进行特征匹配,然后基于ⅡR滤波背景法对背景影像进行更新及动态目标的检测,并根据道路约束条件与颜色特征相结合跟踪方法实现对动态目标的跟踪及动作预测。试验结果表明该方法可准确对运动目标进行检测与跟踪,且对慢速运动目标也具有较好的响应能力,实现了对道路动态目标的实时检测与跟踪。     

Incorporation of GPS and IP camera for people tracking

Using the global positioning system (GPS) for people tracking continues to get easier. A person can transmit his/her GPS location from the carried mobile devices. The location is usually displayed as a dot on a digital map. However, a dot on the map is insufficient to reveal the person’s actual situation, e.g., an accident being happening. If the GPS is incorporated with an IP (Internet Protocol) camera, the camera image is critical in revealing the person’s actual situation and to improve the above-mentioned insufficient information. We present an approach to facilitate such incorporation. The approach consists of three phases: locating, tracking and monitoring collision. When the GPS coordinates of a person are within the field-of-view (FOV) of a camera, the approach enters the locating phase. The GPS coordinates are transformed to specify a candidate area (CA) in the image. The update of GPS coordinates is used to filter those moving objects within the CA until only one remains. After the person is located, he is being tracked using the shortest Euclidean distance method to find the most likely object in the next image. If the person collides with other objects while being tracked, a template matching technique, the sum of absolute difference (SAD), is used to locate the person in the collision area. The tracking is done after the person leaves the FOV of the camera. In the experimental studies, the tracking of one to three persons was performed using the implemented prototype. The average locating error of the tracking phase is only 5 pixels. The highest and average tracking success rates are 95.9% and 90.6%, respectively. These results show that the proposed approach is accurate and feasible for people tracking by incorporating GPS and IP cameras.     

Integrated compression of vehicle spatio‐temporal trajectories under the road stroke network constraint

With fast growth of all kinds of trajectory datasets, how to effectively manage the trajectory data of moving objects has received a lot of attention. This study proposes a spatio‐temporal data integrated compression method of vehicle trajectories based on stroke paths coding compression under the road stroke network constraint. The road stroke network is first constructed according to the principle of continuous coherence in Gestalt psychology, and then two types of Huffman tree—a road strokes Huffman tree and a stroke paths Huffman tree—are built, based respectively on the importance function of road strokes and vehicle visiting frequency of stroke paths. After the vehicle trajectories are map matched to the spatial paths in the road network, the Huffman codes of the road strokes and stroke paths are used to compress the trajectory spatial paths. An opening window algorithm is used to simplify the trajectory temporal data depicted on a time–distance polyline by setting the maximum allowable speed difference as the threshold. Through analysis of the relative spatio‐temporal relationship between the preceding and latter feature tracking points, the spatio‐temporal data of the feature tracking points are all converted to binary codes together, accordingly achieving integrated compression of trajectory spatio‐temporal data. A series of comparative experiments between the proposed method and representative state‐of‐the‐art methods are carried out on a real massive taxi trajectory dataset from five aspects, and the experimental results indicate that our method has the highest compression ratio. Meanwhile, this method also has favorable performance in other aspects: compression and decompression time overhead, storage space overhead, and historical dataset training time overhead.     

System structure and calibration models of intelligent photogrammetron

This paper describes the structure,geometric model and geo-metric calibration of Photogrammetron I-the first type of photogrammetron which is designed to be a coherent stereo photogrammetric system in which two cameras are mounted on a physical base but driven by an intelligent agent architecture.The system calibration is divided into two parts:the in-lab calibration determines the fixed parameters in advance of system operation,and the insitu calibration keeps tracking the free parameters in real-time during the system operation.In a video surveillance set-up, prepared control points are tracked in stereo image sequences,so that the free parameters of the system can be continuously updated through iterative bundle adjustment and kalman filtering.     

System structure and calibration models of intelligent photogrammetron

This paper describes the structure, geometric model and geometric calibration of Photogrammetron I—the first type of photogrammetron which is designed to be a coherent stereo photogrammetric system in which two cameras are mounted on a physical base but driven by an intelligent agent architecture. The system calibration is divided into two parts: the in-lab calibration determines the fixed parameters in advance of system operation, and the in-situ calibration keeps tracking the free parameters in real-time during the system operation. In a video surveillance set-up, prepared control points are tracked in stereo image sequences, so that the free parameters of the system can be continuously updated through iterative bundle adjustment and Kalman filtering.     

双目序列影像3维运动物体定位跟踪

将计算机视觉中立体和运动视觉相结合,通过数字摄影测量方法,对智能视觉监控中计算机系统所获得的双序列图像通过物方“图像”分析法完成对运动物体空间位置的定位、量测及其跟踪,其中包括:摄像机检校,立体-运动双匹配约束,运动参数的求解及其云台运动控制等内容。给出采用所述方法,从真实双目序列影像中获取物体以匀速直线运动和匀加速直线运动云台运动控制的实验结果。     

Robot Tracking by Color Image

A method based on local HSV image and the shape of object to recognize object is proposed for robot tracking. After the color segment, the knowledge of the shape of objects is used to recognize objects. The robot tracking result testifies the avail-ability of the method.     

基于彩色图像的机器人视觉跟踪

采用基于局部图像的HSV闽值分割和基于形状提取相结合的方法识别物体。经过颜色分割，利用物体基本形状的先验知识，根据具体要求和需要，识别出物体的边缘，计算出物体的质心位置，并通过机器人伺服实验实现了机器人的视觉跟踪。     

监控视频中动态目标与地理空间信息的融合与可视化方法

监控视频的动态前景目标智能分析是平安城市、智慧园区等安防建设的重要基础,将监控视频与地理空间数据融合可为静态的地理数据赋予动态属性。针对传统监控视频与地理信息数据集成仅仅将视频数据投射至地理空间,造成存储难、视频内容理解难度大等问题,本文提出了前景动态目标与地理空间信息的融合模型,通过推导出的映射模型将图像空间中的动态前景目标及跟踪轨迹映射至地理空间中,达到将监控视频与地理信息有机融合的目的。根据不同的应用需求,本文设计了4种多图层融合显示模式,实现了监控视频中的动态前景目标在地理空间的可视化。     

An intelligent geospatial processing unit for image classification based on geographic vector agents (GVAs)

Spatial modeling methods usually use pixels and image objects as fundamental processing units to address real‐world objects, geo‐objects, in image space. To do this, both pixel‐based and object‐based approaches typically employ a linear two‐staged workflow of segmentation and classification. Pixel‐based methods segment a classified image to address geo‐objects in image space. In contrast, object‐based approaches classify a segmented image to identify geo‐objects from raster datasets. These methods lack the ability to simultaneously integrate the geometry and theme of geo‐objects in image space. This article explores Geographical Vector Agents (GVAs) as an automated and intelligent processing unit to directly address real‐world objects in the process of remote sensing image classification. The GVA is a distinct type of geographic automata characterized by elastic geometry, dynamic internal structure, neighborhoods and their respective rules. We test this concept by modeling a set of objects on a subset IKONOS image and LiDAR DSM datasets without the setting parameters (e.g. scale, shape information), usually applied in conventional Geographic Object‐Based Image Analysis (GEOBIA) approaches. The results show that the GVA approach achieves more than 3.5% improvement for correctness, 2% improvement for quality, although no significant improvement for completeness to GEOBIA, thus demonstrating the competitive performance of GVAs classification.     

Time‐geographic Derivation of Feasible Co‐presence Opportunities from Network‐constrained Episodic Movement Data

Certain datasets on moving objects are episodic in nature – that is, the data is characterized by time gaps during which the position of the object is unknown. In this article, a model is developed to study the sparsely sampled network‐constrained movement of several objects by calculating both potential and feasible (i.e. more likely) co‐presence opportunities over time. The approach is applied to the context of a static sensor network, where the location of an object is only registered when passing a sensor location along a road network. Feasibility is incorporated based on the deviation from the shortest path. As an illustration, the model is applied to a large Bluetooth tracking dataset gathered at a mass event. The model output consists of maps showing the temporal evolution of the distribution of feasible co‐presence opportunities of tracked visitors over the network (i.e. the number of visitors that could have been present together). We demonstrate the model's usefulness in studying the movement and distribution of a crowd over a study area with relatively few sampling locations. Finally, we discuss the results with a special emphasis on the distinction between feasible and actual presence, the need for further validation and calibration, and the performance of the implementation.     

基于MPC的半潜式无人艇导航轨迹跟踪控制研究

针对新型半潜式无人艇在导航航行过程中轨迹跟踪误差较大的问题,提出基于模型预测控制（MPC）的轨迹跟踪控制方法．并建立新型半潜式无人艇的运动模型,基于实际参数,构建MPC目标函数和系统约束条件,将半潜式无人艇轨迹跟踪问题转化为最优值问题;利用仿真软件,对控制算法进行了仿真分析. 利用卫星定位设备进行了导航轨迹跟踪试验．仿真与试验结果表明:基于MPC的轨迹跟踪控制方法提高了半潜式无人艇的导航轨迹跟踪精度,跟踪精度比原有PID控制方法提高了50％左右.      

Identification of multi-scale corresponding object-set pairs between two polygon datasets with hierarchical co-clustering

In this paper, we propose a means of finding multi-scale corresponding object-set pairs between two polygon datasets by means of hierarchical co-clustering. This method converts the intersection-ratio-based similarities of two objects from two datasets, one from each dataset, into the objects’ proximity in a geometric space using a Laplacian-graph embedding technique. In this space, the method finds hierarchical object clusters by means of agglomerative hierarchical clustering and separates each cluster into object-set pairs according to the datasets to which the objects belong. These pairs are evaluated with a matching criterion to find geometrically corresponding object-set pairs. We applied the proposed method to the segmentation result of a composite image with 6 NDVI images and a forest inventory map. Regardless of the different origins of the datasets, the proposed method can find geometrically corresponding object-set pairs which represent hierarchical distinctive forest areas.     

Three-image matching. For 3-D linear object tracking

This paper will discuss strategies for trinocular image rectification and matching for linear object tracking. It is well known that a pair of stereo images generates two epipolar images. Three overlapped images can yield six epipolar images in situations where any two are required to be rectified for the purpose of image matching. In this case, the search for feature correspondences is computationally intensive and matching complexity increases. A special epipolar image rectification for three stereo images, which simplifies the image matching process, is therefore proposed. This method generates only three rectified images, with the result that the search for matching features becomes more straightforward. With the three rectified images, a particular line-segment-based correspondence strategy is suggested. The primary characteristics of the feature correspondence strategy include application of specific epipolar geometric constraints and reference to three-ray triangulation residuals in object space.     

空基下视时序影像瞬时成像模型研究

物方空间的物体随着时间的推移进行着绝对运动,运动导致了相对位置的变化,时间序列影像记录了物方三维空间的动态变化。本文基于下视时间序列影像的动态特性,在共线方程中引入时间元素,提出了空基下视时间序列影像瞬时成像模型,描述了动态“物像”间的瞬时投影关系;针对地表不同类型动态物体,构建了“由像到物”的应用模型,实现了从像方动态特征计算地表物体特征的目的。通过仿真和真实航空下视序列影像的试验与分析,验证了序列影像瞬时成像模型能够定量计算像地动态特征。