Deep learning for geometric and semantic tasks in photogrammetry and remote sensing

ABSTRACT

During the last few years, artificial intelligence based on deep learning, and particularly based on convolutional neural networks, has acted as a game changer in just about all tasks related to photogrammetry and remote sensing. Results have shown partly significant improvements in many projects all across the photogrammetric processing chain from image orientation to surface reconstruction, scene classification as well as change detection, object extraction and object tracking and recognition in image sequences. This paper summarizes the foundations of deep learning for photogrammetry and remote sensing before illustrating, by way of example, different projects being carried out at the Institute of Photogrammetry and GeoInformation, Leibniz University Hannover, in this exciting and fast moving field of research and development.     

新形势下摄影测量学课程教学改革探讨

随着视频卫星、无人机和计算机技术的普及与发展,摄影测量技术已经从传统地势侦察与地形测绘推广应用于智慧城市建设的各个领域,多学科的交叉融合是摄影测量发展的必经之路。为了满足传统摄影测量向智能摄影测量发展要求,本文在原有摄影测量教学的基础上,引入倾斜摄影测量与计算机视觉方面的相关知识,优化和完善摄影测量教学内容,丰富教学实践。本文教学改革探讨旨在扩展摄影测量教学前言知识,满足现代社会对创新型和实践型摄影测量人才的需求,为摄影测量学科适应新时代的发展提供了理论基础。     

数字摄影测量发展的机遇与挑战

论述了数字摄影测量给学科、产业带来的机遇、变革与挑战,指出数字摄影测量的发展已超出了摄影测量的范畴,它是摄影测量与计算机（计算机视觉）界共同的责任。讨论了数字摄影测量与数字地球的关系。     

Modelling and representation issues in automated feature extraction from aerial and satellite images

New digital systems for the processing of photogrammetric and remote sensing images have led to new approaches to information extraction for mapping and Geographic Information System (GIS) applications, with the expectation that data can become more readily available at a lower cost and with greater currency. Demands for mapping and GIS data are increasing as well for environmental assessment and monitoring. Hence, researchers from the fields of photogrammetry and remote sensing, as well as computer vision and artificial intelligence, are bringing together their particular skills for automating these tasks of information extraction. The paper will review some of the approaches used in knowledge representation and modelling for machine vision, and give examples of their applications in research for image understanding of aerial and satellite imagery.     

INDUSTRIAL PHOTOGRAMMETRY: NEW DEVELOPMENTS AND RECENT APPLICATIONS

Over the past decade, photogrammetry has been increasingly applied as a precise three dimensional measuring tool in industrial and engineering works. Analytical photogrammetry is now routinely employed in tasks of measurement as diverse as machine tool inspection and fixture checking, structural deformation monitoring, the provision of control databases to guide industrial robots and the measurement of structures in earth orbit. Traditional photogrammetric techniques and instrumentation are usually inappropriate for industrial work and the development of complete, fully integrated close range photogrammetric systems has recently gained considerable impetus. This paper details some of the significant new developments in industrial photogrammetry, with particular reference to technological advances in the Simultaneous Triangulation and Resection System (STARS) designed by Geodetic Services, Inc. for industrial photograininetry. Practical aspects of industrial measurement are also presented in a review of some of the varying tasks that GSI has carried out in recent years. Emphasis is placed on demonstrating the flexibility, accuracy, reliability and economy of photogrammetry, as well as the progress made in automating this measurement technique.     

The specific character of limit errors in close range photogrammetry

Close-range photogrammetry is to determine the shape and size of the object, instead of it's absolute position. Therefore, at first, any translation and rotation of the photogrammetric model of the object caused by whole geodesic, photographic and photogrammetric procedures in close-range photogrammetry could not be considered. However, it is necessary to analyze all the reasons which cause the deformations of the shape and size and to present their corresponding theories and equations. This situation, of course, is very different from the conventional topophotogrammetry. In this paper some specific characters of limit errors in close-range photogrammetry are presented in detail, including limit errors for calibration of interior elements for close-range cameras, the limit errors of relative and absolute orientations in close-range cameras, the limit errors of relative and absolute orientations in close-range photogrammetric procedures, and the limit errors of control works in close-range photogrammetry. A theoretical equation of calibration accuracy for close-range camerais given. Relating to the three examples in this paper, their theoretical accuracy requirement of interior elements of camera change in the scope of ±(0.005–0.350) mm. This discussion permits us to reduce accuracy requirement in calibration for an object with small relief, but the camera platform is located in violent vibration environment. Another theoretical equation of relative RMS of base lines (mS/S) and the equation RMS of start direction are also presented. It is proved that themS/S could be equal to the relative RMS ofmΔX/ΔX. It is also proved that the permitting RMS of start direction is much bigger than the traditionally used one. Some useful equations of limit errors in close-range photogrammetry are presented as well. Suggestions mentioned above are perhaps beneficial for increasing efficiency, for reducing production cost.     

Photogrammetry Education for Multidisciplinary Geomatics in China

IntroductionThe word photogrammetry comes from Greekwords photos ( meaning “light”) , gramma( meaning that which is drawn or written) andmetron ( meaning“to measure”) . It originallysignifies measuring graphically by means oflight[1].The development o…     

Photogrammetry education for multidisciplinary geomatics in China

After briefly reviews the history of photogrammetry education in China, the development of undergraduate and graduate program, and the corresponding curricula design are analyzed by use of the data from Wuhan University in which the photogrammetry is awarded as the state-level key discipline. The academic educational program of photogrammetry in universities has trained students to perform tasks in all fields of the photogrammetric profession. In recent years, the nature of photogrammetry is changing and multidisciplinary geomatics are developing very rapidly, the educational program of photogrammetry has also changed in new concepts and structures to adapt very new technologies and the extension of the field. Finally, the prospect of photogrammetry education for the requirements of multidisciplinary geomatics is proposed. The growing interest in fast and accurate 3D spatial data collection (such as city modeling and digital earth) results in the increasing need of photogrammetry as principal tool, photogrammetric courses are therefore requested to be up-to-date and to become one kind of the fundamental professional courses for university geomatics and remote sensing degree programs.     

开放型摄影测量实验基地的设计与建立

摄影测量技术的学习和掌握离不开实验。文中提出建立开放型摄影测量实验基地模式,该基地应该具有如下功能:航空摄影,航测控制测量与调绘,数码相机检校,航测内业处理与测图。在该基地上,学生可以根据摄影测量的学习内容,选择相关实验项目,理论联系实际地学习掌握摄影测量基本理论和方法;学生也可以在基地上自主设计、模拟摄影测量生产过程,训练摄影测量技能,并进行摄影测量试验研究。该实验基地的设计理念和建立模式,将为高校现有摄影测量课程的教学和实验提供一种创新形式。     

浅谈VirtuoZo AAT自动空中三角测量的几点体会

随着信息技术和计算机技术的迅速发展,数字摄影测量已取代传统的模拟摄影测量和解析摄影测量,使摄影测量进入数字化时代。VirtuoZo全数字摄影测量系统在测绘工作中的广泛使用,节省了大量的人力,提高了工作效率,减少大量作业时间,提高产品精度。笔者总结了实际作业中常见的问题及解决方法,以供大家参考。     

THE SPECIFIC CHARACTER OF LIMIT ERRORS IN CLOSE RANGE PHOTOGRAMMETRY

1　ＩｎｔｒｏｄｕｃｔｉｏｎＴｈｅａｐｐｌｉｃａｔｉｏｎｏｆｃｌｏｓｅ＿ｒａｎｇｅｐｈｏｔｏｇｒａｍｍｅｔｒｙｉｓｔｏｄｅｔｅｒｍｉｎｅｔｈｅｆｏｒｍａｎｄｓｉｚｅ ,ｒａｔｈｅｒｔｈａｎｔｈｅａｂ ｓｏｌｕｔｅｐｏｓｉｔｉｏｎ ,ｏｆａｎｏｂｊｅｃｔｉｎｔｈｅｆｉｅｌｄｏｆａｒｃｈｉｔｅｃ ｔｕｒｅ ,ｉｎｄｕｓｔｒｙａｎｄｂｉｏｍｅｄｉｃｉｎｅ .Ａｎｙｉｎｔｅｇｒａｌｔｒａｎｓ ｌａｔｉｏｎａｎｄｒｏｔａｔｉｏｎｃａｕｓｅｄｂｙｄｉｆｆｅｒｅｎｔｏｐｅｒａｔｉｏｎｐｒｏｃｅｓｓｉｎｃｌｏｓｅ＿ｒａｎｇ…     

Medical photogrammetric measurement: overview and prospects

Since the earliest days of photogrammetry, there have been photogrammetrists who have directed research effort towards medical measurement. Although the specific motivation for these studies has not always been disclosed, it is probably because of the various benefits that photogrammetry can offer to humanity as a painless and non-invasive means of providing medical practitioners with spatial measurement relating to the human body. The intention of this paper is to reflect on the place of the many medical developments within the photogrammetric world. The various photogrammetric applications in medicine are summarised to identify the characteristics of medical photogrammetry, and it becomes evident that medical photogrammetrists have developed a range of body measurements using widely varying photogrammetric techniques, in response to the demands for specialised spatial measurement tools for a wide variety of medical ends. This volume of research activity has made medical measurement a substantial and varied sector of non-mapping photogrammetry. It is also clear that there are numerous challenges to the design of medical photogrammetric systems. They must give paramount consideration to the human patients and their comfort; they must yield not simply spatial data about the body but rather specific medical information. At the same time, it is of concern that non-photogrammetric scientists and engineers have developed medical measurement systems based on both photogrammetric and other optical techniques. Fortunately, photogrammetrists can also contribute to medical problems through their spatial data experiences. It is concluded that the many distinctive challenges mean that medical photogrammetry has not yet been as successful at changing the world as the extensive efforts applied to it suggest, but nevertheless, medical photogrammetry deserves to be recognised as a category of close range photogrammetry which remains a valuable pursuit which should not be abandoned because of any difficulties.     

倾斜摄影测量数据轻量化处理技术研究

随着无人机技术的日益成熟,倾斜摄影测量数据可有效支撑实景三维展示、地理信息数据挖掘等,已广泛用于旅游实景地理信息数字化。本文主要针对大数据量倾斜摄影测量数据显示缓慢问题,探索数据轻量化处理技术并进行验证分析。以航飞获取的山西省平遥古城景区倾斜摄影测量数据为源数据,采用多种技术对源数据进行处理分析,并采用Cesium平台对优化前后数据源效率进行验证,经过优化的倾斜摄影测量数据可实现浏览效率大幅度提升,其中金字塔级别优化技术对倾斜摄影测量数据浏览速度提升最大,约占50%。     

从摄影测量到计算机视觉

首先回顾了摄影测量的历史,从透视几何、成像设备、摄影平台、测量法和测量工具等4个方面较系统地总结了前人的贡献。其次,简要介绍了计算机视觉的起源,并从几何角度分析了计算机视觉与摄影测量之间的紧密联系,探讨了两者在实用上的一些区别。再次,从语义方面,分析了遥感学科的发展,与机器学习和计算机视觉之间的关系,以及目前深度学习和连接主义的盛行。最后,展望了摄影测量的未来,指出与计算机视觉、人工智能等学科的进一步交叉融合是摄影测量发展的必然之路。     

数字航空摄影三维重建理论与技术发展综述

航空摄影测量作为摄影测量学最重要的分支之一,近年来得到了长足的发展。倾斜航空摄影和无人机摄影测量等多种新作业模式的出现,给传统航空摄影测量带来新的挑战的同时也催生出了诸多新的解决方案。此外,人工智能领域计算机视觉技术和深度学习技术中的新理论、新方法不断融入航空摄影测量中,推动航空摄影测量向智能化、自动化方向发展。当代航空摄影测量学已经是多种传感器融合、多种数据采集方式结合、传统摄影测量和人工智能技术交叉的产物。三维重建是航空摄影测量的核心问题之一。本文阐述了当代航空摄影三维重建技术的发展趋势和存在的问题,着重从航空影像的同名连接点自动提取与匹配、区域网平差、密集匹配和单体化建模4个方面对当前的研究现状进行了总结讨论,给出了当前国内外主流的航空影像摄影测量处理框架。     

摄影测量与遥感的现状及发展趋势

简要介绍了当前摄影测量与遥感的现状,并就摄影测量与遥感的发展趋势谈一谈个人的观点。     

The Role of a Digital Intelligent Camera in Automating Industrial Photogrammetry

In the past few years the automation of digital industrial photogrammetric systems has increased dramatically. Due to digital image processing software, coded targets and automatic matching methods, a huge number of photogrammetric measurement tasks can be fully automated. In many cases a "one button click" is enough to provide the three dimensional co-ordinates of measured points without any manual interaction, immediately after acquiring the images. The technology of intelligent cameras is a logical step towards automated photogrammetric measurements. An intelligent camera, which has an integrated computer, analyses the image immediately after it is taken. This technology provides not only a much shorter processing time for the images but also more control over the measurement process just when it is needed, during image acquisition. This takes place in the form of real time feedback.
This paper describes the role of a digital intelligent camera in the automation of an industrial photogrammetric measurement system and gives an overview of existing automation techniques in industrial photogrammetry. As an example of an intelligent camera, the performance of the new INCA digital intelligent camera, developed and manufactured by Geodetic Services, Inc. (GSI) and distributed by Leica, is described.     

侧视雷达影像距离-共面模型严密定位

本文对现有侧视雷达遥感影像的几何构像模型在摄影测量应用中的不足进行了理论分析,以外方位元素作为定向参数,依次建立了像点坐标显函数形式的距离-共面(R-Cp)方程、地心直角坐标系中星载侧视雷达影像的距离-共面方程、以及星载侧视雷达遥感影像的严密定位模型,并进行了稀少控制点条件的星载TerraSAR-X影像定位试验。理论和试验表明,本文所建立的R-Cp严密定位模型,使得雷达影像像点量测坐标作为摄影测量观测值的属性得到体现,容易吸收和利用光学遥感影像基于共线方程模型发展起来的成熟的数字摄影测量数据处理方法,有利于提高雷达遥感影像摄影测量数据处理的严密性、稳定性、可靠性及数据处理效率。     

虚拟航空摄影测量实验系统的设计与实现

随着网络教学的兴起,虚拟现实技术用于实验教学,成为强化实验室建设、改革实验教学手段的一个重要发展方向。介绍虚拟建模和开发软件,并以我院仿真模拟摄影测量与遥感实验室为原型,采用VRML软件为主,三维建模软件SkecthUP和3Ds Max为辅,阐述系统设计与实现的具体过程,构建虚拟航空摄影测量实验系统。该系统主要包括虚拟地形地貌沙盘(虚拟校园模型和虚拟栅格地形)、虚拟全数字摄影测量工作站模型、虚拟无人机和控制轨道动态模型,以及虚拟实验室场景。具有数字摄影测量工作站动态操作演示、无人飞机航空摄影测量操作演示,以及实验室参观展示等功能。研究成果无实体实验室的时空限制,有利于学生开展自主学习,有效地提高摄影测量实验教学质量,其对于虚拟实验系统的研发具有一定的借鉴意义。     

影像解译中的深度学习可解释性分析方法

深度学习的迅速发展极大地推进了多种图像解译任务的精度提升,然而深度学习网络模型的“黑箱”性质让使用者难以理解其决策机理,这不仅不利于模型结构优化和安全增强等,还会极大地增加训练调参成本。对此,本文围绕影像智能解译任务,对深度学习可解释性国内外研究进展进行了综合评述与对比分析。首先,将当前可解释性分析方法分为激活值最大化分析法、代理模型分析方法、归因分析法、扰动分析法、类激活图分析法及样例分析法等6类方法,并对方法的原理、侧重点及优缺点进行了回顾。其次,对8种衡量各类分析方法所提供解释的可靠性的评估指标进行了回顾,并梳理了当前公开可用的可解释性开源算法库。在当前开源算法库的基础上,以遥感影像智能解译任务中的可解释性分析为例,验证了当前深度学习可解释性方法对遥感影像的适用性;试验结果表明当前可解释性方法在遥感解译中还存在一定的局限性。最后,总结了现有基于自然影像的可解释性算法在遥感影像解译分析中存在的问题,并展望了设计面向遥感影像特性的可解释性分析方法的发展前景,旨在为相关研究者提供参考,推动面向遥感影像解译的可解释性方法研究,从而为深度学习技术在遥感影像解译任务中的应用提供可靠的理论支持与算法设计指导。