Shape from shading—a method of integration of sonar images and bathymetric data for ocean mapping

Shape from shading is one of the methods that derive geometric information of objects from analysis of monocular images. Application of this technique to underwater sonar images enables the conversion of imposed reflectance characteristics in sonar images to shape information, namely, slopes, about the seafloor surface. A combination of this shape information and available sparse distributed depth points results in improved dense bathymetric data.

The reconstruction of shape models of seafloor surfaces from sonar images is treated as an inverse problem and is solved by the regularization theory. Sparse gridded points are used for boundary constraints. The regularization is implemented as a relaxation procedure with a hierarchical structure of multiresolution grids.     

On observing underwater objects through a wavy water surface: A new algorithm for image correction and laboratory experiment

The possibility of correcting refraction distortions in images of the sea bottom or an underwater object observed through a wavy water surface is studied. A fast-operating algorithm of recovering the “regular” image by its random realization and spatial distribution of surface slopes at the instant of observation has been proposed and approbated. A technique of determining surface slopes by the surface image with the following correction of the underwater object image based on the data about the surface slopes has been developed under conditions of a laboratory experiment.     

A hierarchical classification system for object recognition inunderwater environments

In this paper, a hierarchical system, in which each level is composed by a neural-based classifier, is proposed to recognize objects in underwater images. The system has been designed to help an autonomous underwater vehicle in sea-bottom survey operations, like pipeline inspections. The input image is divided into square regions (macro-pixels) and a neural tree is used to classify each region into different object classes (pipeline, sea-bottom, or anodes). Each macro-pixel is then analyzed according to some geometric and environment constraints: macro-pixels with doubt classification are divided into four parts and re-classified. The process is iterated until the desired accuracy is reached. Experimental results, which have been performed on a large set of real underwater images acquired in different sea environments, demonstrate the robustness and the accuracy of the proposed system     

A voting-based approach for fast object recognition in underwateracoustic images

This paper describes a voting-based approach for the fast automatic recognition of man-made objects and related attitude estimation in underwater acoustic images generated by forward-looking sonars or acoustic cameras. In general, the continuous analysis of sequences of images is a very heavy task for human operators and this is due to the poor quality of acoustic images. Hence, algorithms able to recognize an object on the basis of a priori knowledge of the model and to estimate its attitude with reference to a global coordinate system are very useful to facilitate underwater operations like object manipulation or vehicle navigation. The proposed method is capable of recognizing objects and estimating their two-dimensional attitude by using information coming from boundary segments and their angular relations. It is based on a simple voting approach directly applied to the edge discontinuities of underwater acoustic images, whose quality is usually affected by some undesired effects such as object blurring, speckle noise, and geometrical distortions degrading the edge detection. The voting approach is robust, with respect to these effects, so that good results are obtained even with images of very poor quality. The sequences of simulated and real acoustic images are presented in order to test the validity of the proposed method in terms of average estimation error and computational load     

Underwater vehicle obstacle avoidance and path planning using amulti-beam forward looking sonar

This paper describes a new framework for segmentation of sonar images, tracking of underwater objects and motion estimation. This framework is applied to the design of an obstacle avoidance and path planning system for underwater vehicles based on a multi-beam forward looking sonar sensor. The real-time data flow (acoustic images) at the input of the system is first segmented and relevant features are extracted. We also take advantage of the real-time data stream to track the obstacles in following frames to obtain their dynamic characteristics. This allows us to optimize the preprocessing phases in segmenting only the relevant part of the images. Once the static (size and shape) as well as dynamic characteristics (velocity, acceleration,…) of the obstacles have been computed, we create a representation of the vehicle's workspace based on these features. This representation uses constructive solid geometry (CSG) to create a convex set of obstacles defining the workspace. The tracking takes also into account obstacles which are no longer in the field of view of the sonar in the path planning phase. A well-proven nonlinear search (sequential quadratic programming) is then employed, where obstacles are expressed as constraints in the search space. This approach is less affected by local minima than classical methods using potential fields. The proposed system is not only capable of obstacle avoidance but also of path planning in complex environments which include fast moving obstacles. Results obtained on real sonar data are shown and discussed. Possible applications to sonar servoing and real-time motion estimation are also discussed     

Low-bit-rate coding of underwater video using wavelet-basedcompression algorithms

Recent advances in autonomous underwater vehicle (AUV) and underwater communication technology have promoted a surge of research activity within the area of signal and information processing. A new application is proposed herein for capturing and processing underwater video onboard an untethered AUV, then transmitting it to a remote platform using acoustic telemetry. Since video communication requires a considerably larger bandwidth than that provided by an underwater acoustic channel, the data must be massively compressed prior to transmission from the AUV. Past research has shown that the low contrast and low-detailed nature of underwater imagery allows for low-bit-rate coding of the data by wavelet-based image-coding algorithms. In this work, these findings have been extended to the design of a wavelet-based hybrid video encoder which employs entropy-constrained vector quantization (ECVQ) with overlapped block-based motion compensation. The ECVQ codebooks were designed from a statistical source model which describes the distribution of high subband wavelet coefficients in both intraframe and prediction error images. Results indicate that good visual quality can be achieved for very low bit-rate coding of underwater video with our algorithm     

Recovery of underwater visibility and structure by polarization analysis

Underwater imaging is important for scientific research and technology as well as for popular activities, yet it is plagued by poor visibility conditions. In this paper, we present a computer vision approach that removes degradation effects in underwater vision. We analyze the physical effects of visibility degradation. It is shown that the main degradation effects can be associated with partial polarization of light. Then, an algorithm is presented, which inverts the image formation process for recovering good visibility in images of scenes. The algorithm is based on a couple of images taken through a polarizer at different orientations. As a by-product, a distance map of the scene is also derived. In addition, this paper analyzes the noise sensitivity of the recovery. We successfully demonstrated our approach in experiments conducted in the sea. Great improvements of scene contrast and color correction were obtained, nearly doubling the underwater visibility range.     

一种基于双通道的水下图像增强卷积神经网络

近年来各国对于海洋生物的保护意识日益强烈,用来监测海洋生物生存状态的水下机器人装备的研发是保护海洋生物资源的关键。水下相机是这类机器人在水下进行海洋生物监测时的光学感知设备。然而水下环境复杂,拍摄到的图像模糊不清,为解决水下图像模糊等问题,提出了一种基于双通道的水下图像增强卷积神经网络。在网络的编码器中采用双通道结构,其中一个通道采用了密集连接和高效通道注意力机制,提取水下图像的细节特征,另一个通道采用多尺度结构,提取原始图像的多尺度语义特征。接着,在网络中引入残差注意力模块和自适应特征融合模块,进一步优化了特征。最后将优化后的特征输入解码器重建出增强后的水下图像。试验表明：提出的网络算法在UIQM指标和Entropy指标上分别为3.005 6和7.654 7,较第二名的算法分别高出0.097 5和0.123 2。     

一种基于平稳小波变换的海底管线边缘检测方法

海底影像存在着对比度低、噪声污染严重、图像质量差等问题，采用传统算子的海底管线边缘图像中含有大量的无用和断裂边缘信息。文中将多尺度边缘检测和匹配跟踪相结合，提出利用平稳小波变换的海底管线边缘检测方法；在提取边缘的同时利用匹配跟踪手段对噪声干扰进行抑制，提高图像目标边缘检测质量。通过对海底管线和测试图像边缘检测实验表明，文中所提出的方法在抑制图像噪声的干扰、提高水下目标边缘完整性方面明显优于传统的边缘提取算子，证明该算法的有效性。     

A new global alignment approach for underwater optical mapping

Lately, underwater vehicles have become important tools for exploration, monitoring and creation of maps of the seabed. Within mapping applications, the maps obtained from optical data are becoming essential in different study areas such as biological, geological and archaeological surveys, or in detection of benthic temporal changes. However, the underwater medium is very challenging for optical sensors and does not allow the area of interest to be imaged in a single image. Therefore, image mosaicing methods are necessary. Although recent advances in detection of correspondences between images have resulted in highly effective image registration methods, global alignment methods are still needed to obtain a globally coherent mosaic. In this paper, we propose a new global alignment method which works on the mosaic frame and does not require non-linear optimisation. Additionally, a simple image rectifying method is presented to reduce the down-scaling effect which might occur when minimising errors defined in the mosaic frame. Moreover, this rectifying method can also be seen as an alternative and straightforward way of incorporating different sensor information if available. The proposed framework has been tested with underwater image sequences. The resulting method is faster than its counterparts while providing the same level of registration quality.     

侧扫声纳在海底管道悬空调查中的应用

海底管道在铺设及运营过程中,受各种因素的影响,会形成部分悬空状态。为保证油管的安全运营,研究了一种基于声学的快速检测方法:主要利用侧扫声纳获取油管附近声学影像,通过数据判读的方法分析并获取管线状态。然后使用该方法,在册镇海底管线调查中进行了应用测试。声学检测结论与水下探摸结果进行比较,测试结果表明该方法可高效、快速、准确并低成本地检测管线状态,可为海底管道运营与治理提供有力的技术支持。     

基于深度卷积神经网络的海洋牧场岩礁性生物图像分类

水下视频可直观记录和反映海洋牧场生物资源的现状和变动,目前亟待开展基于图像的海洋牧场生物识别分类方法研究,以充分发挥图像处理技术在海洋牧场生物群落监测领域的应用潜力。利用采集自我国北方烟威地区包含鱼礁、藻床和泥沙三种图像背景的水下视频,开展了图像增强、图像分类数据集的建立和3种分类模型的应用。对比了基于绿通道的色彩补偿和限制对比度的自适应直方图均衡等方法在海洋牧场水下图像增强上的效果。建立了北方海洋牧场常见岩礁生物图像分类数据集,包括花鲈(Lateolabraxjaponicus)、(Lizahaematocheilus)、许氏平鲉(Sebastes schlegelii)等鱼类11种、棘皮类3种和蟹类1种,共23 211张图像。基于飞桨深度学习框架和PaddleX全流程开发工具,选择AlexNet、MobileNetV3和ResNet50三种图像分类卷积神经网络进行迁移学习,并分别验证了其在含噪音水下图像上的鲁棒性。结果表明,三种模型在测试集的类准确率分别达到96.64%、94.75%和99.23%,其中ResNet50模型在含有高斯噪音的图像集验证具有更好的鲁棒性。综之,基于深度学习的计算机视觉技术在我国海洋牧场生物群落监测中具有较大应用潜力,可为我国海洋牧场监测和管理提供新的思路和方法。     

Hull hydrodynamic optimization of autonomous underwater vehicles operating at snorkeling depth

Traditionally autonomous underwater vehicles (AUVs) have been built with a torpedo-like shape. This common shaping is hydrodynamically suboptimal for those AUVs required to operate at snorkeling condition near the free surface. In this case, the wave resistance associated to the wavy deformation of the sea surface induced by the motion of the platform is an important component of the drag. This work has investigated the optimum hull shape of an underwater vehicle moving near the free surface. Specifically a first-order Rankine panel method has been implemented to compute the wave resistance on a body of revolution moving close to the free surface. A simulated annealing algorithm was then employed to search those set of parameters defining the hull shape that minimize the wave resistance. The optimization was constrained to keep constant the total volume of the vehicle. The total drag of scaled models of the torpedo-like and resulting optimum shapes was measured in the naval tank of the University of Trieste. Measurements showed a smaller resistance of the optimized shape in the range of the considered Froude numbers.     

Development of real-time acoustic image recognition system using by autonomous marine vehicle

Automation of complicated underwater tasks require acoustic image based object recognition. This paper presents an acoustic image based real-time object recognition system. We proposed an acoustic image predictor to estimate an object's shape in advance. Depending on the acoustic camera's position, the predictor generates optimal template for recognition. The proposed method is implemented in our autonomous marine vehicle. For real-time processing, efficient recognition strategies are addressed. The vehicle detects an object and localizes it for recognition. In the detection process, the acoustic image's specific characteristics are used as the detection cues. In the localization process, the vehicle's horizontal and vertical positioning strategies are described. Efficient template generation method to minimize computing power is addressed. This realizes real-time recognition using the vehicle. To estimate the proposed system's accuracy and reliability, a recognition test was carried out in the field. The vehicle successfully recognized two different objects with high accuracy.     

Object detection and tracking method of AUV based on acoustic vision

This paper describes a new framework for object detection and tracking of AUV including underwater acoustic data interpolation, underwater acoustic images segmentation and underwater objects tracking. This framework is applied to the design of vision-based method for AUV based on the forward looking sonar sensor. First, the real-time data flow (underwater acoustic images) is pre-processed to form the whole underwater acoustic image, and the relevant position information of objects is extracted and determined. An improved method of double threshold segmentation is proposed to resolve the problem that the threshold cannot be adjusted adaptively in the traditional method. Second, a representation of region information is created in light of the Gaussian particle filter. The weighted integration strategy combining the area and invariant moment is proposed to perfect the weight of particles and to enhance the tracking robustness. Results obtained on the real acoustic vision platform of AUV during sea trials are displayed and discussed. They show that the proposed method can detect and track the moving objects underwater online, and it is effective and robust.     

A detection model of underwater topography with a series of SAR images acquired at different time

underwater topography is one of oceanic features detected by Synthetic Aperture Radar. Underwater topography SAR imaging mechanism shows that tidal current is the important factor for underwater topography SAR imaging. Thus under the same wind field condition, SAR images for the same area acquired at different time include different information of the underwater topography. To utilize synchronously SAR images acquired at different time for the underwater topography SAR detection and improve the precision of detection, based on the detection model of underwater topography with single SAR image and the periodicity of tidal current, a detection model of underwater topography with a series of SAR images acquired at different time is developed by combing with tide and tidal current numerical simulation. To testify the feasibility of the presented model, Taiwan Shoal located at the south outlet of Taiwan Strait is selected as study area and three SAR images are used in the underwater topography detection. The detection results are compared with the field observation data of water depth carried out by R/V Dongfanghong 2, and the errors of the detection are compared with those of the single SAR image. All comparisons show that the detection model presented in the paper improves the precision of underwater topography SAR detection, and the presented model is feasible.     

Evolution history and trend of the modern Huanghe River Delta

The evolvement history of modern Huanghe River Delta tidal flat, coastline and underwater terrain were studied based on the analysis of remote sensing images and water depth data. Based on the analysis of seafloor terrain evolution on different historical stages, a formula simulating the erosion and deposition evolvement model of subaqueous Huanghe River Delta slope was proposed, and the evolvement trend of the subaqueous delta terrain was predicted. The result shows that the equilibrium transition zone is near the water depth of 12 m with seabed erosion in shallower water and accumulation in deeper water during the first 150 a after the river channel was deserted. In the meantime, the underwater slope became gentler and the coastal erosion rate became slow gradually. Then, the subaqueous delta slope changed to up concave from upper convex, and the shape of subaqueous delta disappeared. The coast type changed to silt-mud coast about 100-150 a after the river course was deserted. The erosion depth in the foot of the seawall is calculated based on the formula.     

Monitoring moored instrument motion by optimal estimation

Many oceanographic applications require the positioning of the underwater sensor at measurement times. We consider here the case of subsurface moored tomographic instruments, where the distance between source and receiver must be known within a few meters. For that purpose, a long baseline array is deployed: this system includes a navigator, attached to the mooring element and an array of three transponders set on the ocean bottom. To process the navigation data collected with such system, we have developed a method based on optimal estimation. The triangulation problem is not a basic spherical constraints one and the specificity of deep underwater positioning, related to the variability of the ocean sound speed profile are pointed out. Correcting terms are proposed and introduced into the system. Simultaneous inversion of all data, defining an overconstrained problem allows to estimate biases and errors. The algorithm is applied here to a dataset collected in the Azores-Canary basin during CAMBIOS experiment.     

面向ARV的视觉辅助水下对接方法研究

针对水下机器人作业过程中的近距离引导水下对接问题,以自治缆控水下机器人(ARV)为研究对象,为实现快速、高精度的水下对接,设计了基于反射光源识别的单目视觉辅助水下对接方法。通过在对接口布置反光带并将其作为目标图像,设计了图像处理和特征点提取的算法,经过图像特征信息的分析处理,优化算法的时间复杂度,提高了特征点提取的准确率和识别效率。最后设计了传统的引导灯方案与反光带方案的对比试验,验证算法的可行性,证明在水下环境下识别反射光源的方法极大提升了位置估计的准确率,同时在计算速度上也有显著提高,弥补了传统水下对接方法中精度不足或计算量大的缺陷,更好地满足了ARV水下对接的需求,能为潜水器实现水下自主对接提供参考依据。