A behavior-based scheme using reinforcement learning for autonomous underwater vehicles

This paper presents a hybrid behavior-based scheme using reinforcement learning for high-level control of autonomous underwater vehicles (AUVs). Two main features of the presented approach are hybrid behavior coordination and semi on-line neural-Q/spl I.bar/learning (SONQL). Hybrid behavior coordination takes advantages of robustness and modularity in the competitive approach as well as efficient trajectories in the cooperative approach. SONQL, a new continuous approach of the Q/spl I.bar/learning algorithm with a multilayer neural network is used to learn behavior state/action mapping online. Experimental results show the feasibility of the presented approach for AUVs.     

Robust and adaptive path following for underactuated autonomous underwater vehicles

This paper proposes a nonlinear robust adaptive control strategy to force a six degrees of freedom underactuated underwater vehicle with only four actuators to follow a predefined path at a desired speed despite of the presence of environmental disturbances and vehicle’s unknown physical parameters. The proposed controller is designed using Lyapunov’s direct method, the popular backstepping and parameter projection techniques. The closed loop path following errors can be made arbitrarily small. Interestingly, it is shown that our developed control strategy is easily extendible to situations of practical importance such as parking and point-to-point navigation. Numerical simulations are provided to illustrate the effectiveness of the proposed methodology.     

Mosaic-based navigation for autonomous underwater vehicles

We propose an approach for vision-based navigation of underwater robots that relies on the use of video mosaics of the sea bottom as environmental representations for navigation. We present a methodology for building high-quality video mosaics of the sea bottom in a fully automatic manner, which ensures global spatial coherency. During navigation, a set of efficient visual routines are used for the fast and accurate localization of the underwater vehicle with respect to the mosaic. These visual routines were developed taking into account the operating requirements of real-time position sensing, error bounding, and computational load. A visual servoing controller, based on the vehicle's kinematics, is used to drive the vehicle along a computed trajectory, specified in the mosaic, while maintaining constant altitude. The trajectory toward a goal point is generated online to avoid undefined areas in the mosaic. We have conducted a large set of sea trials, under realistic operating conditions. This paper demonstrates that without resorting to additional sensors, visual information can be used to create environment representations of the sea bottom (mosaics) and support long runs of navigation in a robust manner.     

Power systems for autonomous underwater vehicles

In this paper, we examine the issues involved in designing battery systems and power-transfer (charging) techniques for Autonomous Underwater Vehicles (AUVs) operating within an Autonomous Ocean Sampling Network (AOSN). We focus on three different aspects of the problem, battery chemistry, pack management and in situ charging. We look at a number of choices for battery chemistry and evaluate these based on the requirements of maximizing power density and low temperature operation particular to AUVs. We look at the issues involved in combining individual cells into large battery packs and at the problems associated with battery monitoring, and the charging and discharging of packs in a typical AUV application. Finally, we present a methodology for charging an AUV battery pack in situ in support of long term deployments at remote sites     

National defense applications of autonomous underwater vehicles

During the 1982 invasion of southern Lebanon, the Israeli Air Force faced the problem of neutralizing the massive Syrian air defense network in the Bekka Valley as a step in gaining air control in the area. They accomplished this, in a matter of days, without the loss of a single aircraft, while inflicting a loss of over 80 aircraft on the Syrian Air Force. The key to this smashing tactical success was the tactic used to suppress the extensive air defense network. The Israelis used remotely piloted vehicles (RPV's) for the high-risk elements of their tactical plan. RPV's conducted in-depth reconnaissance to spot and fingerprint the search and fire control radars in the valley and on adjacent mountainsides and to locate surface-to-air missile (SAM) launch sites. When the attack started, the first waves were RPV's configured to appear as attack aircraft and draw first fire. As the engagement started, manned attack fighter-bombers used antiradiation missiles (ARM's) and other smart ordnance to destroy enemy radars and launch sites as they were engaging the RPV's. While there was some loss of the low-cost RPV's, there were no losses to the Israeli aircraft. A "hornet's nest" of weapons and radars had been effectively neutralized without loss and in a period of a few hours. Recent statements of current U.S. Naval doctrine [1] emphasize a forward area strategy of naval action in or near enemy waters. The Soviet Navy in turn has been attributed as having a "bastion" strategy [2] to operate their SLBM force in Arctic waters and heavily defend the sea approaches to these waters. In other words, we postulate that they will create a "hornet's nest" and the U.S. Navy would intend to attack it, a Naval "Bekka Valley." This article examines the potential need for unmanned systems in the execution of the forward-area strategy. Emphasis is upon medium- and long-range submersibles capable of autonomous operation.     

自治水下机器人一种新型非接触充电模式应用探讨

分析国内外无人潜水器的供电方式现状,指出能源是制约自治水下机器人(AUV)续航能力以及作业时间的一个主要因素.简要介绍非接触式感应电能传输技术的原理、优点.以及国内外的研究成果和发展趋势.针对自治水下机器人原有供电方式的局限,提出用感应电能传输技术结合可充电电池对AUV进行供电设计的方法与重点问题,还探讨了非接触充电方式在AUV以及其它领域的应用前景与技术优势.     

Localization of autonomous underwater vehicles by floating acoustic buoys: a set-membership approach

This paper addresses localization of autonomous underwater vehicles (AUVs) from acoustic time-of-flight measurements received by a field of surface floating buoys. It is assumed that measurements are corrupted by unknown-but-bounded errors, with known bounds. The localization problem is tackled in a set-membership framework and an algorithm is presented, which produces as output the set of admissible AUV positions in a three-dimensional (3-D) space. The algorithm is tailored for a shallow water situation (water depth less than 500 m), and accounts for realistic variations of the sound speed profile in sea water. The approach is validated by simulations in which uncertainty models have been obtained from field data at sea. Localization performance of the algorithm are shown comparable with those previously reported in the literature by other approaches who assume knowledge of the statistics of measurement uncertainties. Moreover, guaranteed uncertainty regions associated to nominal position estimates are provided. The proposed algorithms can be used as a viable alternative to more traditional approaches in realistic at-sea conditions.     

Multivariable self-tuning autopilots for autonomous and remotelyoperated underwater vehicles

The effectiveness of subsea intervention has been found to be dependent upon the capability of an autonomous underwater vehicle's (AUV's) or remotely operated underwater vehicle's (ROV's) auto-positioning system. However, these vessel's dynamics vary considerably with operating condition, and are strongly coupled; they are expensive and difficult to derive, theoretically or through conventional testing, making the design of conventional autopilots difficult to achieve. Multi-input-multi-output self-tuning controllers offer a possible solution. Two such schemes are presented. The first is an implicit linear quadratic online, self-tuning controller, and the other uses a robust control law based on a first-order approximation of the open-loop dynamics and online recursive identification. The controllers' performance is evaluated by examining their behavior when controlling a comprehensive nonlinear simulation of an ROV and its navigation system. An interesting offshoot of this study is the application of recursive system identification techniques to the derivation of ROV models from data gathered from the trials; the potential advantages of this method are discussed     

On-line learning control of autonomous underwater vehicles usingfeedforward neural networks

A neural-network-based learning control scheme for the motion control of autonomous underwater vehicles (AUV) is described. The scheme has a number of advantages over the classical control schemes and conventional adaptive control techniques. The dynamics of the controlled vehicle need not be fully known. The controller with the aid of a gain layer learns the dynamics and adapts fast to give the correct control action. The dynamic response and tracking performance could be accurately controlled by adjusting the network learning rate. A modified direct control scheme using multilayered neural network architecture is used in the studies with backpropagation as the learning algorithm. Results of simulation studies using nonlinear AUV dynamics are described in detail. The robustness of the control system to sudden and slow varying disturbances in the dynamics is studied and the results are presented     

Evolutionary path planning for autonomous underwater vehicles in a variable ocean

This paper proposes a genetic algorithm (GA) for path planning of an autonomous underwater vehicle in an ocean environment characterized by strong currents and enhanced space-time variability. The goal is to find a safe path that takes the vehicle from its starting location to a mission-specified destination, minimizing the energy cost. The GA includes novel genetic operators that ensure the convergence to the global minimum even in cases where the structure (in space and time) of the current field implies the existence of different local minima. The performance of these operators is discussed. The proposed algorithm is suitable for situations in which the vehicle has to operate energy-exhaustive missions.     

Hull hydrodynamic optimization of autonomous underwater vehicles operating at snorkeling depth

Traditionally autonomous underwater vehicles (AUVs) have been built with a torpedo-like shape. This common shaping is hydrodynamically suboptimal for those AUVs required to operate at snorkeling condition near the free surface. In this case, the wave resistance associated to the wavy deformation of the sea surface induced by the motion of the platform is an important component of the drag. This work has investigated the optimum hull shape of an underwater vehicle moving near the free surface. Specifically a first-order Rankine panel method has been implemented to compute the wave resistance on a body of revolution moving close to the free surface. A simulated annealing algorithm was then employed to search those set of parameters defining the hull shape that minimize the wave resistance. The optimization was constrained to keep constant the total volume of the vehicle. The total drag of scaled models of the torpedo-like and resulting optimum shapes was measured in the naval tank of the University of Trieste. Measurements showed a smaller resistance of the optimized shape in the range of the considered Froude numbers.     

Design and implementation of time efficient trajectories for autonomous underwater vehicles

This paper discusses control strategies adapted for practical implementation and efficient motion of autonomous underwater vehicles (AUVs). For AUVs we would like efficiency in both the measured time and the energy consumption, the mission dictating the weight to put on each of these cost. As a first approach to this problem, we focus in this paper on time minimization. Based on the structure of the time optimal trajectories and of the pure motions, we develop an algorithm to design time efficient trajectories corresponding to piecewise constant thrust arcs with few actuator switchings. We do that by solving a new optimization problem where the unknowns are the time period between two actuator switchings as well as the values of the constant thrust arcs. We apply a direct method to compute the solutions numerically. With our algorithm, we gain considerable computational time. Moreover, with as few as three actuator switchings, the duration of our trajectories is within 10% of the optimal trajectories. Since our control strategies have a simple structure they can be implemented on a test-bed vehicle. For the experiments displayed in this paper we use a spherical underwater vehicle which exhibits with almost no preference of direction or orientation for movement; this gives us a very controllable and versatile vehicle.     

Hypothesis grids: improving long baseline navigation for autonomous underwater vehicles

Navigation continues to fundamentally limit our ability to understand the underwater world. Long baseline navigation uses range measurements to localize a remote vehicle using acoustic time-of-flight estimates. For autonomous surveys requiring high precision navigation, current solutions do not satisfy the performance or robustness requirements. Hypothesis grids represent the survey environment capturing the spatial dependence of acoustic range measurement, providing a framework for improving navigation precision and increasing the robustness with respect to non-Gaussian range observations. Prior association probabilities quantify the measurement quality as a belief that subsequent observations will correspond to the direct-path, a multipath, or an outlier as a function of the estimated location. Such a characterization is directly applicable to Bayesian navigation techniques. The algorithm for creating the representation has three main components: Mixed-density sensor model using Gaussian and uniform probability distributions, measurement classification and multipath model identification using expectation-maximization (EM), and grid-based spatial representation. We illustrate the creation of a set of hypothesis grids, the feasibility of the approach, and the utility of the representation using survey data from the autonomous benthic explorer (ABE).     

Adaptive sliding mode control of autonomous underwater vehicles inthe dive plane

The problem of controlling an autonomous underwater vehicle (AUV) in a diving maneuver is addressed. Having a simple controller which performs satisfactorily in the presence of dynamical uncertainties calls for a design using the sliding mode approach, based on a dominant linear model and bounds on the nonlinear perturbations of the dynamics. Nonadaptive and adaptive techniques are considered, leading to the design of robust controllers that can adjust to changing dynamics and operating conditions. The problem of using the observed state in the control design is addressed, leading to a sliding mode control system based on input-output signals in terms of drive-phase command and depth measurement. Numerical simulations using a full set of nonlinear equations of motion show the effectiveness of the proposed techniques     

Modeling and simulation of autonomous underwater vehicles: design and implementation

Autonomous underwater vehicles (AUVs) have many scientific, military, and commercial applications because of their potential capabilities and significant cost-performance improvements over traditional means for performing search and survey. The development of a reliable sampling platform requires a thorough system design and many costly at-sea trials during which systems specifications can be validated. Modeling and simulation provides a cost-effective measure to carry out preliminary component, system (hardware and software), and mission testing and verification, thereby reducing the number of potential failures in at-sea trials. An accurate simulation can help engineers to find hidden errors in the AUV embedded software and gain insights into the AUV operations and dynamics. This paper reviews our research work on real-time physics-based modeling and simulation for our AUVs. The modeling component includes vehicle dynamics, environment and sensor characteristics. The simulation component consists of stand-alone versus hardware-in-the-loop (HIL) implementation, for both single as well as multiple vehicles. In particular, implementation issues with regard to multitasking system resources will be addressed. The main contribution of this paper is to present the rationale for our simulation architecture and the lessons learned.     

Multivariable sliding mode control for autonomous diving andsteering of unmanned underwater vehicles

A six-degree-of-freedom model for the maneuvering of an underwater vehicle is used and a sliding-mode autopilot is designed for the combined steering, diving, and speed control functions. In flight control applications of this kind, difficulties arise because the system to be controlled is highly nonlinear and coupled, and there is a good deal of parameter uncertainty and variation with operational conditions. The development of variable-structure control in the form of sliding modes has been shown to provide robustness that is expected to be quite remarkable for AUV autopilot design. It is shown that a multivariable sliding-mode autopilot based on state feedback, designed assuming decoupled modeling, is quite satisfactory for the combined speed, steering, and diving response of a slow AUV. The influence of speed, modeling nonlinearity, uncertainty, and disturbances, can be effectively compensated, even for complex maneuvering. Waypoint acquisition based on line-of-sight guidance is used to achieve path tracking     

Drift angle compensation-based adaptive line-of-sight path following for autonomous underwater vehicle

The aim of this study is to solve the problem of poor tracking in autonomous underwater vehicle (AUVs) that are operating based on traditional line-of-sight (LOS) method when tracking different paths in a complex marine environment. An adaptive-LOS (ALOS) guidance law with drift angle compensation is proposed, and is employed to calculate the AUV’s desired course (direction of velocity) and heading. First, an appropriate look-ahead distance is derived by the ALOS guidance law in consideration of the predefined path curvature, real-time tracking error and speed of the AUV. Subsequently, proper compensation is provided with respect to the actual drift angle. Compared with traditional LOS operation, this method flexibly adjusts to a suitable look-ahead distance while considering many related factors, providing a better path following performance. Both simulation and experimental results are presented to validate the effectiveness of this method.     

A technique for autonomous underwater vehicle route planning

If an underwater vehicle is to be completely autonomous, it must have the ability to plan paths around obstacles in order to operate safely. Many solutions to the problem of planning the path of a robot around obstacles have been proposed, but all are limited in some way. An algorithm using artificial potential fields to aid in path planning is presented. The planning consists of applying potential fields around obstacles and using these fields to select a safe path. The advantage of using potential fields is that they offer a relatively fast and effective way to solve for safe paths. A trial path is chosen and then modified under the influence of the potential field until an appropriate path is found. By considering the entire path, the problem of being trapped in a local minimum is greatly reduced, allowing the method to be used for global planning. The algorithm was tried with success on many different planning problems. The examples provided illustrate the algorithm's application to two- and three-dimensional planning problems     

A neural net controller for underwater robotic vehicles

Results of a study on the application of neural networks to the control system of underwater robotic vehicles (URVs) are presented. The robustness of the control system with respect to nonlinear dynamic behavior and parameter uncertainties is investigated by computer simulation. The results show the feasibility of using unpredictable changes in the dynamics of the vehicle and its environment     

A waypoint-tracking controller for a biomimetic autonomous underwater vehicle

This work develops a control system for the waypoint-tracking of a biomimetic autonomous underwater vehicle (BAUV). The BAUV swims forward by oscillating its body and caudal fin. It turns by bending its body and caudal fin toward the turning direction. The control algorithm uses the oscillating frequency to control the forward velocity, and applies a body-spline offset parameter to control the heading velocity. The motion of the BAUV is undulatory, so moving averages of swimming velocity and heading errors are used as feedback signals. The stability of the control system is discussed using a Lyapunov function. Finally, the effectiveness of the control algorithm is experimentally confirmed.