Reduced-order optimal control of water flooding using proper orthogonal decomposition

Model-based optimal control of water flooding generally involves multiple reservoir simulations, which makes it into a time-consuming process. Furthermore, if the optimization is combined with inversion, i.e., with updating of the reservoir model using production data, some form of regularization is required to cope with the ill-posedness of the inversion problem. A potential way to address these issues is through the use of proper orthogonal decomposition (POD), also known as principal component analysis, Karhunen–Loève decomposition or the method of empirical orthogonal functions. POD is a model reduction technique to generate low-order models using ‘snapshots’ from a forward simulation with the original high-order model. In this work, we addressed the scope to speed up optimization of water-flooding a heterogeneous reservoir with multiple injectors and producers. We used an adjoint-based optimal control methodology that requires multiple passes of forward simulation of the reservoir model and backward simulation of an adjoint system of equations. We developed a nested approach in which POD was first used to reduce the state space dimensions of both the forward model and the adjoint system. After obtaining an optimized injection and production strategy using the reduced-order system, we verified the results using the original, high-order model. If necessary, we repeated the optimization cycle using new reduced-order systems based on snapshots from the verification run. We tested the methodology on a reservoir model with 4050 states (2025 pressures, 2025 saturations) and an adjoint model of 4050 states (Lagrange multipliers). We obtained reduced-order models with 20–100 states only, which produced almost identical optimized flooding strategies as compared to those obtained using the high-order models. The maximum achieved reduction in computing time was 35%.     

Waterflooding optimization in uncertain geological scenarios

In conventional waterflooding of an oil field, feedback based optimal control technologies may enable higher oil recovery than with a conventional reactive strategy in which producers are closed based on water breakthrough. To compensate for the inherent geological uncertainties in an oil field, robust optimization has been suggested to improve and robustify optimal control strategies. In robust optimization of an oil reservoir, the water injection and production borehole pressures (bhp) are computed such that the predicted net present value (NPV) of an ensemble of permeability field realizations is maximized. In this paper, we both consider an open-loop optimization scenario, with no feedback, and a closed-loop optimization scenario. The closed-loop scenario is implemented in a moving horizon manner and feedback is obtained using an ensemble Kalman filter for estimation of the permeability field from the production data. For open-loop implementations, previous test case studies presented in the literature, show that a traditional robust optimization strategy (RO) gives a higher expected NPV with lower NPV standard deviation than a conventional reactive strategy. We present and study a test case where the opposite happen: The reactive strategy gives a higher expected NPV with a lower NPV standard deviation than the RO strategy. To improve the RO strategy, we propose a modified robust optimization strategy (modified RO) that can shut in uneconomical producer wells. This strategy inherits the features of both the reactive and the RO strategy. Simulations reveal that the modified RO strategy results in operations with larger returns and less risk than the reactive strategy, the RO strategy, and the certainty equivalent strategy. The returns are measured by the expected NPV and the risk is measured by the standard deviation of the NPV. In closed-loop optimization, we investigate and compare the performance of the RO strategy, the reactive strategy, and the certainty equivalent strategy. The certainty equivalent strategy is based on a single realization of the permeability field. It uses the mean of the ensemble as its permeability field. Simulations reveal that the RO strategy and the certainty equivalent strategy give a higher NPV compared to the reactive strategy. Surprisingly, the RO strategy and the certainty equivalent strategy give similar NPVs. Consequently, the certainty equivalent strategy is preferable in the closed-loop situation as it requires significantly less computational resources than the robust optimization strategy. The similarity of the certainty equivalent and the robust optimization based strategies for the closed-loop situation challenges the intuition of most reservoir engineers. Feedback reduces the uncertainty and this is the reason for the similar performance of the two strategies.     

A unified formulation for generalized oilfield development optimization

Oilfield development involves several key decisions, including the number, type (injection/production), location, drilling schedule, and operating control trajectories of the wells. Without considering the coupling between these decision variables, any optimization problem formulation is bound to find suboptimal solutions. This paper presents a unified formulation for oilfield development optimization that seeks to simultaneously optimize these decision variables. We show that the source/sink term of the governing multiphase flow equations includes all the above decision variables. This insight leads to a novel and unified formulation of the field development optimization problem that considers the source/sink term in reservoir simulation equations as optimization decision variables. Therefore, a single optimization problem is formulated to simultaneously search for optimal decision variables by determining the complete dynamic form of the source/sink terms. The optimization objective function is the project net present value (NPV), which involves discounted revenue from oil production, operating costs (e.g. water injection and recycling), and capital costs (e.g., cost of drilling wells). A major difficulty after formulating the generalized field development optimization problem is finding an efficient solution approach. Since the total number of cells in a reservoir model far exceeds the number of cells that are intersected by wells, the source/sink terms tend to be sparse. In fact, the drilling cost in the NPV objective function serves as a sparsity-promoting penalty to minimize the number of wells while maximizing the NPV. Inspired by this insight, we solve the optimization problem using an efficient gradient-based method based on recent algorithmic developments in sparse reconstruction literature. The gradients of the NPV function with respect to the source/sink terms is readily computed using well-established adjoint methods. Numerical experiments are presented to evaluate the feasibility and performance of the generalized field development formulation for simultaneous optimization of the number, location, type, controls, and drilling schedule of the wells.     

A simultaneous perturbation stochastic approximation algorithm for coupled well placement and control optimization under geologic uncertainty

Development of subsurface energy and environmental resources can be improved by tuning important decision variables such as well locations and operating rates to optimize a desired performance metric. Optimal well locations in a discretized reservoir model are typically identified by solving an integer programming problem while identification of optimal well settings (controls) is formulated as a continuous optimization problem. In general, however, the decision variables in field development optimization can include many design parameters such as the number, type, location, short-term and long-term operational settings (controls), and drilling schedule of the wells. In addition to the large number of decision variables, field optimization problems are further complicated by the existing technical and physical constraints as well as the uncertainty in describing heterogeneous properties of geologic formations. In this paper, we consider simultaneous optimization of well locations and dynamic rate allocations under geologic uncertainty using a variant of the simultaneous perturbation and stochastic approximation (SPSA). In addition, by taking advantage of the robustness of SPSA against errors in calculating the cost function, we develop an efficient field development optimization under geologic uncertainty, where an ensemble of models are used to describe important flow and transport reservoir properties (e.g., permeability and porosity). We use several numerical experiments, including a channel layer of the SPE10 model and the three-dimensional PUNQ-S3 reservoir, to illustrate the performance improvement that can be achieved by solving a combined well placement and control optimization using the SPSA algorithm under known and uncertain reservoir model assumptions.     

Determination of lower and upper bounds of predicted production from history-matched models

We present a method to determine lower and upper bounds to the predicted production or any other economic objective from history-matched reservoir models. The method consists of two steps: 1) performing a traditional computer-assisted history match of a reservoir model with the objective to minimize the mismatch between predicted and observed production data through adjusting the grid block permeability values of the model. 2) performing two optimization exercises to minimize and maximize an economic objective over the remaining field life, for a fixed production strategy, by manipulating the same grid block permeabilities, however without significantly changing the mismatch obtained under step 1. This is accomplished through a hierarchical optimization procedure that limits the solution space of a secondary optimization problem to the (approximate) null space of the primary optimization problem. We applied this procedure to two different reservoir models. We performed a history match based on synthetic data, starting from a uniform prior and using a gradient-based minimization procedure. After history matching, minimization and maximization of the net present value (NPV), using a fixed control strategy, were executed as secondary optimization problems by changing the model parameters while staying close to the null space of the primary optimization problem. In other words, we optimized the secondary objective functions, while requiring that optimality of the primary objective (a good history match) was preserved. This method therefore provides a way to quantify the economic consequences of the well-known problem that history matching is a strongly ill-posed problem. We also investigated how this method can be used as a means to assess the cost-effectiveness of acquiring different data types to reduce the uncertainty in the expected NPV.     

A variable-control well placement optimization for improved reservoir development

Determination of well locations and their operational settings (controls) such as injection/production rates in heterogeneous subsurface reservoirs poses a challenging optimization problem that has a significant impact on the recovery performance and economic value of subsurface energy resources. The well placement optimization is often formulated as an integer-programming problem that is typically carried out assuming known well control settings. Similarly, identification of the optimal well settings is usually formulated and solved as a control problem in which the well locations are fixed. Solving each of the two problems individually without accounting for the coupling between them leads to suboptimal solutions. Here, we propose to solve the coupled well placement and control optimization problems for improved production performance. We present an alternating iterative solution of the decoupled well placement and control subproblems where each subproblem (e.g., well locations) is resolved after updating the decision variables of the other subproblem (e.g., solving for the control settings) from previous step. This approach allows for application of well-established methods in the literature to solve each subproblem individually. We show that significant improvements can be achieved when the well placement problem is solved by allowing for variable and optimized well controls. We introduce a well-distance constraint into the well placement objective function to avoid solutions containing well clusters in a small region. In addition, we present an efficient gradient-based method for solving the well control optimization problem. We illustrate the effectiveness of the proposed algorithms using several numerical experiments, including the three-dimensional PUNQ reservoir and the top layer of the SPE10 benchmark model.     

Closed-loop reservoir management on the Brugge test case

This paper proposes an augmented Lagrangian method for production optimization in which the cost function to be maximized is defined as an augmented Lagrangian function consisting of the net present value (NPV) and all the equality and inequality constraints except the bound constraints. The bound constraints are dealt with using a trust-region gradient projection method. The paper also presents a way to eliminate the need to convert the inequality constraints to equality constraints with slack variables in the augmented Lagrangian function, which greatly reduces the size of the optimization problem when the number of inequality constraints is large. The proposed method is tested in the context of closed-loop reservoir management benchmark problem based on the Brugge reservoir setup by TNO. In the test, we used the ensemble Kalman filter (EnKF) with covariance localization for data assimilation. Production optimization is done on the updated ensemble mean model from EnKF. The production optimization resulted in a substantial increase in the NPV for the expected reservoir life compared to the base case with reactive control.     

Computing derivative information of sequentially coupled subsurface models

A generic framework for the computation of derivative information required for gradient-based optimization using sequentially coupled subsurface simulation models is presented. The proposed approach allows for the computation of any derivative information with no modification of the mathematical framework. It only requires the forward model Jacobians and the objective function to be appropriately defined. The flexibility of the framework is demonstrated by its application in different reservoir management studies. The performance of the gradient computation strategy is demonstrated in a synthetic water-flooding model, where the forward model is constructed based on a sequentially coupled flow-transport system. The methodology is illustrated for a synthetic model, with different types of applications of data assimilation and life-cycle optimization. Results are compared with the classical fully coupled (FIM) forward simulation. Based on the presented numerical examples, it is demonstrated how, without any modifications of the basic framework, the solution of gradient-based optimization models can be obtained for any given set of coupled equations. The sequential derivative computation methods deliver similar results compared to FIM methods, while being computationally more efficient.     

Ensemble clustering for efficient robust optimization of naturally fractured reservoirs

In the development of naturally fractured reservoirs (NFRs), the existence of natural fractures induces severe fingering and breakthrough. To manage the flooding process and improve the ultimate recovery, we propose a numerical workflow to generate optimal production schedules for smart wells, in which the inflow control valve (ICV) settings can be controlled individually. To properly consider the uncertainty introduced by randomly distributed natural fractures, the robust optimization would require a large ensemble size and it would be computationally demanding. In this work, a hierarchical clustering method is proposed to select representative models for the robust optimization in order to avoid redundant simulation runs and improve the efficiency of the robust optimization. By reducing the full ensemble of models into a small subset ensemble, the efficiency of the robust optimization algorithm is significantly improved. The robust optimization is performed using the StoSAG scheme to find the optimal well controls that maximize the net-present-value (NPV) of the NFR’s development. Due to the discrete property of a natural fracture field, traditional feature extraction methods such as model-parameter-based clustering may not be directly applicable. Therefore, two different kinds of clustering-based optimization methods, a state-based (e.g., s _w profiles) clustering and a response-based (e.g., production rates) clustering, are proposed and compared. The computational results show that the robust clustering optimization could increase the computational efficiency significantly without sacrificing much expected NPV of the robust optimization. Moreover, the performance of different clustering algorithms varies widely in correspondence to different selections of clustering features. By properly extracting model features, the clustered subset could adequately represent the uncertainty of the full ensemble.     

Optimization of nonconventional wells under uncertainty using statistical proxies

The determination of the optimal type and placement of a nonconventional well in a heterogeneous reservoir represents a challenging optimization problem. This determination is significantly more complicated if uncertainty in the reservoir geology is included in the optimization. In this study, a genetic algorithm is applied to optimize the deployment of nonconventional wells. Geological uncertainty is accounted for by optimizing over multiple reservoir models (realizations) subject to a prescribed risk attitude. To reduce the excessive computational requirements of the base method, a new statistical proxy (which provides fast estimates of the objective function) based on cluster analysis is introduced into the optimization process. This proxy provides an estimate of the cumulative distribution function (CDF) of the scenario performance, which enables the quantification of proxy uncertainty. Knowledge of the proxy-based performance estimate in conjunction with the proxy CDF enables the systematic selection of the most appropriate scenarios for full simulation. Application of the overall method for the optimization of monobore and dual-lateral well placement demonstrates the performance of the hybrid optimization procedure. Specifically, it is shown that by simulating only 10% or 20% of the scenarios (as determined by application of the proxy), optimization results very close to those achieved by simulating all cases are obtained.     

A multiobjective steepest descent method with applications to optimal well control

Multiobjective optimization deals with mathematical optimization problems where two or more objective functions (cost functions) are to be optimized (maximized or minimized) simultaneously. In most cases of interest, the objective functions are in conflict, i.e., there does not exist a decision (design) vector (vector of optimization variables) at which every objective function takes on its optimal value. The solution of a multiobjective problem is commonly defined as a Pareto front, and any decision vector which maps to a point on the Pareto front is said to be Pareto optimal. We present an original derivation of an analytical expression for the steepest descent direction for multiobjective optimization for the case of two objectives. This leads to an algorithm which can be applied to obtain Pareto optimal points or, equivalently, points on the Pareto front when the problem is the minimization of two conflicting objectives. The method is in effect a generalization of the steepest descent algorithm for minimizing a single objective function. The steepest-descent multiobjective optimization algorithm is applied to obtain optimal well controls for two example problems where the two conflicting objectives are the maximization of the life-cycle (long-term) net-present-value (NPV) and the maximization of the short-term NPV. The results strongly suggest the multiobjective steepest-descent (MOSD) algorithm is more efficient than competing multiobjective optimization algorithms.     

Neural networks and their derivatives for history matching and reservoir optimization problems

In geosciences, complex forward problems met in geophysics, petroleum system analysis, and reservoir engineering problems often require replacing these forward problems by proxies, and these proxies are used for optimizations problems. For instance, history matching of observed field data requires a so large number of reservoir simulation runs (especially when using geostatistical geological models) that it is often impossible to use the full reservoir simulator. Therefore, several techniques have been proposed to mimic the reservoir simulations using proxies. Due to the use of experimental approach, most authors propose to use second-order polynomials. In this paper, we demonstrate that (1) neural networks can also be second-order polynomials. Therefore, the use of a neural network as a proxy is much more flexible and adaptable to the nonlinearity of the problem to be solved; (2) first-order and second-order derivatives of the neural network can be obtained providing gradients and Hessian for optimizers. For inverse problems met in seismic inversion, well by well production data, optimal well locations, source rock generation, etc., most of the time, gradient methods are used for finding an optimal solution. The paper will describe how to calculate these gradients from a neural network built as a proxy. When needed, the Hessian can also be obtained from the neural network approach. On a real case study, the ability of neural networks to reproduce complex phenomena (water cuts, production rates, etc.) is shown. Comparisons with second polynomials (and kriging methods) will be done demonstrating the superiority of the neural network approach as soon as nonlinearity behaviors are present in the responses of the simulator. The gradients and the Hessian of the neural network will be compared to those of the real response function.     

Theoretical connections between optimization algorithms based on an approximate gradient

Performing a line search method in the direction given by the simplex gradient is a well-known method in the mathematical optimization community. For reservoir engineering optimization problems, both a modification of the simultaneous perturbation stochastic approximation (SPSA) and ensemble-based optimization (EnOpt) have recently been applied for estimating optimal well controls in the production optimization step of closed-loop reservoir management. The modified SPSA algorithm has also been applied to assisted history-matching problems. A recent comparison of the performance of EnOpt and a SPSA-type algorithm (G-SPSA) for a set of production optimization test problems showed that the two algorithms resulted in similar estimates of the optimal net-present-value and required roughly the same amount of computational time to achieve these estimates. Here, we show that, theoretically, this result is not surprising. In fact, we show that both the simplex, preconditioned simplex, and EnOpt algorithms can be derived directly from a modified SPSA-type algorithm where the preconditioned simplex algorithm is presented for the first time in this paper. We also show that the expectation of all these preconditioned stochastic gradients is a first-order approximation of the preconditioning covariance matrix times the true gradient or a covariance matrix squared times the true gradient.     

Hybrid differential evolution and particle swarm optimization for optimal well placement

There is no gainsaying that determining the optimal number, type, and location of hydrocarbon reservoir wells is a very important aspect of field development planning. The reason behind this fact is not farfetched—the objective of any field development exercise is to maximize the total hydrocarbon recovery, which for all intents and purposes, can be measured by an economic criterion such as the net present value of the reservoir during its estimated operational life-cycle. Since the cost of drilling and completion of wells can be significantly high (millions of dollars), there is need for some form of operational and economic justification of potential well configuration, so that the ultimate purpose of maximizing production and asset value is not defeated in the long run. The problem, however, is that well optimization problems are by no means trivial. Inherent drawbacks include the associated computational cost of evaluating the objective function, the high dimensionality of the search space, and the effects of a continuous range of geological uncertainty. In this paper, the differential evolution (DE) and the particle swarm optimization (PSO) algorithms are applied to well placement problems. The results emanating from both algorithms are compared with results obtained by applying a third algorithm called hybrid particle swarm differential evolution (HPSDE)—a product of the hybridization of DE and PSO algorithms. Three cases involving the placement of vertical wells in 2-D and 3-D reservoir models are considered. In two of the three cases, a max-mean objective robust optimization was performed to address geological uncertainty arising from the mismatch between real physical reservoir and the reservoir model. We demonstrate that the performance of DE and PSO algorithms is dependent on the total number of function evaluations performed; importantly, we show that in all cases, HPSDE algorithm outperforms both DE and PSO algorithms. Based on the evidence of these findings, we hold the view that hybridized metaheuristic optimization algorithms (such as HPSDE) are applicable in this problem domain and could be potentially useful in other reservoir engineering problems.     

A real-time operation optimization model for flood management in river-reservoir systems

In this paper, a new methodology has been developed for real-time flood management in river-reservoir systems. This methodology is based upon combining a Genetic Algorithm (GA) reservoir operation optimization model for a cascade of two reservoirs, a hydraulic-based flood routing simulation model in downstream river system, a Geographical Information System (GIS) based database, and application of K-Nearest Neighbor (K-NN) algorithm for development of optimal operating rules. The GA optimization model estimates the optimal hourly reservoirs’ releases to minimize the flood damages in the downstream river. GIS tools have also been used for specifying different land-uses and damage functions in the downstream floodplain and it has been linked to the unsteady module of HEC-RAS flood routing model using Hec-GeoRAS module. An innovative approach has also been developed using K-NN algorithm to formulate the optimal operating rules for a system of two cascade reservoirs based on optimal releases obtained from the optimization model. During a flood event, the K-NN algorithm searches through the historical flood hydrographs and optimal reservoir storages determined by the optimization model to find similar situations. The similarity between the hydrographs is quantified based on the slopes of rising and falling limbs of inflow hydrographs and reservoir storages at the beginning of each hourly time step during the flood events for two cascade reservoirs. The developed methodology have been applied to the Bakhtiari and Dez River-Reservoir systems in southwest of Iran. The results show that the proposed models can be effectively used for flood management and real-time operation of cascade river-reservoir systems.     

An Improved Gradual Deformation Method for Reconciling Random and Gradient Searches in Stochastic Optimizations

Constraining stochastic models of reservoir properties such as porosity and permeability can be formulated as an optimization problem. While an optimization based on random search methods preserves the spatial variability of the stochastic model, it is prohibitively computer intensive. In contrast, gradient search methods may be very efficient but it does not preserve the spatial variability of the stochastic model. The gradual deformation method allows for modifying a reservoir model (i.e., realization of the stochastic model) from a small number of parameters while preserving its spatial variability. It can be considered as a first step towards the merger of random and gradient search methods. The gradual deformation method yields chains of reservoir models that can be investigated successively to identify an optimal reservoir model. The investigation of each chain is based on gradient computations, but the building of chains of reservoir models is random. In this paper, we propose an algorithm that further improves the efficiency of the gradual deformation method. Contrary to the previous gradual deformation method, we also use gradient information to build chains of reservoir models. The idea is to combine the initial reservoir model or the previously optimized reservoir model with a compound reservoir model. This compound model is a linear combination of a set of independent reservoir models. The combination coefficients are calculated so that the search direction from the initial model is as close as possible to the gradient search direction. This new gradual deformation scheme allows us for reducing the number of optimization parameters while selecting an optimal search direction. The numerical example compares the performance of the new gradual deformation scheme with that of the traditional one.