Development of stochastic dynamic Nash game model for reservoir operation. I. The symmetric stochastic model with perfect information

Increasing water demands, higher standards of living, depletion of resources of acceptable quality and excessive water pollution due to agricultural and industrial expansions have caused intense social and political predicaments, and conflicting issues among water consumers. The available techniques commonly used in reservoir optimization/operation do not consider interaction, behavior and preferences of water users, reservoir operator and their associated modeling procedures, within the stochastic modeling framework. In this paper, game theory is used to present the associated conflicts among different consumers due to limited water. Considering the game theory fundamentals, the Stochastic Dynamic Nash Game with perfect information (PSDNG) model is developed, which assumes that the decision maker has sufficient (perfect) information regarding the associated randomness of reservoir operation parameters. The simulated annealing approach (SA) is applied as a part of the proposed stochastic framework, which makes the PSDNG solution conceivable. As a case study, the proposed model is applied to the Zayandeh-Rud river basin in Iran with conflicting demands. The results are compared with alternative reservoir operation models, i.e., Bayesian stochastic dynamic programming (BSDP), sequential genetic algorithm (SGA) and classical dynamic programming regression (DPR). Results show that the proposed model has the ability to generate reservoir operating policies, considering interactions of water users, reservoir operator and their preferences.     

Two solution methods for dynamic game in reservoir operation

During the last decade, a number of models have been developed to consider the conflict in dynamic reservoir operation. Most of these models are discrete dynamic models which are developed based on game theory. In this study, a continuous model of dynamic game and its corresponding solutions are developed for reservoir operation. Two solution methods are used to solve the model of continuous dynamic game, namely the Ricatti equations and collocation methods. The Ricatti equations method is a closed form solution, requiring less computational efforts compared with discrete models. The collocation solution method applies Newton's method or a quasi-Newton method to find the problem solution. These approaches are able to generate operating policies for dynamic reservoir operation. The Zayandeh-Rud river basin in central Iran is used as a case study and the results are compared with alternative water allocation models. The results show that the proposed solution methods are quite capable of providing appropriate reservoir operating policies, while requiring rather short computational times due to continuous formulation of state and decision variables. Reliability indices are used to compare the overall performance of the proposed models. Based on the results from this study, the collocation method leads to improved values of the reliability indices for total reservoir system and utility satisfaction of water users, compared to the Ricatti equations method. This is attributed to the flexible structure of the collocation model. When compared to alternative water allocation models, lower values of reliability indices are achieved by the collocation method.     

Probabilistic reservoir operation using Bayesian stochastic model and support vector machine

Available water resources are often not sufficient or too polluted to satisfy the needs of all water users. Therefore, allocating water to meet water demands with better quality is a major challenge in reservoir operation. In this paper, a methodology to develop operating strategies for water release from a reservoir with acceptable quality and quantity is presented. The proposed model includes a genetic algorithm (GA)-based optimization model linked with a reservoir water quality simulation model. The objective function of the optimization model is based on the Nash bargaining theory to maximize the reliability of supplying the downstream demands with acceptable quality, maintaining a high reservoir storage level, and preventing quality degradation of the reservoir. In order to reduce the run time of the GA-based optimization model, the main optimization model is divided into a stochastic and a deterministic optimization model for reservoir operation considering water quality issues.The operating policies resulted from the reservoir operation model with the water quantity objective are used to determine the released water ranges (permissible lower and upper bounds of release policies) during the planning horizon. Then, certain values of release and the optimal releases from each reservoir outlet are determined utilizing the optimization model with water quality objectives. The support vector machine (SVM) model is used to generate the operating rules for the selective withdrawal from the reservoir for real-time operation. The results show that the SVM model can be effectively used in determining water release from the reservoir. Finally, the copula function was used to estimate the joint probability of supplying the water demand with desirable quality as an evaluation index of the system reliability. The proposed method was applied to the Satarkhan reservoir in the north-western part of Iran. The results of the proposed models are compared with the alternative models. The results show that the proposed models could be used as effective tools in reservoir operation.     

Evaluation of stochastic reservoir operation optimization models

This paper investigates the performance of seven stochastic models used to define optimal reservoir operating policies. The models are based on implicit (ISO) and explicit stochastic optimization (ESO) as well as on the parameterization–simulation–optimization (PSO) approach. The ISO models include multiple regression, two-dimensional surface modeling and a neuro-fuzzy strategy. The ESO model is the well-known and widely used stochastic dynamic programming (SDP) technique. The PSO models comprise a variant of the standard operating policy (SOP), reservoir zoning, and a two-dimensional hedging rule. The models are applied to the operation of a single reservoir damming an intermittent river in northeastern Brazil. The standard operating policy is also included in the comparison and operational results provided by deterministic optimization based on perfect forecasts are used as a benchmark. In general, the ISO and PSO models performed better than SDP and the SOP. In addition, the proposed ISO-based surface modeling procedure and the PSO-based two-dimensional hedging rule showed superior overall performance as compared with the neuro-fuzzy approach.     

A stochastic model for optimal operation of inter-basin water allocation systems: a case study

This paper presents a new methodology for optimal operation of inter-basin water transfer systems by conjunctive use of surface water resources in water donor basin and groundwater resources in water receiving basin. To incorporate the streamflow uncertainty, an integrated stochastic dynamic programming (ISDP) model is developed. In the ISDP, the monthly inflow to the reservoir in the water donor basin, the water storage of the reservoir, and the water storage of the aquifer in the water receiving basin are considered as state variables. A water allocation optimization model is embedded in the main structure of ISDP and a new ensemble streamflow prediction model based on K-nearest-neighbourhood algorithm is also developed and linked to the ISDP. By using a new reoptimization process, the ISDP model provides monthly policies for water allocation to users in water donor and receiving basins. As water users can form a coalition to increase their benefits, several solution concepts in cooperative game theory, namely Nash–Harsanyi, Shapley, Nucleolus, Weak Nucleolus, Proportional Nucleolus, Separable Costs Remaining Benefits (SCRBs) and Minimum Costs Remaining Savings are utilized to determine the profit of each water user. In the last step, stakeholders make negotiation over these solution concepts using the Fallback bargaining theory to reach a unanimous agreement on the final distribution of the total benefit. The methodology is applied to an inter-basin water transfer project and the results show that the Shapley and SCRB solutions concepts can provide better distributions for the total benefit and the total benefit of water users is increased by a factor of 1.6 when they participate in a grand coalition.     

Stochastic differential dynamic programming for multi-reservoir system control

: As with all dynamic programming formulations, differential dynamic programming (DDP) successfully exploits the sequential decision structure of multi-reservoir optimization problems, overcomes difficulties with the nonconvexity of energy production functions for hydropower systems, and provides optimal feedback release policies. DDP is particularly well suited to optimizing large-scale multi-reservoir systems due to its relative insensitivity to state-space dimensionality. This advantage of DDP encourages expansion of the state vector to include additional multi-lag hydrologic information and/or future inflow forecasts in developing optimal reservoir release policies. Unfortunately, attempts at extending DDP to the stochastic case have not been entirely successful. A modified stochastic DDP algorithm is presented which overcomes difficulties in previous formulations. Application of the algorithm to a four-reservoir hydropower system demonstrates its capabilities as an efficient approach to solving stochastic multi-reservoir optimization problems. The algorithm is also applied to a single reservoir problem with inclusion of multi-lag hydrologic information in the state vector. Results provide evidence of significant benefits in direct inclusion of expanded hydrologic state information in optimal feedback release policies.     

Stochastic differential dynamic programming for multi-reservoir system control

: As with all dynamic programming formulations, differential dynamic programming (DDP) successfully exploits the sequential decision structure of multi-reservoir optimization problems, overcomes difficulties with the nonconvexity of energy production functions for hydropower systems, and provides optimal feedback release policies. DDP is particularly well suited to optimizing large-scale multi-reservoir systems due to its relative insensitivity to state-space dimensionality. This advantage of DDP encourages expansion of the state vector to include additional multi-lag hydrologic information and/or future inflow forecasts in developing optimal reservoir release policies. Unfortunately, attempts at extending DDP to the stochastic case have not been entirely successful. A modified stochastic DDP algorithm is presented which overcomes difficulties in previous formulations. Application of the algorithm to a four-reservoir hydropower system demonstrates its capabilities as an efficient approach to solving stochastic multi-reservoir optimization problems. The algorithm is also applied to a single reservoir problem with inclusion of multi-lag hydrologic information in the state vector. Results provide evidence of significant benefits in direct inclusion of expanded hydrologic state information in optimal feedback release policies.     

A fuzzy-Markov-chain-based analysis method for reservoir operation

In this study, a fuzzy-Markov-chain-based stochastic dynamic programming (FM-SDP) method is developed for tackling uncertainties expressed as fuzzy sets and distributions with fuzzy probability (DFPs) in reservoir operation. The concept of DFPs used in Markov chain is presented as an extended form for expressing uncertainties including both stochastic and fuzzy characteristics. A fuzzy dominance index analysis approach is proposed for solving multiple fuzzy sets and DPFs in the proposed FM-SDP model. Solutions under a set of α-cut levels and fuzzy dominance indices can be generated by solving a series of deterministic submodels. The developed method is applied to a case study of a reservoir operation system. Solutions from FM-SDP provide a range of desired water-release policies under various system conditions for reservoir operation decision makers, reflecting dynamic and dual uncertain features of water availability simultaneously. The results indicate that the FM-SDP method could be applicable to practical problems for decision makers to obtain insight regarding the tradeoffs between economic and system reliability criteria. Willingness to obtain a lower benefit may guarantee meeting system-constraint demands; conversely, a desire to acquire a higher benefit could run into a higher risk of violating system constraints.     

A neural network based general reservoir operation scheme

Construction of dams and the resulting water impoundments are one of the most common engineering procedures implemented on river systems globally; yet simulating reservoir operation at the regional and global scales remains a challenge in human–earth system interactions studies. Developing a general reservoir operating scheme suitable for use in large-scale hydrological models can improve our understanding of the broad impacts of dams operation. Here we present a novel use of artificial neural networks to map the general input/output relationships in actual operating rules of real world dams. We developed a new general reservoir operation scheme (GROS) which may be added to daily hydrologic routing models for simulating the releases from dams, in regional and global-scale studies. We show the advantage of our model in distinguishing between dams with various storage capacities by demonstrating how it modifies the reservoir operation in respond to changes in capacity of dams. Embedding GROS in a water balance model, we analyze the hydrological impact of dam size as well as their distribution pattern within a drainage basin and conclude that for large-scale studies it is generally acceptable to aggregate the capacity of smaller dams and instead model a hypothetical larger dam with the same total storage capacity; however we suggest limiting the aggregation area to HUC 8 sub-basins (approximately equal to the area of a 60 km or a 30 arc minute grid cell) to avoid exaggerated results.     

Reservoir performance under uncertainty in hydrologic impacts of climate change

Relatively few studies have addressed water management and adaptation measures in the face of changing water balances due to climate change. The current work studies climate change impact on a multipurpose reservoir performance and derives adaptive policies for possible future scenarios. The method developed in this work is illustrated with a case study of Hirakud reservoir on the Mahanadi river in Orissa, India, which is a multipurpose reservoir serving flood control, irrigation and power generation. Climate change effects on annual hydropower generation and four performance indices (reliability with respect to three reservoir functions, viz. hydropower, irrigation and flood control, resiliency, vulnerability and deficit ratio with respect to hydropower) are studied. Outputs from three general circulation models (GCMs) for three scenarios each are downscaled to monsoon streamflow in the Mahanadi river for two future time slices, 2045–65 and 2075–95. Increased irrigation demands, rule curves dictated by increased need for flood storage and downscaled projections of streamflow from the ensemble of GCMs and scenarios are used for projecting future hydrologic scenarios. It is seen that hydropower generation and reliability with respect to hydropower and irrigation are likely to show a decrease in future in most scenarios, whereas the deficit ratio and vulnerability are likely to increase as a result of climate change if the standard operating policy (SOP) using current rule curves for flood protection is employed. An optimal monthly operating policy is then derived using stochastic dynamic programming (SDP) as an adaptive policy for mitigating impacts of climate change on reservoir operation. The objective of this policy is to maximize reliabilities with respect to multiple reservoir functions of hydropower, irrigation and flood control. In variations to this adaptive policy, increasingly more weightage is given to the purpose of maximizing reliability with respect to hydropower for two extreme scenarios. It is seen that by marginally sacrificing reliability with respect to irrigation and flood control, hydropower reliability and generation can be increased for future scenarios. This suggests that reservoir rules for flood control may have to be revised in basins where climate change projects an increasing probability of droughts. However, it is also seen that power generation is unable to be restored to current levels, due in part to the large projected increases in irrigation demand. This suggests that future water balance deficits may limit the success of adaptive policy options.     

Reservoir operation using grey fuzzy stochastic dynamic programming

This paper presents an optimal regulation programme, grey fuzzy stochastic dynamic programming (GFSDP), for reservoir operation. It is composed of a grey system, fuzzy theory and dynamic programming. The grey system represents data by covering the whole range without loss of generality, and the fuzzy arithmetic takes charge of the rules of reservoir operation. The GFSDP deals with the multipurpose decision‐making problem by fuzzy optimization theorem. The practicability and effectiveness of the proposed approach is tested on the operation of the Shiman reservoir in Taiwan. The current M5 operating rule curves of this reservoir also are evaluated. The simulation results demonstrate that this new approach, in comparison with the M5 rule curves, has superior performance with regard to the total water deficit and number of monthly deficits. Copyright © 2002 John Wiley & Sons, Ltd.     

Incorporating inflow uncertainty into risk assessment for reservoir operation

An attempt of using stochastic hydrologic technique to assess the intrinsic risk of reservoir operation is made in this study. A stochastic simulation model for reservoir operation is developed. The model consists of three components: synthetic generation model for streamflow and sediment sequences, one-dimensional delta deposit model for sediment transport processes in reservoirs, and simulation model for reservoir operation. This kind of integrated simulation model can be used to simulate not only the inflow uncertainty of streamflow and sedimentation, but also the variation in operation rules of reservoirs. It is herein used for the risk assessment of a reservoir, and the simulation is performed for different operation scenarios. Simulation for the 100-year period of sediment transport and deposition in the river-reservoir system indicates that the navigation risk is much higher than that of hydropower generation or sediment deposition in the reservoir. The risk of sediment deposition at the river-section near the backwater profile is also high thereby the navigation at the river-segment near this profile takes high risk because of inadequate navigation depth.     

Building more realistic reservoir optimization models using data mining - A case study of Shelbyville Reservoir

In this paper, we promote a novel approach to develop reservoir operation routines by learning from historical hydrologic information and reservoir operations. The proposed framework involves a knowledge discovery step to learn the real drivers of reservoir decision making and to subsequently build a more realistic (enhanced) model formulation using stochastic dynamic programming (SDP). The enhanced SDP model is compared to two classic SDP formulations using Lake Shelbyville, a reservoir on the Kaskaskia River in Illinois, as a case study. From a data mining procedure with monthly data, the past month’s inflow (Q_t−1), current month’s inflow (Q_t), past month’s release (R_t−1), and past month’s Palmer drought severity index (PDSI_t−1) are identified as important state variables in the enhanced SDP model for Shelbyville Reservoir. When compared to a weekly enhanced SDP model of the same case study, a different set of state variables and constraints are extracted. Thus different time scales for the model require different information. We demonstrate that adding additional state variables improves the solution by shifting the Pareto front as expected while using new constraints and the correct objective function can significantly reduce the difference between derived policies and historical practices. The study indicates that the monthly enhanced SDP model resembles historical records more closely and yet provides lower expected average annual costs than either of the two classic formulations (25.4% and 4.5% reductions, respectively). The weekly enhanced SDP model is compared to the monthly enhanced SDP, and it shows that acquiring the correct temporal scale is crucial to model reservoir operation for particular objectives.     

The role of hydrologic information in reservoir operation – Learning from historical releases

Closing the gap between theoretical reservoir operation and the real-world implementation remains a challenge in contemporary reservoir operations. Past research has focused on optimization algorithms and establishing optimal policies for reservoir operations. In this study, we attempt to understand operators’ release decisions by investigating historical release data from 79 reservoirs in California and the Great Plains, using a data-mining approach. The 79 reservoirs are classified by hydrological regions, intra-annual seasons, average annual precipitation (climate), ratio of maximum reservoir capacity to average annual inflow (size ratio), hydrologic uncertainty associated with inflows, and reservoirs’ main usage. We use information theory – specifically, mutual information – to measure the quality of inference between a set of classic indicators and observed releases at the monthly and weekly timescales. Several general trends are found to explain which sources of hydrologic information dictate reservoir release decisions under different conditions. Current inflow is the most important indicator during wet seasons, while previous releases are more relevant during dry seasons and in weekly data (as compared with monthly data). Inflow forecasting is the least important indicator in release decision making, but its importance increases linearly with hydrologic uncertainty and decreases logarithmically with reservoir size. No single hydrologic indicator is dominant across all reservoirs in either of the two regions.     

A non-traditional approach to the analysis of flood hazard for dams

The traditional and still prevailing approach to characterization of flood hazards to dams is the inflow design flood (IDF). The IDF, defined either deterministically or probabilistically, is necessary for sizing a dam, its discharge facilities and reservoir storage. However, within the dam safety risk informed decision framework, the IDF does not carry much relevance, no matter how accurately it is characterized. In many cases, the probability of the reservoir inflow tells us little about the probability of dam overtopping. Typically, the reservoir inflow and its associated probability of occurrence is modified by the interplay of a number of factors (reservoir storage, reservoir operating rules and various operational faults and natural disturbances) on its way to becoming the reservoir outflow and corresponding peak level—the two parameters that represent hydrologic hazard acting upon the dam. To properly manage flood risk, it is essential to change approach to flood hazard analysis for dam safety from the currently prevailing focus on reservoir inflows and instead focus on reservoir outflows and corresponding reservoir levels. To demonstrate these points, this paper presents stochastic simulation of floods on a cascade system of three dams and shows progression from exceedance probabilities of reservoir inflow to exceedance probabilities of peak reservoir level depending on initial reservoir level, storage availability, reservoir operating rules and availability of discharge facilities on demand. The results show that the dam overtopping is more likely to be caused by a combination of a smaller flood and a system component failure than by an extreme flood on its own.     

A stochastic dynamic programming model with fuzzy storage states for reservoir operations

Application of stochastic dynamic programming (SDP) models to reservoir optimization calls for state variables discretization. Reservoir storage volume is an important variable whose discretization has a pronounced effect on the computational efforts. The error caused by storage volume discretization is examined by considering it as a fuzzy state variable. In this approach, the point-to-point transitions between storage volumes at the beginning and end of each period are replaced by transitions between storage intervals. This is achieved by using fuzzy arithmetic operations with fuzzy numbers. In this approach, instead of aggregating single-valued crisp numbers, the membership functions of fuzzy numbers are combined. Running a simulation model with optimal release policies derived from fuzzy and non-fuzzy SDP models shows that a fuzzy SDP with a coarse discretization scheme performs as well as a classical SDP having much finer discretized space. It is believed that this advantage in the fuzzy SDP model is due to the smooth transitions between storage intervals which benefit from soft boundaries.     

Deriving reservoir operational strategies considering water quantity and quality objectives by stochastic fuzzy neural networks

By taking advantage of the close relationship between quality and quantity of water, we investigated the potential improvements of the in-reservoir water quality through the optimization of reservoir operational strategies. However, the few available techniques for optimization of reservoir operational strategies present some limitations, such as restrictions on the number of state/decision variables, the impossibility considering stochastic characteristics and difficulties for considering simulation/prediction models. One technique which presents great potential for overcoming some of these limitations is applied here and investigated for the first time in such complex system. The method, named stochastic fuzzy neural network (SFNN), can be defined as a fuzzy neural network (FNN) model stochastically trained by a genetic algorithm (GA) based model to yield a quasi optimal solution. The term “stochastically trained” refers to the introduction of a new loop within the training process which accounts for the stochastic variable of the system and its probabilities of occurrence. The SFNN was successfully applied to the optimization of the monthly operational strategies considering maximum water utilization and improvements on water quality simultaneous. Results showed the potential improvements on the water quality through means of hydraulic control.     

Storage-yield analysis of surface water reservoirs: the role of reliability constraints and operating policies

Stochastic optimization methods are used for optimal design and operation of surface water reservoir systems under uncertainty. Chance-constrained (CC) optimization with linear decision rules (LDRs) is an old approach for determining the minimum reservoir capacity required to meet a specific yield at a target level of reliability. However, this approach has been found to overestimate the reservoir capacity. In this paper, we propose the reason for this overestimation to be the fact that the reliability constraints considered in standard CC LDR models do not have the same meaning as in other models such as reservoir operation simulation models. The simulation models have fulfilled a target reliability level in an average sense (i.e., annually), whereas the standard CC LDR models have met the target reliability level every season of the year. Mixed integer nonlinear programs are presented to clarify the distinction between the two types of reliability constraints and demonstrate that the use of seasonal reliability constraints, rather than an average reliability constraint, leads to 80–150 % and 0–32 % excess capacity for SQ-type and S-type CC LDR models, respectively. Additionally, a modified CC LDR model with an average reliability constraint is proposed to overcome the reservoir capacity overestimation problem. In the second stage, we evaluate different operating policies and show that for the seasonal (average) reliability constraints, open-loop, S-type, standard operating policy, SQ-type, and general SQ-type policies compared to a model not using any operation rule lead to 190–460 % (200–550 %), 100–200 % (80–300 %), 0–90 % (0–60 %), 30–90 % (0–20 %), and 10–90 % (0–10 %) excess capacity, respectively.     

A stochastic conflict resolution model for water quality management in reservoir–river systems

In this paper, optimal operating rules for water quality management in reservoir–river systems are developed using a methodology combining a water quality simulation model and a stochastic GA-based conflict resolution technique. As different decision-makers and stakeholders are involved in the water quality management in reservoir–river systems, a new stochastic form of the Nash bargaining theory is used to resolve the existing conflict of interests related to water supply to different demands, allocated water quality and waste load allocation in downstream river. The expected value of the Nash product is considered as the objective function of the model which can incorporate the inherent uncertainty of reservoir inflow. A water quality simulation model is also developed to simulate the thermal stratification cycle in the reservoir, the quality of releases from different outlets as well as the temporal and spatial variation of the pollutants in the downstream river. In this study, a Varying Chromosome Length Genetic Algorithm (VLGA), which has computational advantages comparing to other alternative models, is used. VLGA provides a good initial solution for Simple Genetic Algorithms and comparing to Stochastic Dynamic Programming (SDP) reduces the number of state transitions checked in each stage. The proposed model, which is called Stochastic Varying Chromosome Length Genetic Algorithm with water Quality constraints (SVLGAQ), is applied to the Ghomrud Reservoir–River system in the central part of Iran. The results show, the proposed model for reservoir operation and waste load allocation can reduce the salinity of the allocated water demands as well as the salinity build-up in the reservoir.     

Effects of reservoir stratification and watershed hydrology on manganese and iron in a dam‐regulated river

The presence of metals, including manganese (Mn) and iron (Fe), adversely impacts water quality. In seasonally stratified reservoirs, Mn and Fe can accumulate in the water column due to reducing conditions in sediments and be released to downstream rivers through dam discharge. In addition to reservoir stratification influences, the release of metals downstream is influenced by hydrologic conditions in the river. We examined the seasonal and spatial variability of Mn and Fe concentrations in a eutrophic, hydropower reservoir and the downstream river over a two‐year period. Overall, we found that reservoir stratification was a strong predictor of tailrace Mn and Fe concentrations but that tailrace Fe concentrations were also influenced by dam discharge. Downgradient of the tailrace, river discharge and suspended sediment were the dominant predictors of both Mn and Fe concentrations. Using our data, we develop a conceptual model of seasonal and hydrologic drivers of metal concentrations. The model can be modified for other systems aiding drinking water utilities and other water users in forecasting under what seasonal and hydrologic conditions that Mn and Fe concentrations in river systems are likely to be elevated.