Dynamic programming and the principle of optimality: A systematic approach

The dynamic programming recursive procedure has provided an efficient method for solving a variety of sequential decision problems related to water resources systems. In many investigations Bellman's principle of optimality is used as a proof for the optimality of the dynamic programming solutions. In this paper the dynamic programming procedure is systematically studied so as to clarify the relationship between Bellman's principle of optimality and the optimality of the dynamic programming solutions.Our main result is that although the principle is valid, in order to use it as a proof for the optimality of the dynamic programming solution certain modeling requirements should be met.The mathematical model presented in this paper provides a convenient framework for the modeling and analysis of dynamic programming problems encountered by in water resources management studies.The results derived here resolve few of the fundamental questions raised in the literature regarding the validity of Bellman's principle of oplimality and the optimality of the dynamic programming solutions.