A selective improvement technique for fastening Neuro-Dynamic Programming in Water Resources Network Management
World Congress, Volume # 16 | Part# 1
D. de Rigo; A. Castelletti; A. E. Rizzoli; R. Soncini-Sessa; E. Weber
Digital Object Identifier (DOI)
integrated water resources management,stochastic dynamic programming,neurodynamic programming,evolutionary algorithm
An approach to the integrated water resources management based on Neuro-Dynamic Programming (NDP) with an improved technique for fastening its Artificial Neural Network (ANN) training phase will be presented. When dealing with networks of water resources, Stochastic Dynamic Programming provides an effective solution methodology but suffers from the so-called "curse of dimensionality", that rapidly leads to the problem intractability. NDP can sensibly mitigate this drawback by approximating the solution with ANNs. However in the real world applications NDP shows to be considerably slowed just by this ANN training phase. To overcome this limit a new training architecture (SIEVE: Selective Improvement by Evolutionary Variance Extinction) has been developed. In this paper this new approach is theoretically introduced and some preliminary results obtained on a real world case study are presented.
 Archibald, T. W., McKinnon, K.I.M., and Thomas, L. C., An aggregate stochastic dynamic programming model of multireservoir systems, Water Resour. Res., 33, pp. 333-340, 1997.  Bellman, R. E., and Dreyfus, S. E., Functional approximations and dynamic programming, Mathematical Tables and Other Aids to Computation, 13, pp. 247- 251, 1959.  Bellman, R. E., Kabala, R., Kotkin, B., Polynomial approximation - a new computational technique in dynamic programming, Mathematical Tables and Other Aids to Computation, 17, pp. 155-161, 1963.  Bertsekas, D. P., Dynamic Programming and Optimal Control, Athena Scientific, Belmont, MA, 1995.  Bertsekas, D. P., and J. N. Tsitsiklis, Neuro-Dynamic Programming, Athena Scientific, Belmont, MA, 1996.  De Rigo, D., Rizzoli, A. E., Soncini-Sessa, R., Weber, E., Zenesi, P., Neuro-Dynamic Programming for the Efficient Management of Reservoir Networks, presented at: MODSIM 2001, Canberra, Australia, 2001.  Georgakakos, A. P., and Marks, D. H., A new method for real-time operation of reservoir systems, Water Resour. Res., 23(7), pp. 1376-1390, 1987.  Georgakakos, A. P., Extended Linear Quadratic Gaussian Control for the real-time operation of reservoir systems, in Dynamic Programming for Optimal Water Resources Systems Analysis, A. Esogbue, ed., Prentice Hall Publishing Company, NJ, pp. 329-360, 1989.  Hornik, K., Multilayer feedforward networks are universal approximators, Neural Networks, 2, pp. 359-366, 1989.  Kreinovich, V., Arbitrary nonlinearity is sufficient to represent all functions by neural networks: a theorem, Neural Networks, vol. 4, pp. 381-383, 1991.  Lamond, B. F., Boukhtouta, A., Optimizing future hydropower production using Markov Decision Processes, Working Paper 95-44, CRAEDO, Université Laval, Quebec, 1997.  Maas, A., M. M. Hufschmidt, R. Dorfam, H. A. Thomas, S. A. Marglin and G. M. Fair, Design of Water Resource Systems, Harvard Univ. Press, 1962.  Piccardi, C. and R. Soncini-Sessa, Stochastic dynamic programming for reservoir optimal control: dense discretization and inflow correlation assumption made possible by parallel computing. Water Resour. Res., 27(2), pp. 729-741, 1991.  Rippl, W., The capacity of storage reservoirs for water supply. Minutes Proc. Inst. Civ. Eng., 71, pp. 270- 278, 1883.  Saad, M., Turgeon, A., Bigras, P., Duquette, R., Learning Disaggregation Technique for the Operation of Long-Term Hydroelectric Power Systems, Water Resour. Res., 30(11), pp. 3195-3203, 1994.  Soncini-Sessa, R., Castelletti, A., Rizzoli, A. E., Weber, E., New ideas in the design of water reservoir management policies, invited paper at: IFAC Workshop "Modelling and Control in Environmental Issues", Yokohama, Japan, 2001.  Tejada-Guibert, J. A., Johnson, S. A., Stedinger, J. R., Comparison of two approaches for implementing multi-reservoir operating policies using dynamic programming, Water Resour. Res., 29, pp. 369-380, 1993.  Turgeon, A., A decomposition method for the long-term scheduling of reservoirs in series, Water Resour. Res., 17, pp. 1565-1570, 1981.  Yakowitz, S., Dynamic programming applications in water resources, Water Resour. Res., 18, pp. 673- 696, 1982.  Yeh, W., Reservoir management and operations models: a state of the art review, Water Resour. Res., 21, pp. 1797-1818, 1985.