Optimal stopping model with unknown transition probabilities

• Autorzy: Horiguchi M. , Piunovskiy A. B.
• Języki publikacji: EN

Abstrakty

EN

This article concerns the optimal stopping problem for a discrete-time Markov chain with observable states, but with unknown transition probabilities. A stopping policy is graded via the expected total-cost criterion resulting from the non-negative running and terminal costs. The Dynamic Programming method, combined with the Bayesian approach, is developed. A series of explicitly solved meaningful examples illustrates all the theoretical issues.

Słowa kluczowe

EN

Markov Decision Process (MDP) unknown transition matrices; dynamic programming; Bayesian method; optimal stopping

• Wydawca: Systems Research Institute, Polish Academy of Sciences
• Czasopismo: Control and Cybernetics
• Rocznik: 2013
• Tom: Vol. 42, no. 3
• Strony: 593--612
• Opis fizyczny: Bibliogr. 30 poz.

Twórcy

autor

Horiguchi M.

• Department of Mathematics, Faculty of Science Kanagawa University 2946 Tsuchiya, Hiratsuka-shi, Kanagawa 259-1293, Japan

autor

Piunovskiy A. B.

• Department of Mathematical Sciences University of Liverpool L69 7ZL, Liverpool, United Kingdom

Bibliografia

• 1. Bäuerle, N. and Rieder, U. (2011) Markov Decision Processes with Applications to Finance. Springer-Verlag, Berlin.
• 2. Bertsekas, D.P. and Shreve, S.E. (1978) Stochastic Optimal Control. Academic Press, New York.
• 3. Dufour, F. and Piunovskiy, A. (2010) Multiobjective stopping problem for discrete-time Markov processes: the convex analytic approach. Journal of Applied Probability 47: 947–966.
• 4. Dynkin, E.B. and Yushkevich A.A. (1979) Controlled Markov Processes and their Applications. Springer-Verlag, New York - Berlin.
• 5. Easley, D. and Kiefer, N.M. (1988) Controlling a stochastic process with unknown parameters. Econometrica 56: 1045–1064.
• 6. Ekstrőm, E. and Lu, B. (2011) Optimal selling of an asset under incomplete information. Int. J. Stoch. Anal. Art. ID 543590, 17 ,2090–3340.
• 7. Ferguson, T.S. (1967) Mathematical Statistics. Academic Press, New York - London.
• 8. González-Trejo, J.I., Hernández-Lerma, O. and Hoyos-Reyes, L.F. (2003) Minimax control of discrete–time stochastic systems. SIAM J. Control Optim 41: 1626–1659.
• 9. DeGroot, M.H. (1970) Optimal Statistical Decisions. McGraw-Hill Book Co., New York.
• 10. van Hee, K.M. (1978) Bayesian Control of Markov Chains. Mathematical Centre Tracts, No. 95. Mathematisch Centrum, Amsterdam.
• 11. Hernández-Lerma, O. and Marcus, S.I. (1985) Adaptive control of discounted Markov decision chains. Journal of Optimization Theory and Applications 46: 227–235.
• 12. Hernández-Lerma, O. (1989) Adaptive Markov Control Processes, volume 79 of Applied Mathematical Sciences. Springer-Verlag, New York.
• 13. Hernández-Lerma, O. and Lasserre, J.B. (1996) Discrete-time Markov Control Processes. Springer, New York.
• 14. Hordijk, A. (1974) Dynamic Programming and Markov Potential Theory. Mathematical Centre Tracts, No. 51. Mathematisch Centrum, Amsterdam.
• 15. Horiguchi, M. (2001a) Markov decision processes with a stopping time constraint. Mathematical Methods of Operations Research 53: 279–295.
• 16. Horiguchi, M. (2001b) Stopped Markov decision processes with multiple constraints. Mathematical Methods of Operations Research 54: 455–469.
• 17. Kurano, M. (1972) Discrete-time Markovian decision processes with an unknown parameter. Average return criterion. Journal of the Operations Research Society of Japan 15: 67–76.
• 18. Kurano, M. (1983) Adaptive policies in Markov decision processes with uncertain transition matrices. Journal of Information & Optimization Sciences 4: 21–40.
• 19. Mandl, P. (1974) Estimation and control in Markov chains. Advances in Applied Probability 6: 40–60.
• 20. Martin, J.J. (1967) Bayesian Decision Problems and Markov Chains. Publications in Operations Research, No. 13. John Wiley & Sons Inc., New York.
• 21. Piunovskiy, A. B. (2006) Dynamic programming in constrained Markov decision processes. Control and Cybernetics 35: 645–660.
• 22. Puterman, M. (1994) Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York.
• 23. Raiffa, H and Schlaifer, R. (1961) Applied Statistical Decision Theory. Studies in Managerial Economics. Division of Research, Graduate School of Business Administration, Harvard University, Boston, Mass.
• 24. Rieder, U. (1975) Bayesian Dynamic Programming. Advances in Applied Probability 7: 330–348.
• 25. Ross, S.M. (1970) Applied Probability Models with Optimization Applications. Holden-Day, San Francisco.
• 26. Ross, S.M. (1983) Introduction to Stochastic Dynamic Programming. Academic Press, San Diego, CA.
• 27. Stadje, W. (1997) An optimal stopping problem with two levels of incomlete information. Mathem. Methods of Oper. Research 45: 119–131.
• 28. Wald, A. (1950) Statistical Decision Functions. John Wiley & Sons Inc., New York.
• 29. Wang, X. and Yi, Y. (2009) An optimal investment and consumption model with stochastic returns. Applied Stoch. Models in Business and Industry 25: 45–55.
• 30. White, D.J. (1969) Dynamic Programming. Mathematical Economic Texts,1. Oliver&Boyd, Edinburgh–London.