Risk sensitive adaptive control of discrete time Markov processes

• Autorzy: Duncan T. E. , Pasik-Duncan B. , Stettner Ł.
• Języki publikacji: EN

Abstrakty

EN

Adaptive control of discrete time Markov processes with an infinite horizon risk sensitive cost functional is investigated. The continuity of the optimal risk sensitive cost with respect to a parameter of the transition probability is verified. Two almost optimal adaptive procedures that are based on the large deviations of the cost functional and discretized maximum likelihood estimates are given. To justify the performance of the adaptive procedure with observations of the cost, some large deviations estimates of the empirical distributions of finite sequences of successive states of Markov processes are obtained. A finite family of continuous control functions, where one control function is fixed after a nonrandom time from each of the adaptive procedures, provides an almost optimal adaptive control.

Słowa kluczowe

EN

adaptive control; risk sensitive control; discrete time controlled; Markov processes; ergodic control

• Wydawca: Wrocław University of Science and Technology
• Czasopismo: Probability and Mathematical Statistics
• Rocznik: 2001
• Tom: Vol. 21, Fasc. 2
• Strony: 493--512
• Opis fizyczny: Bibliogr. 8 poz.

Twórcy

autor

Duncan T. E.

• Department of Mathematics, University of Kansas Lawrence, KS 66045, U.S.A.

autor

Pasik-Duncan B.

• Department of Mathematics, University of Kansas Lawrence, KS 66045, U.S.A.

autor

Stettner Ł.

• Institute of Mathematics, Polish Academy of Sciences, Śniadeckich 8, 00-950 Warsaw, Poland

Bibliografia

• [1] Cl B. Di Masi and Ł. Stettner, Bayesian ergodic adaptive control of discrete time Markov processes, Stochastics Stochastic Rep. 54 (1995), pp. 301-316.
• [2] G. B. Di Masi and Ł. Stettner, Risk sensitive, control of discrete time Markov processes with infinite horizon, SIAM J. Control Optim. 38 (2000), pp. 61-78.
• [3] G. B. Di Masi and Ł. Stettner, Risk sensitive control of discrete time Markov processes with small risk, Systems Control Lett. 40 (2000), pp. 15-20.
• [4] M. D. Donskcr and S. R. S. Varadan, Asymptotic evaluation of certain Markov process expectations for large time. I, Comm, Pure Appl. Math. 28 (1975), pp. 1-47.
• [5] M. D. Donsker and S. R, S. Varadan, Asymptotic evaluation of certain Markov process expectations for large time. III, Comm, Pure Appl, Math. 28 (1976), pp. 384 461.
• [6] T. E. Duncan, B, Pasik-Duncan and Ł, Stettner, Discretized maximum likelihood and almost optimal adaptive control of ergodic Markov models, SIAM J. Control Optim. 36 (1998), pp. 422-446.
• [7] T. E, Duncan, B. Pasik-Duncan and Ł. Stettner, Adapiire control of discrete time Markov processes by the large deviation method, Appl. Math, 27 (3) (2000), pp. 265-285.
• [8] O. Hernandez-Lerma, Adaptive Control Processes, Springer-Verlag, New York-Berlin 1989.