Application of the reinforcement learning for selecting fuzzy rules representing the behavior policy of units in RTS-type games

• Autorzy: Kutyło M , Pluciński M. , Laskowska M

Warianty tytułu

PL

Zastosowanie uczenia ze wzmocnieniem w doborze reguł rozmytych reprezentujących strategię zachowań jednostek w grach typu RTS

• Języki publikacji: EN

Abstrakty

EN

The aim of the presented research was to prove the feasibility of the fuzzy modeling employing in combination with the reinforcement learning, in the process of designing an artificial intelligence that effectively controls the behavior of agents in the RTS-type computer game. It was achieved by implementing a testing environment for “StarCraft”, a widely popular RTS game. The testing environment was focused on a single test-scenario, which was used to explore the behavior of the fuzzy logic-based AI. The fuzzy model’s parameters were adjustable, and a Q-learning algorithm was applied to perform such adjustments in each learning cycle.

PL

W artykule przedstawiono badania możliwości połączenia modelowania rozmytego z uczeniem ze wzmocnieniem w procesie projektowania inteligentnego algorytmu, który będzie efektywnie kontrolował zachowanie agentów w grze typu RTS. Aby osiągnąć założony cel, zaimplementowano testowe środowisko w popularnej grze RTS „StarCraft”. W środowisku tym realizowano jeden założony scenariusz gry, w którym badano zachowanie opracowanego algorytmu rozmytego. Parametry modelu rozmytego były modyfikowane za pomocą metody Q-learning.

Słowa kluczowe

EN

reinforcement learning; Q-learning; fuzzy model; RTS game

PL

uczenie ze wzmocnieniem; algorytm Q-learning; model rozmyty; gra RTS

• Wydawca: Wydawnictwo SIGMA-NOT
• Czasopismo: Przegląd Elektrotechniczny
• Rocznik: 2015
• Tom: R. 91, nr 2
• Strony: 142--146
• Opis fizyczny: Bibliogr. 19 poz., rys., tab., wykr.

Twórcy

autor

Kutyło M

• Faculty of Computer Science and Information Technology, West Pomeranian University of Technology, Z˙ołnierska 49, 71-062 Szczecin, Poland

autor

Pluciński M.

• Faculty of Computer Science and Information Technology, West Pomeranian University of Technology, Z˙ołnierska 49, 71-062 Szczecin, Poland

autor

Laskowska M

• Faculty of Computer Science and Information Technology, West Pomeranian University of Technology, Z˙ołnierska 49, 71-062 Szczecin, Poland

Bibliografia

• [1] Mendel J.M.: Uncertain rule-based fuzzy logic systems: introduction and new directions, Prentice Hall PTR, 2001.
• [2] Piegat A.: Fuzzy modeling and control Physica Verlag, Heidelberg – New York, 2001.
• [3] Kutyło M.: Application of the reinforcement learning for the selection of fuzzy rules representing the policy, in the software production process of the RTS-game, Eng. Thesis, West Pomeranian University of Technology, 2011, (in polish).
• [4] Pluciński M.: Application of the probabilistic RBF neural network in the reinforcement learning of a mobile robot, Polish Journal of Environmental Studies, 16(5B), pp. 32-37, 2007.
• [5] Sutton R.S., Barto A.G.: Reinforcement learning: An introduction, The MIT Press, 1998.
• [6] Er M.J., Deng C.: Online tuning of fuzzy inference systems using dynamic fuzzy Q-learning, IEEE Transactions on Systems, Man and Cybernetics – part B: Cybernetics, 34(3), pp. 1478-1489, 2004.
• [7] Glorennec P.Y., Jouffe J.: Fuzzy Q-learning, Proceedings of the 6th IEEE International Conference on Fuzzy Systems, vol. 2, pp. 659-662, 1997.
• [8] Lin L.J.: Hierarchical learning of robot skills by reinforcement, Proceedings of the IEEE International Conference on Neural Networks, pp. 181-186, 1993.
• [9] Lin C.K.: A Reinforcement learning adaptive fuzzy controller for robots, Fuzzy Sets and Systems, 137(3), pp. 339-352, 2003.
• [10] Maeda Y.: Modified Q-learning method with fuzzy state division and adaptive rewards, Proceedings of the IEEE World Congress on Computational Intelligence, pp. 1556-1561, 2002.
• [11] Pluciński M.: Application of the reinforcement learning in searching of the exploration policy for many vehicles, Polish Journal of Environmental Studies, 17(3B), pp. 347-352, 2008.
• [12] Sutton R.S.: Learning to predict by the methods of temporal differences, Machine Learning, 3, pp. 9-44, 1992.
• [13] Tesauro G.: Practical issues in temporal differences learning, Machine Learning, 8, pp. 257-277, 1992.
• [14] Bekey G.E., Autonomous robots (from biological inspiration to implementation and control), The MIT Press, 2005.
• [15] Connell J., Mahadevan S.: Rapid task learning for real robots, Robot Learning, Springer US, pp. 105-139, 1993.
• [16] Kaelbling L.P., Littman M.L., Moore A.W.: Reinforcement learning: A survey, Journal of Artificial Intelligence Research, 4, pp. 237-285, 1996.
• [17] Millan J.R.: Rapid, safe, and incremental learning of navigation strategies, IEEE Transactions on Systems, Man, and Cybernetics, part B.: Cybernetics, 26(3), pp. 408-420, 1996.
• [18] Cichosz P.: Learning systems, Wyd. Naukowo-Techniczne, Warszawa, 2000, (in polish).
• [19] BWAPI, Available at: http://code.google.com/p/bwapi/wiki/BWAPIManual.