Evaluation of Radio Channel Utility using Epsilon-Greedy Action Selection

• Autorzy: Malon Krzysztof

Identyfikatory

DOI

10.26636/jtit.2021.153621

• Języki publikacji: EN

Abstrakty

EN

This paper presents an algorithm that supports the dynamic spectrum access process in cognitive radio networks by generating a sorted list of best radio channels or by identifying those frequency ranges that are not in use temporarily. The concept is based on the reinforcement learning technique named Q-learning. To evaluate the utility of individual radio channels, spectrum monitoring is performed. In the presented solution, the epsilon-greedy action selection method is used to indicate which channel should be monitored next. The article includes a description of the proposed algorithm, scenarios, metrics, and simulation results showing the correct operation of the approach relied upon to evaluate the utility of radio channels and the epsilon-greedy action selection method. Based on the performed tests, it is possible to determine algorithm parameters that should be used in this proposed deployment. The paper also presents a comparison of the results with two other action selection methods.

Słowa kluczowe

EN

cognitive radio; dynamic spectrum access; spectrum monitoring; machine learning; Q-learning

• Wydawca: Instytut Łączności - Państwowy Instytut Badawczy
• Czasopismo: Journal of Telecommunications and Information Technology
• Rocznik: 2021
• Tom: nr 3
• Strony: 10--17
• Opis fizyczny: Bibliogr. 16 poz., rys., tab.

Twórcy

autor

Malon Krzysztof

• Institute of Communications Systems, Faculty of Electronics, Military University of Technology, Warsaw, Poland

• https://orcid.org/0000-0001-9257-1166

Bibliografia

• [1] Wireless Innovation Forum, „Dynamic Spectrum Sharing Annual Report - 2014", Document WINNF-14-P-0001, version V0.2.16 [Online]. Available: https://www.wirelessinnovation.org/assets/work products/Reports/winnf-14-p-0001-v1.0%20dynamic%20spectrum%20sharing%20annual%20report%202014.pdf
• [2] M. A. McHenry, D. McCloskey, and G. Lane-Roberts, „New York City spectrum occupancy measurements", Shared Spectrum Company, 2005 [Online]. Available: http://www.sharedspectrum.com/wp-content/uploads/4 NSF NYC Report.pdf
• [3] Shared Spectrum Company, „General survey of radio frequency bands - 30 MHz to 3 GHz", 2010 [Online]. Available: https://www.sharedspectrum.com/wp-content/uploads/2021/01/2010 0923-General-Band-Survey-30MHz-to-3GHz.pdf
• [4] E. Biglieri, A. J. Goldsmith, L. J. Greenstein, N. B. Mandayam, and H. V. Poor, Principles of cognitive radio. Cambridge: Cambridge University Press, 2012 (ISBN: 9781139236850).
• [5] L. E. Doyle, Essentials of cognitive radio. Cambridge: Cambridge University Press, 2009 (ISBN: 9780511576577).
• [6] R. S. Sutton and A. G. Barto, Reinforcement learning: an introduction, second edition. Cambridge: The MIT Press, 2018 (ISBN: 9780262039246).
• [7] K-L. A. Yau, P. Komisarczuk, and P. D. Teal, „Applications of reinforcement learning to cognitive radio networks", in Proc. IEEE Int. Conf. on Commun. Workshops, Cape Town, South Africa, 2010, pp. 1-6 (DOI: 10.1109/ICCW.2010.5503970).
• [8] N. Morozs, T. Clarke, and D. Grace, „Distributed heuristically accelerated Q-learning for robust cognitive spectrum management In LTE cellular systems", IEEE Transac. on Mobile Comput., vol. 15, no. 4, pp. 817-825, 2016 (DOI: 10.1109/TMC.2015.2442529).
• [9] C. Claus and C. Boutilier, „The dynamics of reinforcement learning in cooperative multiagent systems", in Proc. of the fifteenth national/tenth Conf. on Artif. Intell./Innovat. Applicat. of Artif. Intell., Madison, WI, USA, 1998, pp. 746-752 [Online]. Available: https://www.aaai.org/Papers/AAAI/1998/AAAI98-106.pdf
• [10] M. Bkassiny, Y. Li, and S. K. Jayaweera, „A survey on machinelearning techniques in cognitive radios", IEEE Commun. Surveys & Tut., vol. 15, no. 3, pp. 1136-1159, 2013 (DOI: 10.1109/SURV.2012.100412.00017).
• [11] K. Malon, P. Skokowski, and J. .opatka, „Optimization of wireless sensor network deployment for electromagnetic situation monitoring", Int. J. of Microwave and Wireless Technol., vol. 10, no. 7, pp. 746-753, 2018 (DOI: 10.1017/S1759078718000211).
• [12] K. Malon, P. Skokowski, and J. .opatka, „Optimization of the MANET topology in urban area using redundant relay points", Int. Conf. on Military Commun. and Informat. Systems (ICMCIS), Warsaw, Poland, 2018, pp. 1-4 (DOI: 10.1109/ICMCIS.2018.8398720).
• [13] P. Skokowski, K. Malon, and J. .opatka, „Properties of centralized cooperative sensing in cognitive radio networks", in Proc. XI Conf. on Reconnaissance and Electron. Warfare Systems, J. .opatka, Eds. Int. Society for Optics and Photon., vol. 10418, pp. 54-62. Oªtarzew, Poland: SPIE, 2017 (DOI: 10.1117/12.2269996).
• [14] P. Skokowski, „Electromagnetic situation awareness building in ad hoc networks with cognitive nodes", Military University of Technology, Warsaw, Poland, 2021 (in Polish, in print).
• [15] K. Sithamparanathan and A. Giorgetti, Cognitive Radio Techniques. Artech House, 2012 (ISBN: 9781608072040).
• [16] ITU-R Report SM.2256-1, „Spectrum occupancy measurements and evaluation", Int. Telecommun. Union, 2016 [Online]. Available: https://www.itu.int/pub/R-REP-SM.2256.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2021).