Millimeter Wave Beamforming Training : A Reinforcement Learning Approach

• Autorzy: Mohamed Ehab Mahmoud

Identyfikatory

DOI

10.24425/ijet.2021.135949

• Języki publikacji: EN

Abstrakty

EN

Beamforming training (BT) is considered as an essential process to accomplish the communications in the millimeter wave (mmWave) band, i.e., 30 ~ 300 GHz. This process aims to find out the best transmit/receive antenna beams to compensate the impairments of the mmWave channel and successfully establish the mmWave link. Typically, the mmWave BT process is highly-time consuming affecting the overall throughput and energy consumption of the mmWave link establishment. In this paper, a machine learning (ML) approach, specifically reinforcement learning (RL), is utilized for enabling the mmWave BT process by modeling it as a multi-armed bandit (MAB) problem with the aim of maximizing the long-term throughput of the constructed mmWave link. Based on this formulation, MAB algorithms such as upper confidence bound (UCB), Thompson sampling (TS), epsilon-greedy (e-greedy), are utilized to address the problem and accomplish the mmWave BT process. Numerical simulations confirm the superior performance of the proposed MAB approach over the existing mmWave BT techniques.

Słowa kluczowe

EN

millimeter wave; beamforming training; multiarmed bandit; reinforcement learning

• Wydawca: Polish Academy of Sciences, Committee of Electronics and Telecommunication
• Czasopismo: International Journal of Electronics and Telecommunications
• Rocznik: 2021
• Tom: Vol. 67, No. 1
• Strony: 95--102
• Opis fizyczny: Bibliogr. 28 poz., rys., schem., wykr.

Twórcy

autor

Mohamed Ehab Mahmoud

• 1) Electrical Engineering Dept., College of Engineering, Prince Sattam Bin Abdulaziz University, Wadi Aldwaser 11991, Saudi Arabia
• 2) Electrical Engineering Dept., Faculty of Engineering Aswan University, Aswan 81542, Egypt

Bibliografia

• [1] S. Andreev, V. Petrov, M. Dohler, and H. Yanikomeroglu, “Future of Ultra-Dense Networks Beyond 5G: Harnessing Heterogeneous Moving Cells” IEEE Communications Magazine, vol. 57, no. 16, pp. 86-92, 2019.
• [2] T. S. Rappaport, et al., “Wireless Communications and Applications Above 100 GHz: Opportunities and Challenges for 6G and Beyond” IEEE ACCESS, vol. 7, pp. 78729-78757, 2019.
• [3] Z. Chen, et al., “A Survey on Terahertz Communications,” China Communications, vol. 16, no. 2 pp. 1673-5547, 2019.
• [4] E. M. Mohamed, M. A. Abdelghany, and M Zareei “An Efficient Paradigm for Multiband WiGig D2D Networks,” IEEE ACCESS, vol. 7, pp. 70032-70045, 2019.
• [5] E. M. Mohamed, et al., “Relay Probing for Millimeter Wave Multi-Hop D2D Networks,” IEEE ACEESS, vol. 8, pp. 30560-30574, 2020.
• [6] T. S. Rappaport, et al., “Broadband millimeter-wave propagation measurements and models using adaptive-beam antennas for outdoor urban cellular communications,” IEEE Trans. on Antenn. and Propag., vol. 61, no. 4, pp. 1850-1859, 2013.
• [7] T. S. Rappaport, et al., “Overview of millimeter wave communications for fifth-generation (5G) wireless networks—With a focus on propagation models,” IEEE Trans. on Antenn. and Propag., vol. 65, no. 12, pp. 6213-6230, 2017.
• [8] T. Bai, R. Vaze, and R. W. Heath, Jr., “Analysis of Blockage Effects on Urban Cellular Networks,” IEEE Transc. On Wirele. Commun., vol. 13, no. 9, pp. 5070-5083, 2014.
• [9] A. Abdelreheem, E. M. Mohamed, and H. Esmaiel, “Location-based millimeter wave multi-level beamforming using compressive sensing,” IEEE Commun. Lett., vol. 22, pp. 185-188, 2018.
• [10] E. M. Mohamed, K. Sakaguchi, and S. Sampei, “Wi-Fi coordinated WiGig concurrent transmissions in random access scenarios,” IEEE Trans. on Vehi. Techn., vol. 66, no. 11, pp. 10357-10371, 2017.
• [11] A. Alkhateeb, O. El Ayach, G. Leus, and R. W. Heath, “Channel estimation and hybrid precoding for millimeter wave cellular systems,” IEEE Journal of Selec. Topics in Signal Process., vol. 8, pp. 831-846, 2014.
• [12] I. Ahmed, et al., “A survey on hybrid beamforming techniques in 5g: Architecture and system model perspectives,” IEEE Communic. Surv. Tut., vol. 20, no. 4, pp. 3060–3097, Fourthquarter 2018.
• [13] IEEE 802.11ad Standard, “Enhancements for very high throughput in the 60 GHz band,” ed, Dec. 2012.
• [14] C. Jiang, et al, “Machine Learning Paradigms for Next-Generation Wireless Networks” IEEE Wireless Communications, vol. 24, no. 2, pp. 98-105, 2017.
• [15] J. Wang, et al., “Thirty Years of Machine Learning: The Road to Pareto Optimal Wireless Networks” IEEE Communic. Surv. Tut. (early access) 2020.
• [16] Hur, S., Kim, T., Love, D. J., et al.: ‘Millimeter wave beamforming for wireless backhaul and access in small cell networks’, IEEE Trans. on Commun., vol. 61, no. 10, pp. 4391–4403. 2013.
• [17] B. Li, Z. Zhou, H. Zhang, and A. Nallanathan, “Efficient beamforming training for 60-GHz millimeter-wave communications: A novel numerical optimization framework,” IEEE Trans. Veh. Technol., vol. 63, no. 2, pp. 703–717, 2014.
• [18] Gao, X., Dai, L., Yuen, C., Wang, Z.: ‘Turbo-like beamforming based on tabu search algorithm for millimeter-wave massive MIMO systems’, IEEE Trans. On Vehi. Techn., vol. 65, no. 7, pp. 5731–5737, 2016.
• [19] A. Abdelreheem, E. M. Mohamed, H. Esmaiel, “Adaptive location-based millimetre wave beamforming using compressive sensing based channel estimation,” IET Communications, vol. 13, no. 9, pp. 1287-1296, 2019.
• [20] A. M. Nor and E. M. Mohamed, “Li-Fi Positioning for Efficient Millimeter Wave Beamforming Training in Indoor Environment,” Mobile Networks and Applications, vol. 24, no. 2, pp. 517-531, 2019.
• [21] M. B. Booth, V. Suresh, N. Michelusi, and D. J. Love, “Multi-Armed Bandit Beam Alignment and Tracking for Mobile Millimeter Wave Communications,” IEEE Commun. Lett., vol. 23, no. 7, pp.1244-1248, 2019.
• [22] I. Chafaa, E. V. Belmega, and M. Debbah, “Adversarial multi-armed bandit for mm Wave beam alignment with one-bit feedback,” in Proc. 12th EAI Int. Conf. Perform. Eval. Methodol. Tools, pp. 23–30, 2019.
• [23] V. Va, T. Shimizu, G. Bansal, and R. W. Heath, Jr., “Online learning for position-aided millimeter wave beam training,” IEEE Access, vol. 7, pp. 30507–30526, 2019.
• [24] IEEE 802.15.3c Part 15.3: ‘Wireless Medium Access Control (MAC) and Physical Layer (PHY) Specifications for High Rat Wireless Personal Area Neworks (WPANS) Amendment’, 2009.
• [25] Wang, J., Lan, Z., Sum, C., et al ‘Beamforming codebook design and performance evaluation for 60GHz wideband WPANs’. Proc. IEEE Vehi. Techn., Anchorage, 2009, pp. 1-6.
• [26] Zou, W., Cui, Z., Li, B., Zhou, Z., Hu, Y, ‘Beamforming codebook design and performance evaluation for 60GHz wireless communication’. Proc. International Symposium on Communications & Information Technologies (ISCIT), 2011, pp. 30-35.
• [27] S. Agrawal, and N. Goyal, “Further optimal regret bounds for thompson sampling,” in Artificial Intelligence and Statistics, pp. 99-107, 2013.
• [28] F. Wilhelmi, “Collaborative spatial reuse in wireless networks via selfish multi-armed bandits,” in Ad Hoc Networks, vol. 88, pp. 129-141, 2019.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2021).