Reinforcement Learning for Reconfigurable Intelligent Surfaces

This book presents the intersection of two dynamic fields: Reinforcement Learning (RL) and RIS- Assisted Wireless Communications. With an emphasis on both discrete and continuous problems, it introduces a comprehensive overview of RL techniques and their applications in the evolving world of RIS-assisted wireless communications. Chapter 1 introduces the fundamentals of RL and deep RL (DRL), providing a solid foundation for understanding subsequent chapters. It also presents the Q-learning, deep Q-learning, and deep deterministic policy gradient algorithms. Chapter 2 provides a holistic overview of RIS-assisted systems and details several use cases in wireless communications. Then, Chapters 3 and 4 present various applications of the discrete and continuous DRL to RIS-assisted wireless communications. From maximizing the sum-rate to minimizing, the system resources and maximizing the energy efficiency. These chapters showcase the versatility of the DRL algorithms in tackling arange of challenges. This book concludes with Chapter 5, which introduces the challenges and future directions in this field. It explores the particulars of hyperparameter tuning, problem design, and complexity analysis, while also highlighting the potential of hybrid DRL, multi-agent DRL, and transfer learning techniques for advancing wireless communication systems. Optimizing RIS-Assisted Wireless Systems requires powerful algorithms to cope with the dynamic propagation environment. DRL is envisioned as one of the key enabling techniques to exploit the full potential of RIS-Assisted Wireless Communication Systems. It empowers these systems to intelligently adapt to dynamic wireless environments, maximize performance metrics, and adjusts their configurations to accommodate diverse use cases efficiently. This book serves as a valuable resource, shedding light on the potential of DRL to optimize RIS-Assisted Wireless Communication, enabling researchers, engineers, advanced level students in computer science and electrical engineering and enthusiasts to grasp the intricacies of this topic. It offers a comprehensive understanding of the principles, applications, and challenges, making it a reference to recognize the full potential of the RIS technology in modern wireless communication systems.

Auteur

Alice Faisal received her B.Sc. degree in electrical and computer engineering from Effat University, Jeddah, Saudi Arabia, in 2020. She received her M.Sc. degree in electrical engineering from Memorial University, St. John's, NL, Canada, in 2022. Her research interests include wireless communications, 5G enabling technologies, reconfigurable intelligent surfaces, and full-duplex communications.Ibrahim Al-Nahhal (IEEE senior member) is a research associate and per-course instructor at Memorial University, Canada, since 2021. He received the B.Sc. (Honours), M.Sc., and Ph.D. degrees in Electronics and Communications Engineering from Al-Azhar University in Cairo, Egypt-Japan University for Science and Technology, Egypt, Memorial University, Canada, in 2007, 2014, and 2020, respectively. Between 2008 and 2012, he was an engineer in industry, and a Teaching Assistant at the Faculty of Engineering, Al-Azhar University in Cairo, Egypt. From 2014 to 2015, he was a physical layer expert atNokia (formerly Alcatel-Lucent), Belgium. He holds three patents. He co-authored 30+ peer-reviewed journals and conference papers in top-ranked venues. He serves as Editor of IEEE Wireless Communications Letters. He served as a Technical Program Committee and Reviewer for various prestigious journals and conferences. He was awarded the Exemplary Reviewer of IEEE Communications Letters in 2017. His research interests are reconfigurable intelligent surfaces, full-duplex communications, integrated sensing and communication, channel estimation, machine learning, design of low-complexity receivers for emerging technologies, spatial modulation, multiple-input multiple-output communications, sparse code multiple access, and optical communications.Octavia A. Dobre is a Professor and Canada Research Chair Tier 1 with Memorial University, Canada. She was a Visiting Professor with Massachusetts Institute of Technology, USA and Université de Bretagne Occidentale, France. Her research interests encompass wireless communication and networking technologies, as well as optical and underwater communications. She has (co-)authored over 450 refereed papers in these areas.Dr. Dobre serves as the Director of Journals of the Communications Society. She was the inaugural Editor-in-Chief (EiC) of the IEEE Open Journal of the Communications Society and the EiC of the IEEE Communications Letters.Dr. Dobre was a Fulbright Scholar, Royal Society Scholar, and Distinguished Lecturer of the IEEE Communications Society. She obtained Best Paper Awards at various conferences, including IEEE ICC, IEEE Globecom, IEEE WCNC, and IEEE PIMRC. Dr. Dobre is an elected member of the European Academy of Sciences and Arts, a Fellow of the Engineering Insitute of Canada, a Fellow of the Canadian Academy of Engineering, and a Fellow of the IEEE. Telex M. N. Ngatched is an Associate Professor at McMaster University, Canada. His research interests include 5G and 6Genabling technologies, optical wireless communications, hybrid optical wireless and radio frequency communications, and artificial intelligence and machine learning for communications. Dr. Ngatched serves as an Area Editor of the IEEE Open Journal of the Communications Society, an Associate Technical Editor of the IEEE Communications Magazine, an Editor of the IEEE Communications Society On-Line Content, and Co-Chair and Technical Program Committee member for many prominent IEEE conferences in his area of expertise.

Contenu

Preface.- Chapter. 1. Reinforcement Learning Background.- Chapter. 2. RIS-AssistedWireless Systems.- Chapter. 3. Applications of RL for Continuous Problems in RIS-Assisted Communication Systems.- Chapter. 4. Applications of RL for Discrete Problems in RIS-Assisted Communication Systems.- Chapter. 5. Challenges and Future Work.-

<p