Determining node duty cycle using Q-learning and linear regression for WSN

Abstract

Wireless sensor network (WSN) is effective for monitoring the target environment,which consists of a large number of sensor nodes of limited energy. An efficient medium access control (MAC) protocol is thus imperative to maximize the energy efficiency and performance of WSN. The most existing MAC protocols are based on the scheduling of sleep and active period of the nodes, and do not consider the relationship between the load condition and performance. In this paper a novel scheme is proposed to properly determine the duty cycle of the WSN nodes according to the load,which employs the Q-learning technique and function approximation with linear regression. This allows low-latency energy-efficient scheduling for a wide range of traffic conditions, and effectively overcomes the limitation of Q-learning with the problem of continuous state-action space. NS3 simulation reveals that the proposed scheme significantly improves the throughput, latency, and energy efficiency compared to the existing fully active scheme and S-MAC.

Keywords

wireless sensor network / media access control / duty-cycle scheduling / Q-learning / linear regression

References

[1] Alemdar A, Ibnkahla M. Wireless sensor networks: applications and challenges. In: Proceedings of the 9th International Symposium on Signal Processing and Its Applications. 2007, 1–6

[2] AlSkaif T, Bellalta B, Zapata M G, Ordinas J M B. Energy efficiency of MAC protocols in low data rate wireless multimedia sensor networks: a comparative study. Ad Hoc Networks, 2017, 56: 141–157

[3] Van Dam T, Langendoen K. An adaptive energy-efficient MAC protocol for wireless sensor networks. In: Proceedings of the 1st International Conference on Embedded Networked Sensor Systems. 2003, 171–180

[4] Zheng R, Kravets R. On-demand power management for ad hoc networks. Ad Hoc Networks, 2005, 3(1): 51–68

[5] Zheng R, Hou J C, Sha L. Asynchronous wakeup for ad hoc networks. In: Proceedings of the 4th ACM International Symposium on Mobile Ad Hoc Networking & Computing. 2003, 35–45

[6] Ye W, Heidemann J, Estrin D. An energy-efficient MAC protocol for wireless sensor networks. In: Proceedings of the 21st Annual Joint Conference of the IEEE Computer and Communications Societies. 2002, 1567–1576

[7] Ye W, Heidemann J, Estrin D. Medium access control with coordinated adaptive sleeping for wireless sensor networks. IEEE/ACM Transactions on Networking (ToN), 2004, 12(3): 493–506

[8] Jung E S, Vaidya N H. An energy efficient MAC protocol for wireless LANs. In: Proceedings of the 21st Annual Joint Conference of the IEEE Computer and Communications Societies. 2002, 1756–1764

[9] Liu S, Fan KW, Sinha P. CMAC: an energy-efficient MAC layer protocol using convergent packet forwarding for wireless sensor networks. ACM Transactions on Sensor Networks (TOSN), 2009, 5(4): 29

[10] Lewis F L, Vamvoudakis K G. Reinforcement learning for partially observable dynamic processes: adaptive dynamic programming using measured output data. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2011, 41(1): 14–25

[11] Kosunalp S, Chu Y, Mitchell P D, Grace D, Clarke T. Use of Q-learning approaches for practical medium access control in wireless sensor networks. Engineering Applications of Artificial Intelligence, 2016, 55: 146–154

[12] Polastre J, Hill J, Culler D. Versatile low power media access for wireless sensor networks. In: Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems. 2004, 95–107

[13] Du S, Saha A K, Johnson D B. RMAC: a routing-enhanced duty-cycle MAC protocol for wireless sensor networks. In: Proceedings of IEEE INFOCOM 2007 — the 26th IEEE International Conference on Computer Communications. 2007, 1478–1486

[14] Tong F, Tang W, Xie R, Shu L, Kim Y C. P-MAC: a cross-layer duty cycle MAC protocol towards pipelining for wireless sensor networks. In: Proceedings of IEEE International Conference on Communications (ICC). 2011, 1–5

[15] Lin P, Qiao C, Wang X. Medium access control with a dynamic duty cycle for sensor networks. In: Proceedings of IEEE Wireless Communications and Networking Conference. 2004, 1534–1539

[16] Polastre J, Hill J, Culler D. Versatile low power media access for wireless sensor networks. In: Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems. 2004, 95–107

[17] Buettner M, Yee G V, Anderson E, Han R. X-MAC: a short preamble MAC protocol for duty-cycled wireless sensor networks. In: Proceedings of the 4th International Conference on Embedded Networked Sensor Systems. 2006, 307–320

[18] Sun Y, Gurewitz O, Johnson D B. RI-MAC: a receiver-initiated asynchronous duty cycle MAC protocol for dynamic traffic loads in wireless sensor networks. In: Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems. 2008, 1–14

[19] Niu J, Deng Z. Distributed self-learning scheduling approach for wireless sensor network. Ad Hoc Networks, 2013, 11(4): 1276–1286

[20] Sutton R S, Barto A G. Reinforcement Learning: An Introduction. MIT Press, 2018

RIGHTS & PERMISSIONS

2020 Higher Education Press