# Highly adaptive linear actor-critic for lightweight energy-harvesting IoT applications

1 LFIM - Laboratoire Fonctions Innovantes pour circuits Mixtes
UGA - Université Grenoble Alpes, DSCIN - Département Systèmes et Circuits Intégrés Numériques : DRT/LIST/DSCIN
Abstract : Reinforcement learning (RL) has received much attention in recent years due to its adaptability to unpredictable events such as harvested energy and workload, especially in the context of edge computing for Internet-of-Things (IoT) nodes. Due to limited resources in IoT nodes, it is difficult to achieve self-adaptability. This paper studies online reactivity issues of fixed learning rate in the linear actor-critic (LAC) algorithm for transmission duty-cycle control. We propose the LAC-AB algorithm that introduces into the LAC algorithm an adaptive learning rate called Adam for actor update to achieve better adaptability. We introduce a definition of “convergence” when quantitative analysis of convergence is performed. Simulation results using real-life one-year solar irradiance data indicate that, unlike the conventional setups of two decay rate $\beta_1$,$\beta_2$ of Adam, smaller $\beta_1$ such as 0.2–0.4 are suitable for power-failure-sensitive applications and 0.5–0.7 for latency-sensitive applications with $\beta_2$ $\in$ [0.1,0.3]. LAC-AB improves the time of reactivity by 68.5–88.1% in our application; it also fine-tunes the initial learning rate for the initial state and improves the time of fine-tuning by 78.2–84.3%, compared to the LAC. Besides, the number of power failures is drastically reduced to zero or a few occurrences over 300 simulations.
Keywords :
Document type :
Journal articles

https://hal-cea.archives-ouvertes.fr/cea-03534312
Contributor : Jean-Frédéric CHRISTMANN Connect in order to contact the contributor
Submitted on : Wednesday, January 19, 2022 - 1:03:46 PM
Last modification on : Thursday, February 17, 2022 - 10:08:06 AM
Long-term archiving on: : Wednesday, April 20, 2022 - 6:37:56 PM

### File

jlpea-1142703-english.pdf
Publisher files allowed on an open archive

### Citation

Jean-Frédéric Christmann, Sota Sawaguchi, Suzanne Lesecq. Highly adaptive linear actor-critic for lightweight energy-harvesting IoT applications. Journal of Low Power Electronics and Applications, MDPI, 2021, 11 (2), pp.17. ⟨10.3390/jlpea11020017⟩. ⟨cea-03534312⟩

Record views