Apprentissage profond par renforcement et optimisation pour l'estimation de l'état et la commande des systèmes non linéaires interconnectés sous contraintes

Keywords

Deep Reinforcement Learning, Interconnected Nonlinear Systems

Subject details

This doctoral thesis project is part of the scientific collaboration between CRAN - University of Lorraine and TIC Lab - International University of Rabat. It aims to develop a methodological approach based on AI techniques to address issues related to state estimation (or software sensors) and control of interconnected nonlinear systems under constraints, for which model-based approaches have not provided a solution to date. A broad class of dynamic systems, whether physical or artificial, can be modeled by interconnected nonlinear differential equations: Communication, transportation (Oudani et al. 2024) or energy networks (Thabet et al. 2010, Jadbabaie et al., 2020; Ren & Beard, 2020). These interconnected nonlinear systems often operate under significant constraints such as limited measurement capabilities, restricted communication bandwidth and/or limitations in terms of energy or computation, which negatively affect their observability and controllability performances (Mu et al., 2023; Li et al., 2023). These limitations are particularly problematic in decentralized and distributed systems, where state estimation must be performed with limited information exchange, while the presence of unknown inputs, external disturbances, and model uncertainties further complicates the design of robust observers and controllers (Boutat-Baddas et al., 2021; Bel Haj Frej et al., 2016, Kang et al., 2023; Narayanan & Jagannathan, 2017; Nejabat & Nikoofard, 2023). The use of hybrid approaches based on both models and Artificial Intelligence techniques opens up a very promising field but remains relatively unexplored. In a recent study (Yaghmaie et al. 2019) based on reinforcement learning technique, with specific activation functions, the authors show that there could be solutions to optimization problems under constraints for a class of linear dynamic systems, where the model based approach that has not provided a solution to date. Other recent research has emphasized the integration of machine learning techniques to enhance predictive accuracy, optimize decision-making under constraints, and develop AI driven control strategies capable of handling uncertainty and incomplete data scenarios (Liu et al., 2023). Particularly, reduced-order modeling techniques significantly reduce computational complexity, while reinforcement learning-based control strategies provide adaptive and decentralized decision-making mechanisms. It is clear that approaches based solely on models for estimating and controlling interconnected nonlinear systems under constraints quickly show their limitations and impose very restrictive conditions to ensure a minimum level of performance. Expected research should focus on developing more sophisticated learning frameworks, particularly through the integration of deep reinforcement learning. The first part of the thesis will focus on the development of a distributed approach to build deep neural networks for each subsystem of the interconnected model while minimizing a global criterion that considers all interactions in the network. The reward function would consider the global dynamic model and will be used to ensure minimum performance. The construction of these networks (universal approximators) will be inspired by a recent work (Farkane et al. 2024) that we have developed to solve a class of nonlinear PDEs (large scale systems, for which we obtained remarkable performance in terms of accuracy and speed compared to Eulerian discretization techniques). Construction of distributed strategies for reinforcement learning based state estimation and control design will focus on cases where: • Sensors of each subsystem are limited • Communication is restricted • Energy is limited The second part of the thesis, equally important, will focus on evaluating these AI-based algorithms in terms of robustness, accuracy, and computation time through numerical simulation of various types of interconnected networks under constraints.