CFC2023

Reinforcement learning-based optimisation of LES closure models

  • Beck, Andrea (University of Stuttgart)
  • Kurz, Marius (University of Stuttgart)

Please login to view abstract download link

Reinforcement learning (RL) is considered as the third learning paradigm, besides unsupervised and supervised learning . In RL, the training data is generated iter- atively by the ML method itself. In order to do so, the learning task is framed as a Markov Decision Process (MDP), which is solved by an optimal policy. This policy is either approximated directly or through the evaluation of a learned value action function. The learned policy represents the current control strategy for solving the MDP. Its parameters are updated through repeated sampling of the policy’s proposed action space through interaction with the environment of the MDP, which emits reward signals intermittently and estimating the gradient of the objective w.r.t these parameters This optimization of a model or control strategy within the context of a dynamical system makes the RL approach somewhat orthogonal to supervised learning (SL) in that no training samples need to be known a priori, only a definition of a meaningful reward (which could be a single scalar value) is necessary. RL algorithms have been successfully developed to solve complex strategic games, robotics and flow control. In this talk, I will present data-driven approaches to LES modeling for implicitly filtered high order discretizations based on reinforcement learning. Wheres supervised learning of the Reynolds force tensor based on non-local data can provide highly accurate results that provide higher a priori correlation than any existing closures, a posteriori stability remains an issue. I will give reasons for this and introduce reinforcement learning (RL) as an alternative optimization approach. Our initial experiments with this method suggest that is it much better suited to account for the uncertainties introduced by the numerical scheme and its induced filter form on the modeling task. For this coupled RL-DG framework, I will present discretization-aware model approaches for the LES equations and discuss the future potential of these solver-in-the-loop optimizations.