The Rayleigh-B´enard convection originates from a temperature difference imposed across horizontal walls. In the turbulent regime, the flow is multi-structured and multi-scale, characterized by the interplay of small-scale plumes, a mean flow and turbulent fluctuations. The evolution of massively parallel direct numerical simulation (DNS) now allows to tackle calculations at regimes close to experiments. However, it remains difficult to statistically approach all flow scales, store them, or easily replay their sequences. Complementary experimental data despite being noisy and incomplete (probes time series, 2D fields / images), are often well converged and can reach high forcing. Considering the turbulent convection, many efforts have been done to infer hidden quantities. Recently, inference of the full continuous three-dimensional (3-D) velocity has been obtained by deploying physics-informed neural networks (PINNs) to 3-D temperature snapshots measured by a Schlieren technique [1]. Inspired by this approach, we have built reduced models to infer 3D flows from temperature DNS dataset [2]. The tracking of the large scales of the flow has been carried out by using the good segmentation properties of U-Net convolution networks [3]. In this work, we attempt to develop a non-invasive temperature field measurement technique, based on PIV measures and/or shadowgraph images. We compare auto-encoder and U-Net performances, and assess the improvement to add physical constraints. DNS data are used for preliminary tests with a future perspective of merging experimental and numerical databases. This work benefits from the French National Research Agency funding (THERMAL project).