Likelihood and depth-based criteria for validation of numerical simulators, from comparison with experimental data

Amandine Marrel (1, 2) , Héloise Velardo (1, 2) , Antoine Bouloré (3, 4)

1 IRESNE - Institut de recherche sur les systèmes nucléaires pour la production d'énergie bas carbone (CEA - DES)
2 CEA Cadarache
3 LSC - Laboratoire de Simulation du Comportement des Combustibles
4 CEA-DES (ex-DEN) - CEA-Direction des Energies (ex-Direction de l'Energie Nucléaire)

Within the framework of Best-Estimate-Plus-Uncertainty approaches, the assessment of model parameter uncertainties, associated with numerical simulators, is a key element in safety analysis. The results (or outputs) of the simulation should be compared and validated against experimental values, when such data is available. This validation step is required to ensure a reliable use of the simulator for modeling and prediction. In addition, it must take into account both model and experimental uncertainties (measurement uncertainties). This work aims to define quantitative criteria to carry out this validation for multivariate outputs, while taking into account the different sources of uncertainty. For this purpose, different statistical indicators, based on likelihood or statistical depths, are investigated and extended to the multidimensional case. First, the properties of the criteria are studied, either analytically or by simulation, for some specific cases (Gaussian distribution for experimental uncertainties, identical distributions of experiments and simulations, particular discrepancies). Then, some natural extensions to multivariate outputs are proposed, with guidelines for practical use depending on the objectives of the validation (strict/hard or average validation). From this, transformed criteria are proposed to make them more comparable and less sensitive to the dimension of the output. It is shown that these transformations allow for a fairer and more relevant comparison and interpretation of the different criteria. Finally, these criteria are applied to a code dedicated to nuclear material behaviour simulation. The need to reduce the uncertainty of the model parameters is thus highlighted, as well as the outputs on which to focus.

Mots clés: Uncertainty quantification; Model validation; Statistical criteria; Experimental results; Likelihood; Depth statistics; Multivariate output

Domaines: Statistiques [math.ST]

Amandine Marrel, Héloise Velardo, Antoine Bouloré. Likelihood and depth-based criteria for validation of numerical simulators, from comparison with experimental data. 2023. ⟨cea-04020960v2⟩