Uncertainty analysis of model selection based on information criterion; A Case study of a probability estimation model for fuel cladding tube fracture during LOCA

Narukawa, Takafumi ; Udagawa, Yutaka  

Information criteria such as a widely applicable information criterion (WAIC) and a widely applicable Bayesian information criterion (WBIC) enable the selection of models with high predictive accuracy and data fit, yet these criteria come with inherent uncertainties as they are statistical measures. To evaluate the uncertainty in model selection based on these information criteria, we performed numerical experiments using the bootstrap method, which is a resampling technique, on models for estimating the fracture probability of fuel cladding tubes during loss-of-coolant accidents (LOCAs). By calculating WAIC and WBIC for each of 10,000 bootstrap samples, we evaluated the dependency of model selection on these samples. Our key findings reveal that: (1) Sample-derived variation in information criteria was significantly greater than variability between models, underscoring the importance of assessing uncertainty from samples. (2) The Log-probit model, developed in our previous study, was selected as the optimal model for its superior predictive performance and data fit, despite the inherent uncertainties associated with WAIC and WBIC. (3) The presence of outliers at the fracture/non-fracture boundary of fuel cladding tubes may negatively impact the information criteria, suggesting the need for careful consideration when including such data in model parameter estimation.