GPU-acceleration of locally mesh allocated two phase flow solver for nuclear reactors

Onodera, Naoyuki; Idomura, Yasuhiro; Ali, Y.*; Yamashita, Susumu; Shimokawabe, Takashi*; Aoki, Takayuki*

Proceedings of Joint International Conference on Supercomputing in Nuclear Applications + Monte Carlo 2020 (SNA + MC 2020), p.210 - 215, 2020/10

This paper presents a GPU-based Poisson solver on a block-based adaptive mesh refinement (block-AMR) framework. The block-AMR method is essential for GPU computation and efficient description of the nuclear reactor. In this paper, we successfully implement a conjugate gradient method with a state-of-the-art multi-grid preconditioner (MG-CG) on the block-AMR framework. GPU kernel performance was measured on the GPU-based supercomputer TSUBAME3.0. The bandwidth of a vector-vector sum, a matrix-vector product, and a dot product in the CG kernel gave good performance at about 60% of the peak performance. In the MG kernel, the smoothers in a three-stage V-cycle MG method are implemented using a mixed precision RB-SOR method, which also gave good performance. For a large-scale Poisson problem with cells, the developed MG-CG method reduced the number of iterations to less than 30% and achieved 2.5 speedup compared with the original preconditioned CG method.

Communication avoiding multigrid preconditioned conjugate gradient method for extreme scale multiphase CFD simulations

Idomura, Yasuhiro; Onodera, Naoyuki; Yamada, Susumu; Yamashita, Susumu; Ina, Takuya*; Imamura, Toshiyuki*

Supa Kompyuthingu Nyusu, 22(5), p.18 - 29, 2020/09

A communication avoiding multigrid preconditioned conjugate gradient method (CAMGCG) is applied to the pressure Poisson equation in a multiphase CFD code JUPITER, and its computational performance and convergence property are compared against the conventional Krylov methods. The CAMGCG solver has robust convergence properties regardless of the problem size, and shows both communication reduction and convergence improvement, leading to higher performance gain than CA Krylov solvers, which achieve only the former. The CAMGCG solver is applied to extreme scale multiphase CFD simulations with 90 billion DOFs, and its performance is compared against the preconditioned CG solver. In this benchmark, the number of iterations is reduced to , and speedup is achieved with keeping excellent strong scaling up to 8,000 nodes on the Oakforest-PACS.

Locally mesh-refined lattice Boltzmann method for fuel debris air cooling analysis on GPU supercomputer

Onodera, Naoyuki; Idomura, Yasuhiro; Uesawa, Shinichiro; Yamashita, Susumu; Yoshida, Hiroyuki

Mechanical Engineering Journal (Internet), 7(3), p.19-00531_1 - 19-00531_10, 2020/06

https://doi.org/10.1299/mej.19-00531

A dry method is one of practical methods for decommissioning the TEPCO's Fukushima Daiichi Nuclear Power Station. Japan Atomic Energy Agency (JAEA) has been evaluating the air cooling performance of the fuel debris by using the JUPITER code based on an incompressible fluid model and the CityLBM code based on the lattice Boltzmann method (LBM). However, these codes were based on a uniform Cartesian grid system, and required large computational time and cost to capture complicated debris structures. We develop an adaptive mesh refinement (AMR) version of the CityLBM code on GPU based supercomputers and apply it to thermal-hydrodynamics problems. The proposed method is validated against free convective heat transfer experiments at JAEA. It is also shown that the AMR based CityLBM code on 4 NVIDIA TESLA V100GPUs gives 6.7x speedup of the time to solution compared with the JUPITER code on 36 Intel Xeon E5-2680v3 CPUs.

Development of numerical simulation method to evaluate detailed thermal-hydraulics around beam window in ADS

Yamashita, Susumu; Sugawara, Takanori; Yoshida, Hiroyuki

Proceedings of 27th International Conference on Nuclear Engineering (ICONE-27) (Internet), 7 Pages, 2019/12

https://doi.org/10.1299/jsmeicone.2019.27.2019

In order to simulate detailed flow behavior of LBE around the beam window, a numerical simulation code that can evaluate the complicated flow behavior is required. To simulate complicated and large-scale flow behavior, we apply JUPITER which originally developed in JAEA for melt relocation behavior in SAs and that can treat complicated flow behavior and has a capacity of massively parallel computing. In this paper, by using JUPITER, numerical simulations were performed for unsteady thermal-hydraulics simulation around the beam window to know tendency of LBE flow field. In addition, problems to be solved and important parameters to simulate thermal-hydraulic behavior around the beam window will be discussed.

Fuel debris' air cooling analysis using a lattice Boltzmann method

Onodera, Naoyuki; Idomura, Yasuhiro; Kawamura, Takuma; Uesawa, Shinichiro; Yamashita, Susumu; Yoshida, Hiroyuki

Proceedings of 27th International Conference on Nuclear Engineering (ICONE-27) (Internet), 6 Pages, 2019/05

A dry method is one of practical methods for decommissioning the TEPCO's Fukushima Daiichi Nuclear Power Station. Japan Atomic Energy Agency (JAEA) has been evaluating the air cooling performance by using the JUPITER code. However, the JUPITER code requires a large computational cost to capture debris' structures. To accelerate such CFD analyses, we use the CityLBM code, which is based on the lattice Boltzmann method (LBM) and is highly optimized for GPUs. The CityLBM code is validated against free convective heat transfer experiments at JAEA, and the similar accuracy as the JUPITER code is confirmed regarding the prediction capability of heat transfer and the resulting temperature distributions. It is also shown that the elapse time of a CityLBM simulation on GPUs is reduced to 1/6 compared with that of the corresponding JUPITER simulation on CPUs with the same number of GPUs and CPUs. The results show that the LBM is promising for accelerating thermal convective simulations.

Coupled analysis of fuel debris distribution and recriticality by both multiphase/multicomponent flow and continuous energy neutron transport Monte Carlo simulations

Yamashita, Susumu; Tada, Kenichi; Yoshida, Hiroyuki; Suyama, Kenya

Nippon Genshiryoku Gakkai Wabun Rombunshi, 17(3/4), p.99 - 105, 2018/12

https://doi.org/10.3327/taesj.J17.026

In order to reveal melt relocation behaviors of core internals phenomenologically and to reduce the uncertainties of the melt relocation analysis in existing SA analysis codes, in JAEA, the numerical simulation code for melt relocation and accumulation behaviors based on computational fluid dynamics named JUPITER has been developed. In this paper, to consider the estimation method for fuel debris composition and its re-criticality, we performed the melt accumulating and spreading simulation to the pedestal region by JUPITER and also performed re-criticality analysis by Monte Carlo Codes for Neutron Transport Calculations based on Continuous Energy and Multi-group Methods (MVP) using detailed fuel debris composition data obtained by JUPITER. From the coupled analysis on fuel debris distribution by JUPITER and MVP, we had prospects for a detailed possibility of re-criticality of fuel debris with detailed fuel debris distribution.

Numerical simulation of thermal hydraulics around a beam window in accelerator-driven system

Yamashita, Susumu; Yoshida, Hiroyuki

Proceedings of 11th Korea-Japan Symposium on Nuclear Thermal Hydraulics and Safety (NTHAS-11) (Internet), 5 Pages, 2018/11

To investigate detailed flow behaviors around the beam window of accelerator driven system (ADS), large scale simulation for unsteady thermal hydraulics around the beam window was performed using JUPITER. The input data, such as the beam window and nozzle, is designed as accurate as possible using the computer aided design software. As a result, the flow pattern around the beam window is quite different from previous results in which the steady flow is assumed. The flow pattern of the Lead-Bismuth Eutectic around the beam window and the exit of the nozzle were very complicated. According to complicated flow around those structures, the temperature distribution on the beam window is also complicated. Thus, it is confirmed that the complicated flow around structures will affect to the temperature distribution in the structures and the effect of flow field on the temperature must be evaluated.

Development of numerical simulation method to evaluate molten material behaviors in nuclear reactors

Yamashita, Susumu; Yoshida, Hiroyuki

Proceedings of 12th International Topical Meeting on Nuclear Reactor Thermal-Hydraulics, Operation and Safety (NUTHOS-12) (USB Flash Drive), 12 Pages, 2018/10

In order to simulate the relocation phenomena phenomenologically around the reactor core and inside pedestal without assumptions, inputted information, empirical knowledge and given scenarios, a numerical simulation code based on computational fluid dynamics, JUPITER, that can phenomenologically evaluate the melting phenomena has been developed in JAEA. We confirm the applicability of JUPITER to the corium spreading process inside the pedestal by simulating corium spreading behaviors and its distributions under several parameters such as the corium inflow condition. And we investigated detailed fuel debris distribution inside the pedestal and the cavity named sump pit located on the lower part of PCVs, that is distribution for each component of core materials such as stainless steel (SUS), bron carbide (BC), zircaloy (Zry) and uranium dioxide (UO). As a result, since JUPITER uses interface capturing scheme which can treat complicated behavior of interfaces such as a large deformation and a complicated separation among interfaces, the corium was spread with complicated mixing behavior and the distribution of fuel debris tend to accumulate complicatedly inside sump pits. Since existing SA analysis codes difficult to treat such complicated phenomena and the complicated fuel distribution, those result might be contributed to an understanding of circumstances inside PCV and, ultimately, also contributed to reactor decommissioning process.

Development of numerical simulation method to evaluate molten material behaviors in nuclear reactors; Estimation of fuel debris distribution in the pedestal

Yamashita, Susumu; Yoshida, Hiroyuki

Proceedings of 26th International Conference on Nuclear Engineering (ICONE-26) (Internet), 6 Pages, 2018/07

A Numerical simulation method for molten material behavior in nuclear reactors

Yamashita, Susumu; Ina, Takuya; Idomura, Yasuhiro; Yoshida, Hiroyuki

Nuclear Engineering and Design, 322, p.301 - 312, 2017/10

https://doi.org/10.1016/j.nucengdes.2017.06.032

In recent years, significant attention has been paid to the precise determination of relocation of molten materials in reactor pressure vessels of boiling water reactors (BWRs) during severe accidents. To address this problem, we have developed a computational fluid dynamics code JUPITER, based on thermal-hydraulic equations and multi-phase simulation models. Although the Poisson solver has previously been a performance bottleneck in the JUPITER code, this is resolved by a new hybrid parallel Poisson solver, whose strong scaling is extended up to 200k cores on the K-computer. As a result of the improved computational capability, the problem size and physical models are dramatically expanded. A series of verification and validation studies are enabled, which are in agreement with previous numerical simulations and experiments. These physical and computational capabilities of JUPITER enable us to investigate molten material behaviors in reactor relevant situations.