Development of FP16 data/FP32 computation mixed-precision preprocessing for ill-conditioned matrices in multi-phase CFD simulations

Ina, Takuya ; Idomura, Yasuhiro   ; Imamura, Toshiyuki*; Yamashita, Susumu   ; Onodera, Naoyuki   

We have developed mixed-precision preprocessing for the preconditioned conjugate gradients (PCG) method in the multi-phase multi-component thermal-hydraulic code JUPITER. The preconditioner employs a hybrid mixed-precision approach which combines FP16 data and FP32 operations. The roundoff errors are reduced by converting FP16 data to FP32 on cache, holding the intermediate result in FP32, converting the final result to FP16, and returning it to the memory. The developed preconditioner was tested for large-scale problems with 3D structured grids of 3,2002,00014,160. The convergence of the PCG method was maintained even when the FP16 data format was used for ill-condition matrices, and the computational speed was dramatically increased by reducing the memory access. The hybrid FP16/32 mixed-precision implementation achieved 1.79 speedup from the FP64 implementation at 2,000 nodes on Fugaku.