Left-preconditioned communication-avoiding conjugate gradient methods for multiphase CFD simulations on the K computer

Mayumi, Akie; Idomura, Yasuhiro; Ina, Takuya; Yamada, Susumu; Imamura, Toshiyuki*

Proceedings of 7th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems (ScalA 2016) (Internet), p.17 - 24, 2016/11

https://doi.org/10.1109/ScalA.2016.007

The left-preconditioned communication avoiding conjugate gradient (LP-CA-CG) method is applied to the pressure Poisson equation in the multiphase CFD code JUPITER. The arithmetic intensity of the LP-CA-CG method is analyzed, and is dramatically improved by loop splitting for inner product operations and for three term recurrence operations. Two LP-CA-CG solvers with block Jacobi preconditioning and with underlap preconditioning are developed. It is shown that on the K computer, the LP-CA-CG solvers with block Jacobi preconditioning is faster, because the performance of local point-to-point communications scales well, and the convergence property becomes worse with underlap preconditioning. The LP-CA-CG solver shows good strong scaling up to 30,000 nodes, where the LP-CA-CG solver achieved higher performance than the original CG solver by reducing the cost of global collective communications by 69%.