Initialising ...
Initialising ...
Initialising ...
Initialising ...
Initialising ...
Initialising ...
Initialising ...
Asahi, Yuichi; Padioleau, T.*; Latu, G.*; Bigot, J.*; Grandgirard, V.*; Obrejan, K.*
Dai-36-Kai Suchi Ryutai Rikigaku Shimpojiumu Koen Rombunshu (Internet), 8 Pages, 2022/12
We implement a kinetic plasma simulation code with multiple performance portable frameworks and evaluated its performance on Intel Icelake, NVIDIA V100 and A100 GPUs, and AMD MI100 GPU. Relying on the language standard parallelism stdpar and proposed language standard multi-dimensional array support mdspan, we demonstrate a performance portable implementation without harming the readability and productivity. With stdpar, we obtain a good overall performance for a kinetic plasma mini-application in the range of 
20% to the Kokkos version on Icelake, V100, A100 and MI100. We conclude that stdpar can be a good candidate to develop a performance portable and productive code targeting Exascale era platforms, assuming this programming model will be available on AMD and/or Intel GPUs in the future.
Asahi, Yuichi; Padioleau, T.*; Latu, G.*; Bigot, J.*; Grandgirard, V.*; Obrejan, K.*
Proceedings of 2022 International Workshop on Performance, Portability, and Productivity in HPC (P3HPC) (Internet), p.68 - 80, 2022/11
Times Cited Count：0 Percentile：0(Computer Science, Theory & Methods)This paper presents the performance portable implementation of a kinetic plasma simulation code with C++ parallel algorithm to run across multiple CPUs and GPUs. Relying on the language standard parallelism stdpar and proposed language standard multi-dimensional array support mdspan, we demonstrate that a performance portable implementation is possible without harming the readability and productivity. We obtain a good overall performance for a mini-application in the range of 20% to the Kokkos version on Intel Icelake, NVIDIA V100, and A100 GPUs. Our conclusion is that stdpar can be a good candidate to develop a performance portable and productive code targeting the Exascale era platform, assuming this approach will be available on AMD and/or Intel GPUs in the future.
Idomura, Yasuhiro; Obrejan, K.*; Asahi, Yuichi; Honda, Mitsuru*
Physics of Plasmas, 28(1), p.012501_1 - 012501_11, 2021/01
Times Cited Count：6 Percentile：59.92(Physics, Fluids & Plasmas)Tracer impurity transport in ion temperature gradient driven (ITG) turbulence is investigated using a global full-
gyrokinetic simulation including kinetic electrons, bulk ions, and low to medium 
tracer impurities, where is the charge number. It is found that in addition to turbulent particle transport, enhanced neoclassical particle transport due to a new synergy effect between turbulent and neoclassical transports makes a significant contribution to tracer impurity transport. Bursty excitation of the ITG mode generates non-ambipolar turbulent particle fluxes of electrons and bulk ions, leading to a fast growth of the radial electric field following the ambipolar condition. The divergence of 
flows compresses up-down asymmetric density perturbations, which are subject to transport induced by the magnetic drift. The enhanced neoclassical particle transport depends on the ion mass, because the magnitude of up-down asymmetric density perturbation is determined by a competition between the compression effect and the return current given by the parallel streaming motion. This mechanism does not work for the temperature, and thus, selectively enhances only particle transport.
Obrejan, K.; Idomura, Yasuhiro; Honda, Mitsuru*
no journal, ,
The use of tungsten to coat divertors or other plasma facing components inevitably leads to the pollution of the fusion plasma by high-Z tungsten impurities. Several tokamaks have reported accumulation and exhaust of tungsten depending on a heating condition. This kind of collisional and turbulent transport of high-Z impurities is not yet fully understood. To treat high-Z impurities, the gyrokinetic full-f Eulerian code GT5D was extended with an implicit collision solver based on a multi-species linear Fokker-Plank collision operator. A collisional transport benchmark of tungsten was conducted to verify the accuracy of GT5D.
Obrejan, K.; Idomura, Yasuhiro; Honda, Mitsuru*
no journal, ,
To analyze heavy impurity transport, the gyrokinetic full-f Eulerian code GT5D was extended with an implicit collision solver based on a multi-species linear Fokker-Plank collision operator. In the collision solver, a Krylov subspace method is applied to a collision operator, which is given as a convection diffusion operator in velocity space, and severe CFL conditions of heavy impurities are resolved. In addition, flux coordinates are newly applied to resolve magnetic field structures with less grids, and collisional transport simulations are dramatically accelerated. A collisional transport benchmark of tungsten was conducted to verify the accuracy of GT5D.
Idomura, Yasuhiro; Obrejan, K.
no journal, ,
A gyrokinetic full-f Eulerian code GT5D is extended for multiple ion species. Multi-species collision operators between high Z impurities and bulk ions or electrons become multi-scale problems in velocity space, and thus, are numerically difficult problems. This issue is resolved by introducing a filter to suppress the divergence of collisional coefficients at lower energy and an implicit collision operator, and analyses of high Z impurities such as tungsten are realized. We present verification results on neoclassical transport with multiple ion species, and discuss impurity transport properties in nonlinear turbulence simulations.
Obrejan, K.; Idomura, Yasuhiro; Honda, Mitsuru*
no journal, ,
The use of tungsten to coat divertors or other plasma facing components, inevitably leads to the pollution of the fusion plasma by heavy and highly charged (high-Z) impurities. This kind of neoclassical and turbulent transport of high-Z impurities is not yet fully understood. To study high-Z impurity transport, the gyrokinetic full-f Eulerian simulation code GT5D was upgraded with an implicit collision solver based on a multi-species linear Fokker-Plank collision operator. The commonly employed low order approximation of the standard neoclassical theory showed good agreement for low-Z impurities, but large differences were observed for high-Z impurities in the presence of temperature gradients. We tested an improved form of the neoclassical transport theory, that takes into account higher order flows to more accurately recover the friction coefficients in a high collisionality regime, showing good agreement with GT5D. An important consequence of this improved method is the weakening of thermal screening effect of bulk ions on impurity transport.
Idomura, Yasuhiro; Ina, Takuya*; Obrejan, K.; Asahi, Yuichi*; Matsuoka, Seikichi*; Imamura, Toshiyuki*
no journal, ,
Under the post-K project, we have developed computing techniques of the Gyrokinetic Toroidal 5D full-f Eulerian code GT5D towards the next generation computing platforms based on many core processors. We discuss computational challenges related to complicated intra-processor memory hierarchy and limited inter-node communication performance compared with accelerated computation. The former issue is addressed by optimizing data access patterns of a stencil kernel on each many core architecture, and high performance gains are obtained. The latter issue is resolved by using advanced communication avoiding Krylov methods, which enables an order of magnitude reduction of collective communications and improves arithmetic intensity of main computing kernels. By applying these novel computing techniques, the performance of GT5D is dramatically improved on the latest many core platforms, and excellent strong scaling up to the full system size of the Oakforest-PACS (8,192 KNLs) is achieved.
Idomura, Yasuhiro; Obrejan, K.; Asahi, Yuichi*; Matsuoka, Seikichi*
no journal, ,
To enable low cost experimental data analyses Transient plasma responses due to modulated electron heating are investigated in numerical experiments of ion temperature gradient driven trapped electron mode turbulence using gyrokinetic full-f Eulerian code GT5D. It is found that even with electron heating without particle and momentum sources such as electron cyclotron resonance heating, dominant turbulence is changed from ion turbulence to electron turbulence due to changes of the electron temperature gradient and the temperature ratio, and the resulting turbulent transport produces responses of density and rotation profiles.
Obrejan, K.; Idomura, Yasuhiro; Honda, Mitsuru*
no journal, ,
The numerical accuracy of the gyrokinetic full-f Eulerian code GT5D is studied with respect to collisional or neoclassical transport of heavy impurity ions such as tungsten. Collisional transport of tungsten is determined by its collisional interaction with fuel ions such as deuterium and tritium. Because of the extreme mass ratio between deuterium and tungsten, the collision operator becomes a multi-scale problem in velocity space. In this work, numerical convergence of the collision operator is examined by estimating friction matrix elements, and numerical resolution needed for recovering collisional transport theory of heavy impurity ions clarified.
Obrejan, K.; Idomura, Yasuhiro; Honda, Mitsuru*
no journal, ,
The use of tungsten in plasma facing components inevitably leads to the pollution of the fusion plasma by heavy impurities and their accumulation at the core. Although heavy impurity transport has been estimated by the neoclassical transport theory, recent studies exhibited differences between global gyrokinetic simulations and the Hirshman-Sigmar (HS) moment approach, a commonly employed local theory. We performed a thorough benchmark for neoclassical particle transport of various impurities using the newly upgraded multi-species linear Fokker-Plank collision operator of our gyrokinetic full-f Eulerian simulation code GT5D. While good agreement was obtained in the case of flat temperature profiles, the dependence on the temperature gradient was found to be much weaker than the H-S theory in small machine sizes. Good agreement between GT5D and the H-S theory was recovered only for the case of large machine sizes, similar to ITER but beyond that of most current fusion devices.
Obrejan, K.; Idomura, Yasuhiro; Honda, Mitsuru*
no journal, ,
The use of tungsten in plasma facing components inevitably leads to the pollution of the fusion plasma by heavy impurities and their accumulation at the core, and this phenomenon is not yet fully understood. In this study, we performed a thorough comparison of the impurity particle fluxes between the global gyrokinetic full-f Eulerian code GT5D and local collisional transport theory called as the Hirshman-Sigmar moment approach. It is found that the simulation and the theory agree only in flat temperature profile cases or in large device sizes comparable to ITER. The cause of their discrepancy at the current device sizes is investigated in detail by comparing flows, which drive collisional impurity transport.
Idomura, Yasuhiro; Obrejan, K.; Honda, Mitsuru*
no journal, ,
Global neoclassical transport analyses of tungsten impurity are conducted using the global full-f gyrokinetic code GT5D, and the results are compared against local neoclassical transport theory by Hirshman-Sigmar. Systematic scans of the plasma size and the temperature gradient showed that theory and simulations show quantitative agreements when the plasma size is large enough. However, at smaller plasma sizes, it is found that steep temperature gradients enhance their differences, and theory overestimates the thermal screening effect of tungsten transport.
Idomura, Yasuhiro; Obrejan, K.*; Asahi, Yuichi; Honda, Mitsuru*
no journal, ,
Transport mechanisms of tracer impurities in the ion temperature gradient driven (ITG) turbulence are studied using the global full-f gyrokinetic code GT5D, and it is found that neoclassical particle transport is greatly enhanced by an interaction with the ITG turbulence. The radial electric field grows by turbulent bulk particle transport driven by the ITG turbulence. A compression effect of the resulting E
B flows generates up-down asymmetric density perturbations, which are coupled with the magnetic drift to enhance neoclassical particle transport. This effect does not work for the temperature, and selectively enhances only particle transport.
Asahi, Yuichi; Maeyama, Shinya*; Bigot, J.*; Garbet, X.*; Grandgirard, V.*; Obrejan, K.*; Padioleau, T.*; Fujii, Keisuke*; Shimokawabe, Takashi*; Watanabe, Tomohiko*; et al.
no journal, ,
We will demonstrate the performance portable implementation of a kinetic plasma code over CPUs, Nvidia and AMD GPUs. We will also discuss the performance portability of the code with C++ parallel algorithm. Deep learning based surrogate models for fluid simulations will also be demonstrated.
Asahi, Yuichi; Maeyama, Shinya*; Bigot, J.*; Garbet, X.*; Grandgirard, V.*; Obrejan, K.*; Padioleau, T.*; Fujii, Keisuke*; Shimokawabe, Takashi*; Watanabe, Tomohiko*; et al.
no journal, ,
We will demonstrate the performance portable implementation of a kinetic plasma code over CPUs, Nvidia and AMD GPUs. We will also discuss the performance portability of the code with C++ parallel algorithm. Deep learning based surrogate models for fluid simulations will also be demonstrated.
Asahi, Yuichi; Hasegawa, Yuta; Padioleau, T.*; Millan, A.*; Bigot, J.*; Grandgirard, V.*; Obrejan, K.*
no journal, ,
Generally, production-ready scientific simulations consist of many different tasks including computations, communications and file I/O. Compared to the accelerated computations with GPUs, communications and file I/O would be slower which can be major bottlenecks. It is thus quite important to manage these tasks concurrently to suppress these costs. In the present talk, we employ the proposed language standard C++ senders/receivers to mask the costs of communications and file I/O. As a case study, we implement a 2D turbulence simulation code with the local ensemble transform Kalman filter (LETKF) using C++ senders/receivers. In LETKF, the mock observation data are read from files followed by MPI communications and dense matrix operations on GPUs. We demonstrate the performance portable implementation with this framework, while exploiting the performance gain with the introduced concurrency.
