Initialising ...
Initialising ...
Initialising ...
Initialising ...
Initialising ...
Initialising ...
Initialising ...
Nagai, Yuki; Tomiya, Akio*
Journal of the Physical Society of Japan, 93(11), p.114007_1 - 114007_8, 2024/11
Times Cited Count：1 Percentile：48.32(Physics, Multidisciplinary)We proposed a new self-learning Monte Carlo method using Transformer, a key technology in generative AI. By using Transformer's Attention mechanism, which can infer the relevance of distant words in a sentence, we have constructed an effective model that can efficiently capture the long-range correlations that are important in the phase transitions in electronic systems. Furthermore, we reduce the number of parameters by incorporating symmetries that the system must satisfy, such as spin rotation and spatial translation, into the network. We also found a scaling law that the loss decreases as the number of layers is increased.
Tomiya, Akio*; Nagai, Yuki
Proceedings of Science (Internet), 453, p.001_1 - 001_7, 2024/11
Machine learning, deep learning, has been accelerating computational physics, which has been used to simulate systems on a lattice. Equivariance is essential to simulate a physical system because it imposes a strong induction bias for the probability distribution described by a machine learning model. However, imposing symmetry on the model sometimes occur a poor acceptance rate in self-learning Monte-Carlo (SLMC). On the other hand, Attention used in Transformers like GPT realizes a large model capacity. We introduce symmetry equivariant attention to SLMC. To evaluate our architecture, we apply it to our proposed new architecture on a spin-fermion model on a two-dimensional lattice. We find that it overcomes poor acceptance rates for linear models and observe the scaling law of the acceptance rate in machine learning.
Nagai, Yuki; Tanaka, Akinori*; Tomiya, Akio*
Physical Review D, 107(5), p.054501_1 - 054501_16, 2023/03
Times Cited Count：8 Percentile：74.17(Astronomy & Astrophysics)no abstracts in English
Nagai, Yuki; Tomiya, Akio*
no journal, ,
We proposed a new self-learning Monte Carlo method using Transformer, a key technology in generative AI. By using Transformer's Attention mechanism, which can infer the relevance of distant words in a sentence, we have constructed an effective model that can efficiently capture the long-range correlations that are important in the phase transitions in electronic systems. Furthermore, we reduce the number of parameters by incorporating symmetries that the system must satisfy, such as spin rotation and spatial translation, into the network. We also found a scaling law that the loss decreases as the number of layers is increased. In this talk, we discuss a possible application for lattice quantum chromodynamics of this technology.
Nagai, Yuki; Tomiya, Akio*
no journal, ,
We proposed a new self-learning Monte Carlo method using Transformer, a key technology in generative AI. By using Transformer's Attention mechanism, which can infer the relevance of distant words in a sentence, we have constructed an effective model that can efficiently capture the long-range correlations that are important in the phase transitions in electronic systems. Furthermore, we reduce the number of parameters by incorporating symmetries that the system must satisfy, such as spin rotation and spatial translation, into the network. We also found a scaling law that the loss decreases as the number of layers is increased.
