Self-learning Monte Carlo method with equivariant transformers

Nagai, Yuki; Tomiya, Akio*

no journal, , 

We proposed a new self-learning Monte Carlo method using Transformer, a key technology in generative AI. By using Transformer's Attention mechanism, which can infer the relevance of distant words in a sentence, we have constructed an effective model that can efficiently capture the long-range correlations that are important in the phase transitions in electronic systems. Furthermore, we reduce the number of parameters by incorporating symmetries that the system must satisfy, such as spin rotation and spatial translation, into the network. We also found a scaling law that the loss decreases as the number of layers is increased.