Spin excitation continuum from degenerate states in the mixed ferro-antiferromagnetic exchange system CeMgAlO

Gao, B.*; Chen, T.*; Liu, C.*; Klemm, M. L.*; Zhang, S.*; Ma, Z.*; Xu, X.*; Won, C.*; McCandless, G. T.*; Rao, K.*; Murai, Naoki  ; Kawamura, Seiko   ; Moxim, S. J.*; Ryan, J. T.*; Huang, X.*; Wang, X.*; Le, M. D.*; Morosan, E.*; Chan, J. Y.*; Cheong, S.-W.*; Tchernyshyov, O.*; Balents, L.*; Dai, P.*

In the search for unconventional magnetism, exotic quantum states are characterized by a lack of order and a broad spin excitation continuum approaching zero temperature. We study the two-dimensional triangular-lattice effective spin-system CeMgAlO, which shows slight disorder but no magnetic ordering down to 100 millikelvin. Spin-wave analysis in the magnetic-field-polarized state determines the spin Hamiltonian featuring a mixed ferromagnetic-antiferromagnetic nearest-neighbor exchange interaction. This places the system near an exactly solvable point of the spin-triangular-lattice XXZ model with extensive ground-state degeneracy. In zero field, neutron spectroscopy reveals a prominent continuum; we show that this arises from an ensemble average of spin-wave spectra across the degenerate ground-state manifold. This demonstrates that the role of weak quenched disorder can be quantitatively constrained: It inhibits unique ground-state selection and stabilizes a local distribution within the degenerate manifold, yielding continuum-like spectra that necessitate a critical reevaluation of the experimental signatures of exotic quantum states.