What is the quantum anomalous Hall effect?
Figure from C-X. Liu, S-C. Zhang, and X-L. Qi. "The Quantum Anomalous Hall
Effect: Theory and Experiment," Annual Review of Condensed Matter Physics 7, 301-321 (2016) (arXiv link).
In short, the quantum anomalous Hall effect (QAHE) is the quantized version of the regular anomalous Hall effect (AHE). That is, a quantized Hall effect that does not require an external magnetic field. Historically, Edwin Hall first discovered the (now regular) Hall effect in 1879, in a metallic sheet perpendicular to a magnetic field carrying a current. The magnetic field deflects electrons preferentially to one side through the Lorentz force, leading to a voltage $V_H$ developing across the sample but perpendicular to both magnetic field and the applied current.
As von Klitzing discovered in 1980, a similar effect occurs in two-dimensional electron systems at low temperatures and strong fields, except with a quantized response. This is of course the integer quantum Hall effect. Specifically, the conductance satisfies $\sigma=\frac{I}{V_H} = \nu \frac{e^2}{h}$, where $V_H$ is the Hall voltage, $\nu$ is an integer, and $h$ is Planck's constant. (At even lower temperatures the particularly exotic fractional quantum Hall effect occurs, where $\nu$ can take fractional values. That is beyond the scope of this post.) Heuristically, we can imagine the case where the applied field is so strong that electrons in the center of the sample are stuck in cyclotron orbits, as in panel (d) of the figure above. However, near the edge of the sample we get so-called skipping orbits, which results in that the states near the upper edge only move in one direction, and states near the lower edge move in the opposite direction. These edge states are called chiral. For more details see e.g. these notes. The integer $\nu$ turns out to be the (first) Chern number, also known as the TKNN invariant.
In 1881 Hall published his finding that the Hall voltage is significantly higher in ferromagnetic materials than in nonmagnetic ones. This was dubbed the anomalous Hall effect (AHE). Whereas the regular Hall effect could be explained using relatively simple physics, it did take a long time to develop a full understanding of the AHE. By the 1930s, it had been empirically demonstrated that the Hall resistivity in ferromagnetic metals satisfies
$$
\rho_{xy}= R_0 H_z + R_s M_z,
$$
where $H_z$ is the applied magnetic field, and $M_z$ is the spontaneous magnetization. The first term represents the regular Hall effect (which tends to saturate in metals), and the second contains the AHE. While the parameter $R_0$ depends mainly on carrier density, $R_s$ is material specific. Note that AHE remains at zero field, assuming time-reversal symmetry is spontaneously broken, as it tends to be in ferromagnetic materials.
Given the discovery of the IQH it was natural to ask whether we can also have a quantized anomalous Hall effect (QAHE), that is can we have a quantized Hall conductance induced by a mechanism other than a magnetic field? Perhaps even in zero field? The first proposal of such a system was due to Haldane in 1988, previously discussed on the site here. This type of system is protected again by the Chern number, and known as a Chern insulator.
To complete the picture, note that spin-orbit coupling acts similar to a magnetic field, but with opposite sign for opposite sign spins. Using this approach, it is possible to construct what looks like two copies of the AHE in a way that doesn't break time-reversal symmetry, resulting in the spin Hall effect, or two copies of the QAHE resulting in the quantum spin Hall effect, also known as 2D topological insulators. The latter are protected by a $\mathbb{Z}_2$ topological invariant.
What is the source of the anomalous Hall effect?
As mentioned above, the AHE resisted theoretical description for a long time. In part because it doesn't have one single physical origin, but has contributions from magnetism, spin-orbit coupling and disorder effects. In part it's because we didn't have all the appropriate theoretical concepts until the 80s and 90s, and it took still longer until we had the computational materials modeling in place to help sort out which mechanism(s) would dominate in a given material. There's an excellent and comprehensive review by Nagaosa, Sinova, Onoda, MacDonald, and Ong, "Anomalous Hall effect," Reviews of Modern Physics 82, 1539-1592 (2010) (arXiv link) that covers the history and mechanisms in detail. It also has this figure, which lays out the mechanisms clearly.
The intrinsic mechanism is due to the Berry phase of the electron bands in the system, and results in a modification of the electron group velocity (so-called "anomalous velocity") that is perpendicular to an applied electric field. Since you say you're not sure how the term appears, I will refer you to another excellent review article: Di Xiao, Ming-Che Chang, and Qian Niu, "Berry phase effects on electronic properties," Reviews of Modern Physics 82, 1959-2007 (2010) (arXiv link), specifically the derivations leading up to Eq. (3.6). However, the point here is that the intrinsic contributions come from Berry curvatures, which comes from the band structure itself. When summed over occupied bands in a ferromagnetic metal, the net current from this mechanism typically remains finite.
There are also two extrinsic mechanisms in the presence of disorder, known as side-jump and skew scattering. Skew scattering is due to asymmetric disorder scattering when spin-orbit coupling is present. It is well-described by standard Boltzmann theory, and tends to dominate in highly conductive ferromagnets. The side-jump story is a complicated one. It can be related to a transverse displacement of an incoming electron hitting an impurity, but ultimately its role is less clear and often less interesting. To quote Nagaosa et al.,
A practical approach, which is followed at present for materials in which $\sigma^{AH}$ seems to be independent of $\sigma_{xx}$ is to first calculate the intrinsic contribution to the AHE. If this explains the observation (and it appears that it
usually does), then it is deemed that the intrinsic mechanism dominates. If not, we can take some comfort from understanding on the basis of simple model results that there can be other contributions to $\sigma^{AH}$ which are also independent of $\sigma_{xx}$ and can for the most part be identified with the side-jump mechanism.
Rule of thumb: Often the intrinsic mechanism dominates in systems with strong spin-orbit coupling, so it does seem likely to be the most important mechanism in TMDs (transition metal dichalcogenides). However, there may be some material and sample dependence.
What is the source of the quantum anomalous Hall effect?
This question has a much simpler answer: it's the intrinsic mechanism. To have a quantized response, we need something integer-valued to arise from one of these three mechanisms. The way to get that is to integrate the Berry curvature over a full band, and getting a nonzero Chern number. When can we integrate over a full band? Well, we want to avoid the typical situation in a metal, where there is an open Fermi sea, which would mean integrating over only some fraction of the states in a band (the occupied states). Instead, we generally want an insulating system. (This of course has the side effect of reducing the number of mobile carriers, thus decreasing the role of the extrinsic disorder mechanisms.)
There were proposals that the QAHE state can occur in graphene with strong Rashba spin-orbit coupling and exchange effects (also here). One way to achieve the exchange effects is to place the graphene sheet close to a ferromagnetic material, as has been explored experimentally e.g. here. However, so far I don't think these efforts have succeeded in obtaining a quantized response. Generally it is difficult to achieve strong enough spin-orbit coupling in graphene. (Some readers may be familiar with the attempts of doping graphene to achieve a quantum spin Hall insulator.)
Given the similarity between TMDs and graphene, I imagine these references can be good conceptual starting points also for TMDs. With the huge interest in Moiré structures, which allow tunable interaction strengths, there likely is a lot more literature on multilayer systems too. For example, here's one paper proposing a quantum anomalous Hall effect regime in twisted bilayer TMDs. I'm however not up to date on the experimental situation in these materials.
