What is the Hamiltonian in the adiabatic connection?

Question

I am trying to understand the adiabatic connection as motivation for hybrid functionals that include Hartree-Fock like exchange. While it sounds intuitive that the exchange of the noninteracting Kohn-Sham system is given by HF exchange, I fail to see it formally. Here is a short summary of my understanding and the definitions.

The Kohn Sham system is a noninteracting system of electrons that yields the same ground state density as the interacting system. My understanding of "noninteracting" is that it means that we can write down a system of one particle operators such that the total wavefunction would be a Hartree product, but we also demand anti-symmetry, and thus the ground state wavefunction is a Slater determinant.

The equations for the KS orbitals in atomic units are $$ (-\frac{1}{2}\Delta + v_0(r))\phi_i(r) = \varepsilon_i\phi_i(r) $$ where $v_0(r)$ is an effective one-particle potential that is constructed to yield the groundstate density of our real interacting system. The effective potential is found by variation of the energy of the real system as functional of the Kohn-Sham orbitals , which is achieved by writing it as $$ E = T_0 +J+V_{ext}+E_{XC} $$ where $T_0=-\frac{1}{2}\sum^N_i\langle \phi_i|\Delta\phi_i\rangle$ is the kinetic energy, calculated using the KS orbitals, $J=\frac{1}{2}\int\int\frac{ n(r')n(r)}{|r-r'|}drdr'$ is a classical density Coulumb interaction, $V_{ext}= -\frac{1}{2}\sum_a \int \frac{n(r)Z_a}{|r-r_a|}dr$ is the energy due to external potential, and $E_{xc}$ is the unknown exchange-correlation energy. The one-electron density is obtained from the KS orbitals according to $n(r)=\sum_i|\phi_i(r)|^2$. The first variation of this energy with respect to the KS orbitals combined with the orthogonality condition for the orbitals allows us to identify the effective potential
$$ v_0 = v_{ext} + v_{xc} + v_{J} $$ where $v_{ext}(r)=-\frac{1}{2}\sum_a \frac{Z_a}{|r-r_a|}$ and $v_J(r) =\frac{1}{2}\int\frac{ n(r')}{|r-r'|}dr'$ and $v_{xc} = \frac{\delta E_{xc}}{\delta n}$.

So far everything is simple KS-DFT and clear.

Now comes the the adiabatic connection to justify Hybrid functionals. The exchange correlation energy can be calculated by the following integration $$ E_{xc} = \int^1_0 d\lambda E_{xc,\lambda}. $$ $E_{xc,\lambda=0}$ is then identified as exact Hartree-Fock exchange, and the integral is approximated as a weighted sum of $E_{xc,\lambda=0}$ and $E_{xc,\lambda=1}$. I do not understand how the identification of $E_{xc,\lambda=0}$ as Hartree-Fock exchange is made.

The parameter $\lambda$ connects the exact Hamiltonian of the interacting system with the KS Hamiltonian of noninteracting electrons. I have difficulties to connect these statements with the above derivation. I thought that the Kohn-Sham Hamiltonian is given by $$ \hat H_{KS} = \sum_i (-\frac{1}{2}\Delta_i + v_0(r_i)) $$ and the groundstate solution to this Hamiltonian is the determinant of the KS orbitals. The total energy of this Hamiltonian is then simply $E_{KS} = \sum_i \varepsilon_i$. This energy is the energy of the uncorrelated system and generally different to energy of the real system. Are these statements correct or am I making a mistake already at this point? My understanding is that the adiabatic connection connects this Hamiltonian with the Hamiltonian of the interaction system given by $$ H = \hat T + \hat V_{ee} + V_{ext} $$ such that $H(\lambda=1) = H$ and $H(\lambda=0)=H_{KS}$. But when I look at these Hamiltonians I fail to see how the adiabatic connection formula arises and how exact Hartree Fock exchange enters the picture.

My question is if my identification of $H_{KS}$ as Hamiltonian for $\lambda=0$ correct or wrong?

Answer 1

Your $\lambda=0$ definition corresponds to the complete neglect of interelectronic interactions. There is nothing to prevent you from doing that in principle, but the resulting approximation you would get is probably horrible.

By including Coulomb and exact exchange in the $\lambda=0$ definition, you get already a lot closer to the real physics, since these account for a big part of the total energy, the rest being called "correlation". Remember, the only thing that really matters is the total energy; the division into "exchange" and "correlation" really only comes from our inability to solve the Schrödinger equation. Exact exchange just comes from the simplest wave function theory, that is, Hartree-Fock, and it is useful because it turns out that it is e.g. straightforward to derive the local density approximation for Hartree-Fock exchange. Its usefulness is that the simple expression typically captures most of the total energy, and the rest that we call "correlation energy" (which we could also call "stupidity energy" per Richard Feynman's suggestion) is often a small cleanup term.

If the difference between $\lambda=0$ and $\lambda=1$ is small, then one can also use e.g. perturbation theory to develop approximations for the exact result. But, if you try doing this for the full electron interaction term, you will almost certainly fail.

Answer 2

I have resolved my problem. The source of my confusion was mistaking the integrand $E_{xc,\lambda}$ at specific values of $\lambda$ with $E_{xc}$ itself. Hybrid functionals can be motivated by linear interpolation of the integrand and it is this integrad that equals Hartree-Fock exchange at $\lambda=0$. This is clearer in more explicit notation, i.e.

$$ E_{xc,\lambda} = \frac{dE_{xc}(\lambda)}{d\lambda}\approx\frac{dE_{xc}(\lambda)}{d\lambda}|_{\lambda=0} + \lambda\left(\frac{dE_{xc}(\lambda)}{d\lambda}|_{\lambda=1} -\frac{dE_{xc}(\lambda)}{d\lambda}|_{\lambda=0} \right) $$

where the coupling parameter dependent correlation-exchange functional is given by $$ E_{xc}(\lambda) = \min_{\Psi\rightarrow n}\langle \Psi_\lambda|\hat T+ \lambda \hat V_{ee}|\Psi_\lambda\rangle - T_0 -\lambda J. $$ For the non-interacting system the exchange-correlation functional is obtained in the $\lambda=0$ limit and becomes zero as expected.

The derivative with respect to $\lambda$ is obtained using the Hellman Feynmann theorem,
$$ \frac{dE_{xc}(\lambda)}{d\lambda} = \min_{\Psi\rightarrow n}\langle \Psi_\lambda| \hat V_{ee}|\Psi_\lambda\rangle - J $$ The wavefunction $\Psi_\lambda$ equals the Kohn-Sham determinant at $\lambda=0$ and goes to the exact wavefunction of the fully interacting Hamiltonian at $\lambda=1$. Evaluating the derivative at $\lambda=0$ is thus equal to Hartree-Fock exchange. And the Exchange-correlation functional can be approximated integrating the linear interpolation from above, $$ E_{xc} = \int_0^1 \frac{dE_{xc}(\lambda)}{d\lambda}d\lambda \approx \frac{1}{2}\left( \frac{dE_{xc}(\lambda)}{d\lambda}|_{\lambda=1} + \frac{dE_{xc}(\lambda)}{d\lambda}|_{\lambda=0} \right) $$

The KS-Hamiltonian is also recovered if we define the adiabatic connection in terms of Hamiltonians according to $$ H_\lambda = T + \lambda V_{ee} +V_\lambda $$ where $H_1$ is fully interacting Hamiltonian with the external potential $V_1=V_{ext}$, while at $\lambda=0$ the Hamiltonian becomes the KS Hamiltonian, with the effective potential $V_0=\sum_i v_0$.