# Is there any relation between static correlation and finite-temperature smearing?

I am new to this field, so there is a chance that I am mixing two entirely different concepts together, but it seems to be that static correlation and finite-temperature smearing in DFT are somehow related.

As I have understood static correlation comes into existence when the HOMO-LUMO gap is small, so that the ground state and the excited state have energies close to each other. This means one configuration (i.e. one slater determinant) is not good enough to describe the system, and multi-configurational methods have to be used.

Finite-temperature smearing (which I have seen mostly in DFT codes) is again used in cases where the HOMO and LUMO levels are close in energy. As they are close in energy they can exchange positions during the SCF iterations, making it difficult to attain convergence. So, the energy levels are allowed to have fractional populations, and the electrons are smeared across orbitals with some scheme like Fermi-Dirac. This fixes the problem (only the SCF convergence or some other problem I am not sure).

Now they both look similar to me in that they are attempting to solve the same problem. Intuitively, the electron-smearing looks like a cheap way to represent the average state of a multi-configuration system.

So, is there any relation between these two? If so, then can electron-smearing return the static correlation energy? If not, then how/why are they different?

Edit: I found this paper by Grimme et. al. (https://onlinelibrary.wiley.com/doi/abs/10.1002/anie.201501887), where finite temperature DFT is used to visualise static correlation in molecules. However, that paper only deals with visualising the fractional orbital density, but does not go into detail about the relation between correlation and temperature smearing, so my question still remains.

This is a nice thought, but the two are not actually related. In principle, static correlation is completely accounted for in the exchange-correlation functional of Kohn-Sham DFT and there is no need for any mixing of the Kohn-Sham states, in fact it would lead to an incorrect result. In practice, the exchange-correlation functional is approximated and so static correlation is not fully captured. You could take the Kohn-Sham states as a basis in which to perform a wavefunction-based correlated many-body quantum mechanical method, in which case your electronic states could be represented as a set of partial occupancies of the Kohn-Sham states, but that is not why (or how) partial occupancies arise in Kohn-Sham DFT. (NB partial occupancies can also be used in time-dependent DFT to express excited states in the basis of the original Kohn-Sham states.)

There are three main reasons why partial occupancies are often used in Kohn-Sham DFT -- strictly speaking, Mermin-Kohn-Sham DFT, since it entails Mermin's extension and modification to the Kohn-Sham energy functional:

## SCF instabilities

If the Kohn-Sham valence band/HOMO and conduction band/LUMO are close (especially if they are degenerate) then the system can be unstable with respect to the occupation of those states. As a simple example, suppose we have a 2-state system with states A and B. Suppose we start with A occupied, and the particle density comes entirely from state A,

$$\rho(\vec{r})=\rho_A(\vec{r})=\vert\psi_A(\vec{r})\vert^2$$.

We now construct the Kohn-Sham potential and hence the Hamiltonian, and evaluate the energy expectation values of the two states,

$$E_A = \langle \psi_A \vert \hat{H}[\rho_A] \vert \psi_A\rangle$$

$$E_B = \langle \psi_B \vert \hat{H}[\rho_A] \vert \psi_B\rangle$$

It is perfectly possible that $$E_B, in which case we now realise that it is state B which should have been occupied, and thus compute the density as

$$\rho(\vec{r})=\rho_B(\vec{r})=\vert\psi_B(\vec{r})\vert^2$$.

Now we recalculate the energy expectation values with the new Hamiltonian,

$$E^\prime_A = \langle \psi_A \vert \hat{H}[\rho_B] \vert \psi_A\rangle$$

$$E^\prime_B = \langle \psi_B \vert \hat{H}[\rho_B] \vert \psi_B\rangle$$

Is it still true that state B is a lower-energy state than A, i.e. $$E^\prime_B < E^\prime_A$$? Not necessarily!

If in fact $$E^\prime_A < E^\prime_B$$ then we would conclude that A is the lower energy state, and that it is state A which should contribute to the density -- we are right back where we started! This phenomenon is called occupancy sloshing.

This sloshing instability can be quenched by using an eigenvalue smearing function and partial occupancies, just as happens in real systems at finite temperature; however, there is no need for this energy smearing to actually correspond to any physical temperature. The Fermi-Dirac smearing function has quite a long tail, and leads to fractionally occupied states across a wide energy range, which in turn requires a large number of extra states to be computed. For this reason, it is common to use smearing functions which decay more rapidly, such as Gaussians.

## Brillouin Zone sampling

A second reason why partial occupancies are sometimes used is in periodic systems such as materials, where the Kohn-Sham states form bands and the Brillouin zone must be sampled. A common method to do this is with a Monkhorst-Pack (or Chadi-Cohen) mesh of "k-points", and the Kohn-Sham eigenvalues are sampled at these reciprocal-space points. These eigenvalues are taken to be representative of the energy expectation values across the small region of the Brillouin Zone centred on that k-point.

If the valence and conduction Kohn-Sham bands cross in energy somewhere in the Brillouin zone, it will not usually occur on one of the sampled k-points. When determining which of the Kohn-Sham states A or B is occupied, the correct answer is neither A nor B, but that one is occupied for part of the region and the other for the remaining part -- in other words, they are fractionally occupied. Since we do not know how the states behave away from these sampling points, it is common to use a Gaussian smearing function as an indicator of the probable occupation.

(NB an alternative is to try to interpolate the Kohn-Sham eigenvalues across the Brillouin zone, which is the approach underlying the tetrahedron integration methods.)

Since this problem arises from the finite k-point sampling, using a denser sampling mesh allows a smaller energy broadening to be used.

## Finite-temperature simulations

A third case where partial occupancies are used is when the phenomenon being modelled only occurs at finite Kohn-Sham temperature, and in this case a Fermi-Dirac smearing function is usually applied. Since typical "physical" temperatures lead to rather small thermal broadening (typically a tenth of the usual smearing width) the Fermi-Dirac function is sometimes convolved with a broader Gaussian function, akin to $$0~K$$ simulations.

Note that the Kohn-Sham states are not actually electronic (or quasi-particle) states, they are fictitious auxiliary states. They are often used as approximations to the electronic states, and it is in this context that a physical finite-temperature smearing is justifiable. However, there is no requirement that the temperature-dependence of the Kohn-Sham states reflects the temperature-dependence of the electronic states.

• So does that mean DFT gives static correlation in spite of having a single determinant? How would that work? Jan 26 '21 at 21:34
• Also, I just came across this paper by Grimme: onlinelibrary.wiley.com/doi/abs/10.1002/anie.201501887, where finite temperature DFT results are used to visualize the static correlation in a system, so I am no longer sure if your answer is correct. Jan 26 '21 at 21:40
• @ShoubhikRMaiti it does in theory, but not in practice, since we don't have the exact functional. The density functional approximations we have generally yield unreliable results for systems with static correlation. Jan 27 '21 at 15:50
• @ShoubnikRMaiti Kohn-Sham DFT doesn't use any determinants -- the single-particle wavefunctions are fictitious and the many-body wavefunction is a product state, not a Slater determinant. The beauty of the Hohenberg-Kohn theorem is that you can choose any auxiliary system you like, you don't even need a wavefunction at all. The difficult part is that we don't know the "true" functional, so in practice you have to be more careful with the wavefunction choice and exchange-correlation approximation. Jan 28 '21 at 10:02