PySCF: atomic basis Fock matrix changes after mo_coeff update

Question

I've noted a curious thing with PySCF when updating the molecular orbital coefficients of the mean-field object. If one prints out the Fock matrix in the atomic basis (F_ao) from the mean_field, the values can differ once the mo_coeff has been updated.

The following is an extreme example:

from pyscf import gto, scf
import numpy as np

molecule = gto.Mole()
geometry = """
  O   0.000000   0.000000   0.000000
  H   0.758602   0.000000   0.504284
  H   0.758602   0.000000  -0.504284
"""
molecule.atom = geometry
molecule.basis = "3-21g"
molecule.build()

mean_field = scf.RHF(molecule)
mean_field.scf()

F_ao1 = mean_field.get_fock()
mean_field.mo_coeff = np.loadtxt('0.txt')
F_ao2 = mean_field.get_fock()

for row in range(len(F_ao1[0])):
    for col in range (len(F_ao1[0])):
        m = print("{:10.6f}".format(F_ao1[row][col]), end = " ")
    print(m)
print()

for row in range(len(F_ao2[0])):
    for col in range (len(F_ao2[0])):
        m = print("{:10.6f}".format(F_ao2[row][col]), end = " ")
    print(m)
print()

where 0.txt is a file containing a 13x13 matrix with all elements 0.

I had assumed that the Fock matrix in the atomic basis would not be recalculated and was a stored object? Is there an explanation for this behaviour?

---

For reference, the output is:

---F_ao1---
-20.312687  -5.701111  -4.466295   0.000000   0.000000   0.036176  -0.000000   0.000000   0.007744  -0.589952  -1.509333  -0.589952  -1.509333 None
 -5.701111  -1.628973  -2.433482   0.000000   0.000000   0.057360   0.000000   0.000000   0.041249  -0.715556  -1.164825  -0.715556  -1.164825 None
 -4.466295  -2.433482  -2.132193   0.000000   0.000000   0.087107   0.000000   0.000000   0.196043  -0.702254  -1.079343  -0.702254  -1.079343 None
  0.000000   0.000000   0.000000   0.329725   0.000000  -0.000000  -0.772527   0.000000  -0.000000   0.477856   0.266686  -0.477856  -0.266686 None
  0.000000   0.000000   0.000000   0.000000   0.199071   0.000000  -0.000000  -0.799305   0.000000   0.000000   0.000000  -0.000000  -0.000000 None
  0.036176   0.057360   0.087107  -0.000000   0.000000   0.270456  -0.000000   0.000000  -0.791339   0.385320   0.258909   0.385320   0.258909 None
 -0.000000   0.000000   0.000000  -0.772527  -0.000000  -0.000000  -0.228986  -0.000000  -0.000000   0.350672   0.173939  -0.350672  -0.173939 None
  0.000000   0.000000   0.000000   0.000000  -0.799305   0.000000  -0.000000  -0.018129   0.000000   0.000000   0.000000  -0.000000  -0.000000 None
  0.007744   0.041249   0.196043  -0.000000   0.000000  -0.791339  -0.000000   0.000000  -0.151176   0.286736   0.251041   0.286736   0.251041 None
 -0.589952  -0.715556  -0.702254   0.477856   0.000000   0.385320   0.350672   0.000000   0.286736  -0.114823  -0.639373  -0.132769  -0.341354 None
 -1.509333  -1.164825  -1.079343   0.266686   0.000000   0.258909   0.173939   0.000000   0.251041  -0.639373  -0.609333  -0.341354  -0.539374 None
 -0.589952  -0.715556  -0.702254  -0.477856  -0.000000   0.385320  -0.350672  -0.000000   0.286736  -0.132769  -0.341354  -0.114823  -0.639373 None
 -1.509333  -1.164825  -1.079343  -0.266686  -0.000000   0.258909  -0.173939  -0.000000   0.251041  -0.341354  -0.539374  -0.639373  -0.609333 None

---F_ao2---
-32.880976  -7.706070  -6.427702   0.000000   0.000000   0.024455   0.000000   0.000000   0.003460  -0.829238  -2.157992  -0.829238  -2.157992 None
 -7.706070 -10.178522  -8.072464   0.000000   0.000000   0.160239   0.000000   0.000000   0.096904  -1.784625  -3.309188  -1.784625  -3.309188 None
 -6.427702  -8.072464  -8.320350   0.000000   0.000000   0.152962   0.000000   0.000000   0.252444  -2.656440  -4.381549  -2.656440  -4.381549 None
  0.000000   0.000000   0.000000  -8.774770   0.000000  -0.000000  -4.261563   0.000000   0.000000   1.468854   0.828412  -1.468854  -0.828412 None
  0.000000   0.000000   0.000000   0.000000  -8.692929   0.000000   0.000000  -4.186167   0.000000   0.000000   0.000000   0.000000   0.000000 None
  0.024455   0.160239   0.152962  -0.000000   0.000000  -8.742686   0.000000   0.000000  -4.232005   1.162911   0.707275   1.162911   0.707275 None
  0.000000   0.000000   0.000000  -4.261563   0.000000   0.000000  -5.400436   0.000000   0.000000   2.523505   1.731794  -2.523505  -1.731794 None
  0.000000   0.000000   0.000000   0.000000  -4.186167   0.000000   0.000000  -5.159485   0.000000   0.000000   0.000000   0.000000   0.000000 None
  0.003460   0.096904   0.252444   0.000000   0.000000  -4.232005   0.000000   0.000000  -5.305976   1.995294   1.491102   1.995294   1.491102 None
 -0.829238  -1.784625  -2.656440   1.468854   0.000000   1.162911   2.523505   0.000000   1.995294  -4.947012  -3.548389  -0.288794  -1.280573 None
 -2.157992  -3.309188  -4.381549   0.828412   0.000000   0.707275   1.731794   0.000000   1.491102  -3.548389  -4.642069  -1.280573  -2.706780 None
 -0.829238  -1.784625  -2.656440  -1.468854   0.000000   1.162911  -2.523505   0.000000   1.995294  -0.288794  -1.280573  -4.947012  -3.548389 None
 -2.157992  -3.309188  -4.381549  -0.828412   0.000000   0.707275  -1.731794   0.000000   1.491102  -1.280573  -2.706780  -3.548389  -4.642069 None
```

Answer 1

The Fock matrix is defined as $F_{\mu \nu} = \partial E / \partial P_{\mu \nu}$ where ${\bf P}$ is the density matrix and $E$ is the total energy functional (here: restricted Hartree-Fock i.e. RHF). The RHF density matrix is given by ${\bf P} = 2 {\bf C}_{\rm occ} {\bf C}^{\rm T}_{\rm occ}$ where ${\bf C}_{\rm occ}$ are the occupied orbital coefficients.

Since the energy is non-linear in the density (Hartree-Fock is quadratic in the density matrix), the Fock matrix depends on the density matrix: ${\bf F}={\bf F}({\bf P})$. If you change the density matrix, the Fock matrix will also change.

By examining the properties of the density matrix, you can see that the density matrix does not change if you only mix occupied orbitals with occupied orbitals, and/or virtual orbitals with virtual orbitals. However, if you rotate occupied orbitals into virtual orbitals (and vice versa), the density matrix changes, and thereby also the Fock matrix and the total energy change.

For a more thorough discussion see e.g. our recent open-access overview in Molecules 25, 1218 (2020).