How is "basis set projection" done?

Question

I recently came across something called the basis set projection in Q-chem's manual. What it seems to do is to take the converged SCF in a smaller basis set, and then "projects" that onto a larger basis set (by either constructing the Fock matrix from the previous density matrix, or by expanding the previous MOs onto the larger basis set). This is then used as a guess for the SCF with the larger basis set. The manual page of Q-chem does not have any reference on this, and I can't find any paper by a google search.

However, this method seems to be well-established, because even old versions of GAMESS-US have an option of GUESS=RDMINI in the $GUESS section, which seems to be doing the same thing—reading the converged orbitals with a MINI basis set and then project that for a better guess.

What is the mathematical procedure of this projection, and why/how does it work? (And if possible, please mention the references, I would like to read more on this!)

Answer 1

Supplementing Nike's answer above.. This is actually quite elementary math. Let's say I have an orbital in a basis $|\psi\rangle = \sum_i c_i |i\rangle$ and I want to find the expansion in some other basis set $|J\rangle$. How do I do this?

In the new basis set, one has the resolution of the identity $\sum_{JK} |J\rangle \langle J|K\rangle^{-1} \langle K|\approx 1$, where $\langle J|K\rangle^{-1}$ denotes the element of the (pseudo)inverse overlap matrix. You can just plug this expression into the equation above to get $|\psi\rangle = \sum_i c_i |i\rangle = \sum_{iJK} |J\rangle \langle J|K\rangle^{-1} \langle K| i\rangle = \sum_J c_J |J\rangle $ from which you can read that the expansion coefficients in the new basis are given by $c_J = \sum_{iK} \langle J|K\rangle^{-1} \langle K| i\rangle c_i$.

To project a density, you can just project the orbitals one by one with the expression above; you can write this in matrix form as ${\bf C}_1={\bf S}_{11}^{-1}{\bf S}_{12}{\bf C}_2$ where the C's are the matrices of orbital coefficients. The orbitals may no longer be orthonormal as a result of the projection, so typically one reorthonormalizes them in the next phase before building the density matrix in the new basis set.

The "basis set guess" turns out to be of excellent accuracy; for instance, it was the top performer in my recent assessment of self-consistent field guesses, J. Chem. Theory Comput. 15, 1593 (2019), where I used the single-zeta pcseg-0 basis set. If you're trying to do a SCF calculation in a large basis set, you'll probably save quite a lot of computational time by seeding it with a calculation with a smaller, reasonable basis set. (STO-3G, MINI, 3-21G etc may not be good enough for this purpose.)

PS. A more elegant approach that doesn't introduce any projection errors is to compute the Fock matrix directly in the new basis with the orbitals expressed in the old basis; this approach described in Chem. Phys. Lett. 531, 229 (2012) requires the evaluation of cross-basis two-electron integrals which aren't available in most quantum chemistry codes.

Addendum 1: as Nike's answer states for Q-Chem, it's also possible to use the approach to project the Fock matrix between basis sets. However, this may lead to odd results. If the new basis is larger than the old one (which is the typical use case), then the null space of the old basis, i.e. any functions in the new basis that can't be represented in the old basis, will yield zero elements in the projected Fock matrix. If you had any orbitals with positive orbital energies in the guess calculation, when you diagonalize the projected Fock operator, you'll actually occupy garbage orbitals, instead, per the Aufbau principle. Projecting the orbitals is thereby a better solution.

Answer 2

What you're describing is very common, and is not limited to GAMESS and Q-Chem.

First, here's how to do it in MOLPRO, MRCC, GAMESS, Q-Chem (in fact the only electronic structure software that I regularly use which doesn't allow this type of projection into a bigger basis set, is CFOUR):

1) MOLPRO:

basis=cc-pVDZ
hf           
basis=cc-pVQZ
hf

The HF calculation with the QZ basis set will be done wtih an initial guess taken from the DZ basis set. The manual might not say what is exactly happening, but it's likely to be very similar to what is done in MRCC and Q-Chem, which is to use the density obtained from the previous SCF calculation (see below).

2) MRCC:

Do your calculation with the smaller basis set, such as cc-pVDZ, then save the SCFDENSITIES and VARS files and make sure they're in the folder when you're running the calculation with the bigger basis set, such as cc-pVQZ, and used the keyword scfiguess=restart which means "SCF initial guess" is restarted from the SCFDENSITIES file.

Also it seems in the 2020 manual that there is now a new way to do it:

"To restart an SCF calculation with the cc-pVQZ basis set from the densities obtained with the cc-pVDZ basis give:
basis=cc-pVQZ
basis_sm=cc-pVDZ
scfiguess=small"

3) GAMESS(US):

You already mentioned to add: GUESS=RDMINI to the GUESS=RDMINI

4) Q-Chem:

As with the first method I suggested with MRCC, do the calculation in the smaller basis set, then run a new calculation with the bigger basis set and the keyword SCF_GUESS = READ.

Another way is to use (as mentioned in the link you provided) the keyword BASIS2 to give the name of the smaller basis set that you want to use, then you have the option of either using BASISPROJTYPE=FOPPROJECTION (if you want to expand the MOs from the BASIS2 calculation into the larger basis) BASISPROJTYPE=OVPROJECTION (if you want to construct the Fock matrix in the larger basis set using the density matrix of the initial, smaller basis set, which is what the MRCC manual says that MRCC does, then diagonalize the density matrix in the larger basis to obtain the natural orbitals as an initial guess for the SCF calculation in the larger basis).

---

Here's some references related to your question, that you may find helpful and/or interesting:

• This 1973 paper by William Fink: ""Projection between basis sets: A means of defining the electronic structure of functional subunits" explains a way to do basis set projection involving an orthogonal basis set, then gives an example where a representation is obtained for a methyl group, from the result of an SCF calculation on methane.

• This more recent 2014 paper by Soriano and Palacios: "Theory of projections with non-orthogonal basis sets: Partitioning techniques and effective Hamiltonians" may shed some light for you for non-orthogonal bases.

• While this is a purely linear algebra perspective with no chemistry context, and is for projections onto subspaces, you may still learn a lot from this Khan Academy lesson "Projections onto subspaces with orthonormal bases".

Answer 3

Note: I have corrected some omissions from my initial answer to avoid any confusion. For a rigorous explanation of the process, I would recommend looking at Susi's answer.

The projection process for MOs is actually fairly straightforward. In the following, let $m$ be the dimension of the new basis, $n$ the dimension of the old. Even though getting a guess should only require projecting the occupied orbitals, I will show the steps assuming you are projecting all the MOs.

1. Projection $$\mathbf{C}_{\text{new}(m\times n)}=\mathbf{S}^{-1}_{\text{new}(m\times m)}\mathbf{M}_{(m\times n)}\mathbf{C}_{\text{old}(n\times n)}$$ Here $\mathbf{M}$ is the mixed overlap matrix for the two bases, i.e. $M_{ij}=\langle\psi_i|\phi_j\rangle$ with $\psi$ functions in the new basis and $\phi$ in the old. I don't have a full explanation of why this gives the least-squares projection, but it makes a certain amount of intuitive sense: the mixed overlap tells you how similar elements of each basis are, so this should say roughly say how much of the space of an old basis function is represented by a new basis function. In Gaussian, a matrix $\mathbf{U}$ is also left multiplied in this transformation. This matrix is derived from a Cholesky decomposition and is used to remove linear dependencies from the new basis.

2. Orthogonalization $$\mathbf{C}'_{\text{new}(m\times n)}=\mathbf{C}_{\text{new}(m\times n)}\mathbf{Q}^{-1/2}_{\text{new}(n\times n)}$$ $$\mathbf{Q}=\mathbf{C}^T_{\text{new}(n\times m)}\mathbf{C}_{\text{new}(m\times n)}$$ This step ensures the new MOs are orthogonal to each other (try multiplying out $\mathbf{C}'^T\mathbf{C}'$ to see that it gives the identity matrix). The reason we form $\mathbf{Q}$ is that we can't directly form a unique inverse for $\mathbf{C}_\text{new}$ since it is rectangular.

3. Create additional orbitals: If we want to have a full set of $m$ MOs for the new basis, we can simply generate additional orthogonal vectors from the projected orbitals using (for example) Gram-Schmidt orthogonalization.