Replica Exchange
Methods like temperature replica exchange and Hamiltonian replica exchange can help to sample phase space more uniformly. For temperature replica exchange, we have the base temperature that we are interested in (often room temperature or body temperature) and a number of other replicas with higher temperatures. For instance, we could have 10 replicas with temperatures of 300, 350, 400, 450, 500, 550, 600, 650, 700, and 750 K. We run simulations at these temperatures simultaneously. At the higher temperatures, enthalpic barriers easier to cross (they are smaller relative to thermal energy, $k_\mathrm{B}T$) and the kinetics are faster. For example, while the cis and trans amide conformations of proline have similar energy, there is a large barrier between (>10 kcal/mol) them. At 300 K, the transitions between the cis and trans will never occur on the simulation timescale (microseconds). You could probably run continuously for a year and never see such a transition. However, transitions can be observed within a few nanoseconds at 750 K. So, our high temperature simulations will explore more of conformational space. We can take advantage of this by applying a Monte Carlo procedure that improves sampling at the base temperature (300 K), while maintaining correct $NVT$ ensemble at this temperature.
For example, we can let the simulations run for each replica and every 1000 steps attempt exchanges of atomic coordinates between the replicas. We calculate the probability of the exchange by:
$$ p_\mathrm{exchange} = \min \left\{ 1,\frac{\exp(- \beta_i V_j - \beta_j V_i)}{\exp(-\beta_i V_i - \beta_j V_j)} \right\},\tag{1}$$
where $i$ and $j$ are the indices of two replicas, $V_i$ is the potential energy of replica $i$ before the exchange attempt, and $\beta_i = 1/(k_\mathrm{B} T_i)$ is the inverse temperature of the ensemble for replica $i$. We can see $\frac{\exp(- \beta_i V_j - \beta_j V_i)}{\exp(-\beta_i V_i - \beta_j V_j)}$, as the ratio of the probability of observing the states of the two replicas after exchange to the probability of observing the states of the two replicas before exchange. If this ratio is larger than one, then we accept the exchange immediately. If this ratio is less than one, we generate a random number on [0,1] and accept the exchange if that number is less than the probability ratio. If the atomic configurations are exchanged, we reset the velocities from a Maxwell-Boltzmann distribution at the destination temperature (alternatively, we can include the kinetic energy in the probability calculation and exchange velocities too).
Hence, barriers can be crossed at high temperature and the resulting atomic configurations can be passed down to lower temperatures. The reason that we often need many temperatures is that, if the temperatures are too far apart, the energies distributions will not overlap, $p$ will be very small, and the exchanges will rarely, if ever, be accepted. We need exchanges to be accepted to get the enhanced sampling we desire. So the temperature range and number of replicas must be chosen carefully to both obtain better exploration of phase space (usually promoted by a higher maximum temperature) and a reasonably high acceptance rate (promoted by having a small difference in temperature between adjacent replicas).
A major problem with the temperature replica exchange scheme is that it does not scale well as the number of degrees of freedom in the system is increased. The energy fluctuation (standard deviation of the energy distribution) of a system in the $NVT$ ensemble is $\sigma_E = \sqrt{k_\mathrm{B} T^2 C_V}$, where $C_V$ is the heat capacity at constant volume. The heat capacity is typically proportional to the number of particles $N$, so the width of the energy distribution grows like $\sqrt{N}$. However, the mean energy typically grows like $N$, so if we double the size of the system, the energy distributions between simulations at 300 and 350 K will overlap less.
Hamiltonian replica exchange can be more targeted and is thus better suited for systems with large numbers of degrees of freedom. In this case, we have a base Hamiltonian, which is our unmodified and presumably realistic force field. The other replicas have an increasingly more biased potential energy. The biasing potential is chosen to help us to sample the transitions we want (for example, conformational transitions in a peptide), while not changing interactions for other degrees of freedom (for example, water–water interactions). Again, we expect faster exploration of phase space for the most biased replicas and the resulting conformations can be exchanged down to the base (unbiased) replica. We attempt exchanges every so many steps and accept them with the probability
$$ p_\mathrm{exchange} = \min \left\{ 1,\frac{\exp(-\beta V_i(\mathbf{X}_j) - \beta V_j(\mathbf{X}_i)}{\exp(-\beta V_i(\mathbf{X}_i) - \beta V_j(\mathbf{X}_j)} \right\},\tag{2}$$
where $\beta$ is the constant temperature for all replicas, $V_i$ is the potential energy function for replica $i$, and $\mathbf{X}_i$ is the set of all atomic coordinates for replica $i$ before the exchange attempt.