# Is the dynamics of a peptide molecule Markovian?

I was wondering if the underlying dynamics of a full atomistic trajectory of a peptide molecule are Markovian.

Note added after the first two answers: I assumed that they were Markovian, as they are simulation of a physical system, but the underlying dynamics and the interactions of atoms change as the molecule's configuration falls into different conformations as time progresses. This makes me think that it may be non-Markovian.

PS: Note that the questions the trajectories in the 3D space, not their projection on the configuration space.

Warning: In the physical literature the epithet "Markov" is often used with regrettable looseness. The term has a magical appeal, which invites itself in an intuitive sense not covered by the definition. [...] it is meaningless to ask whether or not it is Markovian, unless one specifies the variables to be used for its description.

That warning is a quote from one of the most famous books in matter modeling: "Stochastic Processes in Physics and Chemistry" by Nico van Kampen. The book has 15967 citations on Google Scholar at the moment, and that specific quote has been reused elsewhere, such as in this free PDF book on Pg. 51.

When a process is Markovian, it essentially means that the probability density function at time $$t_{n+1}$$ can entirely be obtained from the function's current value at $$t_n$$, and does not depend explicitly on previous times such as $$t_{n-1}, t_{n-2},$$ etc. It has no memory of the past.

The definition can be generalized beyond just probability distribution functions, because (for example) in quantum mechanics we have the density matrix where the diagonals are probabilities, but the off-diagonals are something that doesn't even exist in classical theory. If the density matrix evolves like:

$$\tag{1} \rho(t_{n+1}) = e^{-\frac{\textrm{i}}{\hbar}Ht_{n+1}}\rho(t_n)e^{\frac{\textrm{i}}{\hbar}Ht_{n+1}}$$

then the evolution we can obtain $$\rho(t_{n+1})$$ from only $$\rho(t_{n})$$. Then we can obtain $$\rho(t_{n+2})$$ from only $$\rho(t_{n+1})$$, and $$\rho(t_{n+3})$$ from only $$\rho(t_{n+2})$$, and what we get is a Markov chain.

If we ignore gravity (for which we don't yet have a quantum theory), we can say that the whole universe undergoes Markovian evolution according to Eq. (1). But if we want to look at a sub-system of the universe (such as a peptide molecule), then we cannot just calculate $$\rho_{\textrm{subsystem}}(t_{n+1})$$ by just knowing $$\rho_{\textrm{subsystem}}$$ at some previous time such as $$t_{n}$$, the evolution of $$\rho_{\textrm{subsystem}}$$ will require starting at $$\rho_{\textrm{subsystem}}(t=0)$$ and doing a Feynman integral over all the history of what the rest of the universe did to the subsytem (for example, see Eq. 8 of my paper which described an open source MATLAB code to easily calculate this Feynman integral).

So neglecting gravity, quantum mechanics tells us that the evolution of the universe is Markovian, and the evolution of any sub-system of the universe (such as a peptide molecule) that is not completely isolated from the influence of any other part of the universe, undergoes non-Markovian evolution.

Here the variable to which I'm referring is the density matrix. Since non-Markovian dynamics can be hard to calculate, we often approximate the dynamics by a Markovian master equation but this is an approximation to the true non-Markovian dynamics of any system that is not the entire universe (including vacuum vibrations in QED) itself.

Now in classical MD, which is an approximation of quantum dynamics, but good enough for most studies (!), the answer by dwhswenson points out that if considering the full phase space (analogous to the full universe, in my above description), then the probability of going to another state follows Markovian dynamics in that it only depends on the current state (positions and velocities of all particles, notices that the we are specifying all the "variables" that van Kampen warns us to make sure we specify!). Notice also that any slight influence from outside a subsystem (i.e. from the rest of the universe) can bring the subsystem out of its equilibrium, and that his answer also says that for this non-equilibrium dynamics, the Markov approximation is no longer valid. Regarding the rest of his answer, it's certainly possible very often to approximate dynamics by a Markov process, and whether or not these approximations are sufficient for your purposes will depend on your specific study.

If considering the full phase space, equilibrium classical MD (with a stochastic thermostat) is Markovian. The probability of going to another microstate (phase space point) is fully determined by the current positions and velocities. That probability is defined by your MD integrator. Since it is a mathematical property, it is true for any time step size. Time steps that are too large could mean that the trajectory wouldn't be a good model of physical reality; MD as an algorithm is what is Markovian.

On the caveats I listed:

• Without the full phase space (i.e., if you ignore velocities and only map coordinates, or some subset of the coordinates) then you may lose the Markovianity. This can be true of any Markovian process, though. Reducing dimensionality is hard.
• Non-equilibrium dynamics can introduce a time dependence on the transition probabilities, which breaks the Markov assumption in a formal sense.
• If you don't have a stochastic component to your integration scheme, then it isn't a stochastic process, and therefore Markov theory doesn't apply.

However, there's a related question that might be more your interest: Can Markov models be used to represent the dynamics of conformational transitions in a peptide? The answer to that is often yes, but with some different reasoning. There's a whole field of people developing Markov state models from classical MD.

Here are a couple of papers on that, both with examples using peptides:

The big difference is in the definition of your states. You will want a coarser discretization than "every possible point in phase space." You'll also want to introduce a lag time: essentially, the number of MD time steps that you combine to call one "step" in your Markov process. Transitions between states are only become Markovian after sufficient lag time.

You'll often hear about "implied timescales" in Markov state modeling. This is a measure of how quickly the process will reach equilibrium. If you increase the lag time in a Markovian process, the implied timescales should not change. This is often used as a check on a Markov state model, but it's important to remember that the implication is actually the other way: Markovian implies independence with respect to lag time, not vice versa.

Aside (added after comment from OP): In condensed phase systems, you can often treat the saved trajectory positions as an approximation to Brownian motion (which is certainly Markovian). This is because there's big difference between the time step used in the dynamics (order of femtosecond) and the frequency at which frames are saved to disk (usually picoseconds or longer), and condensed phase systems are chaotic. General argument: by the time there's no memory of the initial velocities (i.e., if the velocity autocorrelation function has decayed to zero), the initial velocities might as well have been random.

That's far from a proof that it is Markovian, but the argument is that if the velocities don't matter, then positions might as well be "the full universe", as Nike Dattani's answer discusses.

• Thanks! What I understand from reading couple of MSM papers is that these models deal with snapshots at discrete time steps. But what about an MD simulation like this: markovmodel.github.io/mdshare/ALA2? If I want to reproduce this trajectory using a machine learning model, should I assume a Markovian model? (at least the web address implies that!) – Blade Feb 8 at 16:04
• @Blade: Edited to put the info in the answer, but basically the answer is yes, although you might need to have your ML method map a few frames ahead instead of the very next frame. Dynamics will be close to Brownian at 1ps, almost definitely Brownian at 10ps. – dwhswenson Feb 8 at 17:18
• @Blade Oh, and the web site name is because that's Frank Noé's group. Frank was the corresponding author on the first MSM paper I linked to. Much of their work is on developing ways to represent long timescale dynamics with Markov state models. – dwhswenson Feb 8 at 17:22
• Thanks! I did some readings and the way I understand it now is that the trajectories themselves are Makovian, but when we map it on a conformation space they may not necessarily be Markovian (e.g. this study finds time lag threshold for Markovianity: doi:10.1063/1.3328781). Quick question: the dataset from Frank Noé's group doesn't contain velocity info. To predict the trajectories in future time step with Markovian assumption, should I also include velocity information? Do MD methods provide velocity information as well? Should I just divid the displacement over dt=1ps to find the velocity? – Blade Feb 10 at 1:20