What is the state of the art in terms of local atomic environment descriptors for machine learning?

Question

A lot of atomistic machine learning deals with correctly describing atomic neighborhood environments by vectors or fingerprints (see, e.g. J. Chem. Phys. 149, 244102 (2018), Phys. Rev. B 87, 184115 (2013)), before feeding those descriptions to neural networks, kernel ridge regressors or other algorithms

So the question is: what is the current status of local atomic environment descriptors for machine learning? Specifically, I want to know what makes a good descriptor and if there are any classification to them (i.e., what makes them different?).

(This question is complimentary to "What is the current status of machine learning applied to materials or molecular systems?".)

Answer 1

If you are familiar with the Behler-Parrinello symmetry functions implemented in AMP, you may be interested in seeing how they compare to other atom-centered representations in terms of speed and accuracy. Marcel F. Langer, Alex Goeßmann, and Matthias Rupp have recently released their benchmarking efforts including the symmetry functions, the Many-body Tensor Representation, and the Smooth Overlap of Atomic Positions representation. Their work also includes concise summaries of other representations to get you up to speed as well as what exactly makes a good representation:

1. Invariance to rotations, translations, and permutations
2. Uniqueness: "Systems with identical representations that differ in property introduce errors"
3. Continuity/Differentiability
4. Computational efficiency
5. Structure (e.g constant size)
6. Generality, " in the sense of being able to encode any atomistic system"

What distinguishes many representations is the choice of their basis set when encoding physical distances and angles into machine-learning inputs. Where the Behler-Parrinello symmetry functions use Gaussian functions, the Artrith-Urban-Ceder descriptor uses Chebyshev polynomials. The Many-Body Tensor Representation uses a real-space basis, while the Smooth Overlap of Atomic Positions uses spherical harmonics. Michele Ceriotti's group has released an excellent paper connecting these atom-centered representations with a general mathematical formulism.

Dr. Ceriotti is also on a paper with Gabor Csanyi where they have extensively investigated the topic of uniqueness. The paper highlights the limitations of using representations that stop at 3-body terms (i.e distances and angles).

While invariance and equivariance might be handled by the representation, there are several groups working on finding ways to handle equivariance directly with the model architecture. As far as I understand, this is especially necessary when learning tensorial properties rather than scalar properties like energy.

As Greg alluded to, there have also been efforts to create machine learning frameworks where atomic representations can be learned and tuned automatically. Schnet (or Schnetpack) is a framework that uses continuous-filter convolutional neural networks to do so.

I recommend watching these lectures from the recent Institute for Pure and Applied Mathematics program on "Machine Learning for Physics and the Physics of Learning":

Richard G. Hennig: Machine-learning for materials and physics discovery through symbolic regression and kernel methods

Tess Smidt: Euclidean Neural Networks* for Emulating Ab Initio Calculations and Generating Atomic Geometries *also called Tensor Field Networks and 3D Steerable CNNs

Anatole von Lilienfeld: Quantum Machine Learning

Michele Ceriotti: Machine learning for atomic and molecular simulations

Matthias Rupp: How to assess scientific machine learning models? Prediction errors and predictive uncertainty quantification

Gabor Csanyi: Representation and regression problems for molecular structure and dynamics