This question is somewhat broad, but hopefully I can convey my point and elicit some worthwhile discussion.
One of the fundamental difficulties of machine learning is trying to develop a model that will work beyond your training set. The issue is that it has proven very challenging to develop a machine learning model that can extrapolate (e.g. derive insight about unseen, "outer" regions of the input space, with "outer" meaning input parameters larger/smaller than any in the test set). So instead, one often tries to develop the training set such that they can interpolate (e.g. derive insight about unseen, "inner" regions where the input parameters are between ones from the training set). However, the topography of a given input space and how it relates to some output property are precisely the problems one is hoping to solve with machine learning!
So my question is, what are some general techniques that Matter Modelers use to "smartly" sample input? What sort of intuition can we apply to ensure our training set has a wide boundary and that we are interpolating, rather than extrapolating, when applying our model to a test set? For some properties, this is fairly intuitive. For example , in developing a training set to model a potential energy surface, chose their training set by iteratively adding points that maximized the minimum distance from any existing point in the set. Here, the sample space is over physical distance, so there is an intuitive way to determine a boundary. But what sort of guidelines are available to make this selection more generalizable to other types of input without as clear a notion of distance?
- Dral, Pavlo O.; Owens, Alec; Yurchenko, Sergei N.; Thiel, Walter J. Chem. Phys. 146, 244108 (2017); DOI:10.1063/1.4989536