6
$\begingroup$

I built a prediction model with fingerprints from 300 molecules and got an R2 of 0.9 However when I go to perform a cross-validation I get a very low R2. How can I improve this result? I'm using sklearn and PaDEL descriptors, maybe I'm doing something wrong though. Where can I learn how to do it correctly?

$\endgroup$

1 Answer 1

7
$\begingroup$

You don't give a lot of details (e.g., type of model, target property, etc.), but with only 300 molecules, it's very likely that your model is over-fit. Thus when you cross-validate, you get low $R^2$.

Consider that each descriptor you add is at least one parameter. You'll want more data than you have parameters.

  • You want to look for more data if possible
  • You'll want to use LASSO or other method to remove descriptors that don't add much information
  • You'll want to use cross-validation (e.g., 5-fold or 10-fold CV) from the start to minimize the chance of an over fit model

But you also should think carefully about descriptors. Consider whether the descriptors you use would be meaningful for the model you want.

Let's say I wanted a model for solubility prediction. I can think of a few properties / descriptors that might be important:

  • Molecular size (molecular weight or volume)
  • Molecular shape (like dissolves like)
  • Hydrogen bond donors and acceptors
  • Dipole moment / electrostatics
  • etc.

Yet for other properties (e.g., redox potential) most of these are probably less relevant.

If you want a good model, you need to be careful with what descriptors you use.

$\endgroup$
1
  • $\begingroup$ Thank you so much for your reply, it is true I gave little information. As a first attempt I tried to build a model to predict solubility and got good results using few descriptors. As a second attempt I tried to predict pIC50 to a generic cancer cell with PaDEL descriptors but I get really poor results. Maybe I need to make more attempts until I find the ideal descriptors? I get a very high R2 for the full dataset but very low in the cross validation $\endgroup$
    – vorsamaqoy
    Feb 23 at 9:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.