There was some recent media reporting about a purported Google breakthrough on applying machine learning techniques to tackle the protein folding problem, as told for example in this news article, DeepMind AI handles protein folding, which humbled previous software.

Unfortunately there is not much details, as no peer-reviewed paper was published yet. But supposing the paper is eventually published, and the claims confirmed as legitimate and not just hot air, what are the implications? I understand that a reliable way to predict the folding patterns of proteins in silico could be a huge step over the experimental means like x-ray crystallography, that sometimes requires even crystals grown in space at great cost. Besides the large quantitative but incremental cost and time savings, are there any non-obvious qualitative differences in the kind of research enabled if such a breaktrough is confirmed?

  • 3
    $\begingroup$ DeepMind also played a part in previous CASP editions, they did publish a paper at that time (some months later): nature.com/articles/s41586-019-1923-7 with some of the details. They have some of the code and models in a github repo: github.com/deepmind/deepmind-research/tree/master/… but all of the actual interesting stuff, which is commonly the feature engineering/extraction in ML/AI is not open or readily available. They do have some comments on it. $\endgroup$ – IvanP Dec 2 '20 at 16:44
  • 2
    $\begingroup$ Also, there are other similar approaches from around the same time, one of the most noted ones is linkinghub.elsevier.com/retrieve/pii/S2405471219300766 that may give more details on the method. $\endgroup$ – IvanP Dec 2 '20 at 16:50

Imagine if given an amino acid sequence, you could quickly calculate what the shape of the corresponding protein would be.

  • You would be able to predict what effect a mutation would have on the shape of the protein. Switching just one glutamic acid with valine completely changes the shape of hemoglobin to the extent that people with this mutation are said to have a disease called "sickle cell anemia". Likewise, there might be a substitution that we haven't even discovered yet that would increase our capacity to inhale oxygen, leading to super-human strength just as the glutamic acid to valine substitution causes can have severe implications (sick cell anemia).
  • You could do inverse design. For example if you want a protein that looks and can behave like a stapler, you could search for the amino acid sequence that gives you the protein shaped the way you want, a bit like DNA origami but without the need for staples:

enter image description here

So what did DeepMind announce 2 days ago?

First of all, the link you provided is to an article about what they did. This is the original blog post that DeepMind published 2 days ago.

What they announced was that they did very well in the CASP 14 competition. This is a competition for protein folding predictions that has been going on every two years since 1993 and this is the 14th competition that has taken place. In this competition, researchers try to predict the structure of various proteins, and the predictions are evaluated based on the Global Distance Test. In 2018 DeepMind won the CASP 13 competition with a median score of almost 60 GDT, whereas the winning teams in CASP 7 (2006) to CASP 12 (2016) never got far above 40 GDT.

In the 2020 competition, DeepMind shattered this record again, by getting a median score of 92.4 GDT, which means that atom positions were predicted correctly within 1.6 Angstroms.

Was it peer reviewed?

This was a competition, for which impartial judges evaluate contestants quantitatively based on a single number, the GDP. This is even better than peer review. Peer review for journals, does not involve the referees actually testing the authors' software to see if it reproduces some standard benchmark: it is just a process where scientists guess whether or not they believe that the authors' calculation really did do what they said they did. This competition was a bit like a race, where Usain Bolt was determined by Olympic judges to break the record for the 100m sprint, and non-judges can also believe it because they for the most part saw it happen on TV, just like you can check the results of the 2020 protein folding competition here.

Then why are people saying that they haven't submitted the paper yet?

They won a major competition, so now the authors can get credit for it in the way that is most relevant for their careers as researchers: by publishing in Nature or Science and getting citations on Google Scholar for it. They will likely just report the results of the competition, which we already know. They might reveal some details about any algorithmic developments they made between 2018 and 2020, but they are also a private company and might not reveal everything the way a research institute not owned by a for-profit company sometimes would by institutional policy be required.

So was a "50-year-old problem solved", as they claim?

Not really. The particular benchmark set for CASP 14 was solved to an accuracy that had never been achieved by any research group for any previous CASP benchmark set. However, next time the CASP benchmark set can just be made more difficult and they will be back to not being able to predict the protein structures.

It is however a significant achievement that the protein structures in this benchmark set (which was supposed to be hard by 2020 standards) were reproduced with atom positions correct to about 1.6 Angstroms. It means that scientists can now be that sure about their protein folding predictions for proteins of similar complexity to the ones in the CASP 14 competition set.

Update! After this question, someone else asked a question which pointed out that AlphaFold might struggle for some important types of proteins such hemoglobin, here: Does DeepMind's new protein folding software (AlphaFold) also work well for metalloproteins (proteins with metal cofactors)?

  • 2
    $\begingroup$ That's a great answer. Do you think you could add examples of proteins with structures that are easy to fold, medium-hard (like you imply the CASP dataset is), or really hard? $\endgroup$ – svavil Dec 2 '20 at 22:04
  • 2
    $\begingroup$ @svavil Thanks. Well insulin, which is only 51 amino acids, is easy. TAL which is probably the shortest protein known (11 amino acids) is easy. Hemoglobin is hard because it has a quaternary structure, which means multiple separate polypeptide chains that have their own primary or secondary structures, such as alpha helices. A protein that only has primary, secondary tertiary structures, is going to be much easier than hemoglobin that has quaternary structure. Hemoglobin has 4 polypeptide chains. If DeepMind can fold it then let's make the next CASP have a protein with 15 polypeptide chains! $\endgroup$ – Nike Dattani Dec 3 '20 at 0:09
  • 4
    $\begingroup$ It's perhaps worth noting that even in CASP 14, they couldn't solve every problem to a satisfactory level. There are plenty of targets where even the best solution is <80% correct. Consider T1027-D1, Gaussia luciferase, where AlphaFold2 gets 61.11 of atoms correct. $\endgroup$ – Tom Anderson Dec 3 '20 at 13:30
  • 1
    $\begingroup$ It's also worth noting that the GDT_TS metric being used for the competition considers a residue 'correct' if it is within 8 Å of where it should be. An X-ray structure is considered high-resolution if the atom is localised to within 1 Å. 1 - 3 Å is considered okay, and 3 - 6 Å is low resolution. There is also a "high accuracy" metric, GDT_HA, which counts atoms within 4 Å - which is still low-resolution by experimental standards! $\endgroup$ – Tom Anderson Dec 3 '20 at 13:34
  • 3
    $\begingroup$ @NikeDattani It's not really a complete answer. Also, after i made those comments, i had a chance to discuss a bit with a friend who is a structural biologist, and he made a couple of interesting points. Firstly, for him, a better metric is the RMSD for Cα (RMS_CA), which measures how accurately the backbone is placed; AlphaFold2 apparently got an average of 1.6Å, which is really good. Sidechains less good, but hey. Secondly, GLuc is an NMR structure - suggesting either it has a weird structure which won't crystallise, or that AlphaFold2 has learned to mimic crystal structures, but not NMR! $\endgroup$ – Tom Anderson Dec 4 '20 at 18:23

Probably one of the important applications is Computer Aided Drug Discovery (CADD). If the protein structure could be accurately predicted, one could design protein-ligand docking on the binding pockets and run molecule dynamics simulations.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.