4
$\begingroup$

Suppose one has a protein of about 1 million Da, and performs a DLPNO-CCSD(T)/def2-TZVPPD/AutoAux/AutoAux calculation, and then performs a wB97X-2-D/def2-TZVPPD/AutoAux/AutoAux calculation, both via the latest version of the ORCA module and both with default settings for everything, except that the relaxed density be used for the "-2" part. Both on a Surface Pro 8(client-side), via a WebMO Basic server.

Notwithstanding the very likely chance of my Surface breaking down completely(as this is a thought experiment; it's more likely that I'll be consulting the undergrad institution I'm enrolled in and/or commercial sources for the calculations in real life), how much would each calculation cost in CPU time, any last integer-part digits in a scale of hours thrown away(as an approximation must be made to estimate CPU timings this large), and/or, more importantly, which would be cheaper by which order of magnitude?

$\endgroup$
8
  • 2
    $\begingroup$ Well wB97X-D would definitely be cheaper by several orders of magnitude. Running CCSD(T) on a protein is a challenge, very few groups have done that kind of calculation, I would imagine it needs a dedicated HPC cluster with huge amounts of memory and storage. $\endgroup$
    – S R Maiti
    May 26, 2023 at 8:48
  • 4
    $\begingroup$ What is the number of atoms that you are looking at? Agreeing that DLPNO-CCSD(T) is cheaper than regular CCSD(T), you should still consider the enormous amount of memory and computing power that would be required for this computation. $\endgroup$ May 26, 2023 at 9:49
  • 2
    $\begingroup$ You would need to specify the amount of RAM, number if threads, and amount if disk space available. An electronic structure calculation like this will be faster or slower depending on if you make 4GB or 8GB available (for example). $\endgroup$ May 26, 2023 at 10:44
  • 4
    $\begingroup$ A protein of 1 million daltons would be made up of over 100,000 atoms. This is completely infeasible for a code like Orca running at those levels of theory. $\endgroup$ May 26, 2023 at 14:44
  • 2
    $\begingroup$ Yea even Frank Neese advertises the DLPNO-CC as being possible for "hundreds of atoms" not "hundreds of thousands of atoms" and even 100 atoms is not easy.@AwakenYesterday I suggest you write an answer with your insights about the infeasability! $\endgroup$ May 26, 2023 at 15:17

1 Answer 1

4
$\begingroup$

While I cannot give you any hard estimates, there are a number of points that can be addressed in your question. First, this is going to be a huge calculation, no matter what, even a single point with a super simple non-hybrid DFT functional like PBE is going to be very demanding. Second, while you specified Orca as your software of choice, most of the points I make should be fairly generic across the various state-of-the-art linear-scaling CC implementations in Orca, Molpro, MRCC, etc.

With that in mind, using DF/RI (and also probably integral-direct) methods will be unavoidable. Using AutoAux would be a bad idea, it generates large unoptimized auxiliary basis sets that generally should only be used if there is no optimized aux basis for the main basis you are using. I have not checked, but I would think that def2-TZVPPD is popular enough that aux basis availability is not a problem.

I am not familiar with wB97X-2-D and a brief google search had no relevant hits, but "except that the relaxed density be used for the "-2" part" suggests it might be a new double hybrid. This raises more problems, as computing regular RI-MP2 would not be feasible, so you would need to use DLPNO-MP2 or whatever linear-scaling MP2 implementation you have available to compute the PT2 contribution, which might be technical a problem as some implementations might not support using non-HF references.

Either way, trying to use a relaxed (DLPNO-)MP2 density in your double hybrid is probably not going to be feasible, even if it is implemented (which I doubt), computing the relaxed MP2 density is somewhat expensive. But IIRC many double hybrids perform just fine without relaxed MP2 density, so if wB97X-2-D really demands the use of relaxed densities, you would probably want to consider using a different functional that does not require the relaxed MP2 density.

The CC calculation is going to be an enormous challenge to complete. Forget hours, we are deep in the months and years territory. But there is some hope with bleeding edge software and hardware coming in the next 10 years. Take a look at this paper from the MRCC developers, where they have completed a single point LNO-CCSD(T)/def2-QZVP for a protein of 1023 atoms.

One of the most interesting observations they make, is that past ~500 atoms the post-HF part of LNO-CCSD(T) starts getting close to linear scaling, while the DF-HF step has cubic scaling and overtakes LNO-CCSD(T)! Indeed, for their 1 kiloatom LNO-CCSD(T)/def2-QZVP calculation DF-HF took 38 days while after that the post-HF steps were done in just 18 days, all on an old 6-core CPU from 2013 (!!!) and ~100 GB of RAM.

Napkin math time! Assuming your protein is made out of CH2 units (ie. UHMWPE), you will have ~70k atoms. The post-HF part is easy to estimate if we can assume linear scaling, 18 * 70 = ~1300 days (~3.5 years) on their hardware. The DF-HF part is a real fly in the ointment: 38 * 70^3 = ~13 megadays or ~36 thousand years.

Now, before you despair, of course we have much faster CPUs today, and in fact we will soon (~2 years) reach 100x the CPU performance that the paper used (256 cores, ~10 TFLOPS FP64). They claim to have pretty good parallel scaling, so assuming that holds, if you were to buy the fastest server that money can buy in 2025, then start the calculation, you may be able to finish it in 2027. That is of course, assuming no algorithmic improvements or GPU implementation, both of which may make this much more feasible in the meanwhile.

So if you are an undergrad now, a single point CCSD(T)/CBS for this protein might become (barely) feasible by the time you finish your PhD, on something like a then-brand-new supercomputer node.

$\endgroup$
6
  • $\begingroup$ +1 for a great answer! As for finding the functional online, if you searched ωB97X-2-D instead of wB97X-2-D, you might have come across more papers. Also, on what do you base the prediction that we'll reach 100x their CPU performance within ~2 years? $\endgroup$ May 28, 2023 at 2:06
  • $\begingroup$ It is admittedly more napkin math that may be wrong by a factor of few. The LINPACK benchmark measures all-core parallel performance by timing an LU factorization, which can get pretty close to the theoretical maximum FLOPS of a CPU. A 24-core AMD CPU from last year does ~1 TFLOPS. We are at 96 cores now, in ~2 years we will have a 128-core CPU with similar per-core perf. With dual socket that is ~10 TFLOPS in 256 cores. I cannot find a LINPACK result for the CPU in the paper, but its theoretical peak is 0.168 TFLOPS. I rounded that down to 0.1 actual to get the 100x performance gain. $\endgroup$
    – TiborGY
    May 28, 2023 at 12:28
  • $\begingroup$ If we want more cores, can't we just use more nodes? $\endgroup$ May 28, 2023 at 14:14
  • $\begingroup$ Yes, if a particular implementation supports (and scales well with) the use of MPI. Implementations that rely on shared-memory parallelism (OpenMP, pthreads, C++ async, etc.) cannot use multiple nodes. $\endgroup$
    – TiborGY
    May 28, 2023 at 14:19
  • $\begingroup$ since MRCC and CFOUR (for example) use MPI, perhaps the things you wrote about being able to do calculations 2 years from now when 128-core AMD processors are available, could also be said about today? $\endgroup$ May 28, 2023 at 21:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .