19
$\begingroup$

A few years back, I performed a few calculations of metallic slabs using VASP and the Atomic Simulation Environment (ASE). I'm by no means an expert, but I noticed that the calculations were very CPU intensive e.g. ~24 cpu-hours (in parallel) to get a single energy point and gradient calculation.

Considering the size of the system this makes sense, however, I'm wondering, have graphics processing units (GPUs) changed this scenario much? Is the use of GPUs becoming more popular in plane-wave DFT?

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ Out of the dozen or so programs that are listed here as using PW basis sets and having GPU ported features, VASP does appear in this list, so maybe they added the capabilities to run on GPUs slightly after you did those calculations (Quantum ESPRESSO, CP2K and other programs also are listed as using PWs and being able to run at least some parts of the code on GPUs). $\endgroup$
    – Nike Dattani - No Free Time
    Commented May 15, 2020 at 1:51

4 Answers 4

Reset to default
5
$\begingroup$

I recently installed VASP GPU version provided by NVIDIA (here's the installation tutorial) on my machine that has an RTX 2080 Ti GPU. GPU version was announced for vasp-5.4.1 and works fairly well for vasp-5.4.4 too. However, I have only managed to observe around 1.5x - 2x speedup when running VASP on the GPU compared to my Intel(R) Core(TM) i7-8700 CPU with 12 cores. But there is definitely some degree of speedup. Maybe the GPU version of VASP is meant for more powerful (and expensive) double-precision GPUs such as Titan V or Tesla V100. While my GPU performs exceptionally well on running deep learning applications, I feel it can't keep up with its more powerful counterparts for VASP calculations.

Furthermore, I use intel's mpi library to run VASP. Surprisingly, mpirun -n 1 vasp_gpu is much faster than running on 2(or more) cores (mpirun -n 2 vasp_gpu) even though more cores consumed more GPU memory. Using more cores is even slower than running solely on CPU with vasp_std. I am currently investigating what's the reason for this and greatly appreciate if any other user with experience contribute to this answer.

Improve this answer
$\endgroup$
2
  • 3
    $\begingroup$ This is an excellent answer, the tutorial is very clear and informative. You should consider answering materials.stackexchange.com/q/883/52 $\endgroup$
    – Cody Aldaz
    Commented Jun 13, 2020 at 14:04
  • 1
    $\begingroup$ Yes, QM calculations need double precision, which is slow on your card. $\endgroup$
    – Greg
    Commented Jun 14, 2020 at 14:09
12
$\begingroup$

I can't say much about the popularity of GPUs in practical calculations.

From a development point of view the speedups that can be expected from GPUs in plane-wave DFT are only moderate, probably around 2 to 3, maybe 7 if you are optimistic. See for example the paper describing the VASP implementation [1] or this stackoverflow question illustrating speedups of 2 to 3 for the key computational step in plane-wave FFT, namely the fast-fourier transfrom (FFT).

This might not sound so bad, but keep in mind the extra requirements in computational power, which is also about a factor of 2 or higher, the price of the GPU (on top of a CPU you need anyway!) and the added complexity inside the code, which might well prevent the implementation of other faster algorithms in the future. With that in mind, I would say GPUs have potential for large-scale cutting edge calculations, but are not able to change the picture much over CPUs for the type of calculations you describe (a few days or less): You might as well buy another CPU rack and just use that on top.

References

[1] S. Maintz, B. Eck, R. Dronskowski. Speeding up plane-wave electronic-structure calculations using graphics-processing units, Comp. Phys. Comm. 7, 1421 (2011) DOI 10.1016/j.cpc.2011.03.010.

Improve this answer
$\endgroup$
10
$\begingroup$

While I can't comment on concrete speed-ups as I'm not very familiar with GPU programming itself, I would like to point out that most of the computing time is spent in FFT and GEMM (matrix matrix multipicaltions) calls. A friend of mine tested this for the GPAW code where those two thing accounted for > 70 % of the CPU time. I imagine it is similar for other programs.

In principle, speeding up these functions should result in a significant overall performance increase. That said, additions/changes to the core functionality of popular programs like VASP can yield underwhelming results. An example for that would be the k-point parallelism in VASP that shows extreme data redundancy.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for pointing out the GEMMs that's a point I did not go into in my above argument. $\endgroup$
    – Michael F. Herbst
    Commented May 15, 2020 at 17:52
9
$\begingroup$

I experienced[1] about 5-10 times speedup with GPU accelerated Quantum Espresso on Tesla V100 32 Gb compared with Intel Core i7 9700K processor with 8 cores and 32 Gb RAM. The above mentioned system's volume was about 125 cubic Angstroms, it had about 19 atoms, 5 k-points, ecutwfc = 80 Ry, ecutrho = 320 Ry. 24 SCF iterations took 54 seconds with GPU, and sadly I do not have timings for CPU-only calculations. It took some efforts to build GPU version. It is still in beta version, but I didn't face any bugs. So it is reasonable to expect GPU acceleration in the nearest major release.

References:

  1. Mikhail A. Syroeshkin et al. 2‐Carboxyethylgermanium sesquioxide as a promising anode material for Li‐ion batteries. Chemistry Europe, 2020,DOI: 10.1002/cssc.202000852
Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Very nice, this is exactly the type of answer that I'm looking for. It is sad, however, that is one of the most expensive GPU on the market ($12,000) The paper that Micheal posted is 8 years old and boasted a 3-7x speedup. $\endgroup$
    – Cody Aldaz
    Commented May 17, 2020 at 5:09
  • $\begingroup$ QM calculations require double precision. Consumer-level ("gamer") cards have very limited double-precision capability, so you need architectures that are specifically supporting double-precision calculations, like the Tesla cards. Also, $\endgroup$
    – Greg
    Commented Jun 14, 2020 at 14:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .