A few years ago there was a significant difference between CPUs and GPUs for performing calculations. It was quite clear when to go for a CPU or a GPU.
Today there are GPUs with a very large number of cores and threads. I was away from the subject for a while. Is there still such a marked difference? Or can a CPU reasonably well replace a GPU in, say, molecular dynamics?
Where should one put the money?
As other answers have mentioned, it depends a lot on the workload and the availability of GPU-acceleration for the codes you use (or write). In principal, there are multiple platforms, but in practical use right now, Nvidia CUDA has the best performance and largest use.
Nvidia offers a GPU Application directory - listing existing codes with CUDA availability.
Many molecular dynamics codes are already GPU-enabled and show huge speedups over CPU versions (e.g., https://developer.nvidia.com/hpc-application-performance). This includes LAMMPS, AMBER, GROMACS, NAMD.. pretty much everything. Speedup varies, but I've often seen 20x-100x show up.
Fewer quantum codes are GPU-enabled with mixed results, largely dependent on how much work has gone into writing GPU versions of the code. These include VASP, Quantum Espresso, CP2k, Abinit, BigDFT, Gaussian, GAMESS, etc. I've seen some good benchmarks for commercial GPU acceleration of Q-Chem called BrianQC and Terachem but these are not widely deployed. On codes like Quantum Espresso and VASP, even Nvidia is quoting a 10-20x speedup.
For machine learning, speedup can be much, much more dramatic if your workload can be stored on the GPU. Data Center GPUs can be 16GB to 32GB in the Tesla V100 cards, and now 40GB in the A100 cards, although there's some overhead (i.e. you don't get to use all of that for your data and model).
It depends a lot on your application. All problems can be solved on a CPU, some problems can be solved much faster on a GPU.
One thing to consider is the effort required to write your code. If you're writing your own code from scratch, adapting it for the GPU can be a lot of work, and potentially for little to no reward. On the other hand, if you're using software packages or certain libraries, enabling GPU support might be as simple as checking a box.
Are you talking about configuring your own personal/work computer? How much do you plan on running your code on your machine itself (rather than a cluster, or dedicated workstation). If you're going to be running your code on a cluster, the amount of simulation you can get done on your personal machine won't matter much. In that case, you might want just enough CPU and GPU to do development and short tests.
Generally, CPUs give you the best bang for buck. While GPUs are promising, support for them is lacking in many codes. Moreover, even if some codes do support GPUs, the speedups are embarassingly small. Good GPUs are quite expensive: e.g. NVidia cards cost several thousands of dollars, and to get the best efficiency one might need several of them. For this price you can just get more CPUs.
So, short answer: check first if the program you want to run supports GPUs, and what is the speedup for a representative calculation. (Note that the published numbers may not match your use case!) Then, do the math: is spending money on a GPU worth it?
So long as your calculations are parallelizable, using GPU's should provide significant speedup over just using CPU's alone. Nvidia last week at their GTC conference announced their new A100 GPU, which provides up to 20X speedup over its predecessor, the Volta V100. Nvidia debuts the A100, its most powerful graphics processor yet. Each A100 can deliver up to 600 Teraflops of performance.
A new DGX server with 8 A100's, while delivering 5PetaFlops performance (which is equivalent to the performance of the world's largest supercomputer a few years ago), will set you back around $200k. While this can be out of the price range of a lot of (but not all) academic labs, a cost-effective option could be a pay-as-you-go machine instance on AWS, or another cloud provider such as GCP or Azure. An example of a relevant deep learning AMI can be found here.
