Command to run QE-GPU

Question

I have installed quantum espresso with GPU support on a system.

Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-135-generic x86_64)

System Configuration: Processor: Intel Xeon Gold 5120 CPU 2.20 GHz (2 Processor) RAM: 96 GB HDD: 6 TB Graphics Card: NVIDIA Quadro P5000 (16 GB)

Usually, without GPU support, I submit jobs simply using: mpirun pw.x -inp input.in > output.out. The output file in such cases includes these lines:

  Parallel version (MPI), running on    28 processors
      MPI processes distributed on     1 nodes
      K-points division:     npool     =       2
      R & G space division:  proc/nbgrp/npool/nimage =      14
      wavefunctions fft division:  Y-proc x Z-proc =       2      7
      wavefunctions fft division:  task group distribution
                                   #TG    x Z-proc =       2      7

However, I do not understand what additional parameters are to be included while running with GPU support to use it. I read about -ndiag, ntg, npool and other options on the internet but could not understand how to use them for my configuration. I am totally new to parallelization and would be really grateful if someone could give more insight into using these options to use GPU in my calculations fully. Thanks in advance.

Answer 1

Have you already tried the Pw_forum?

I do not use QE and so cannot directly answer your question. However, I'd say that this is one of those things where you are better off finding either a tutorial (in places such as YouTube) or someone you know with experience in compiling software for linux (just to make sure that your compilation is successful).

That said, the basic idea is that you should already have the GPU enabled version of QE installed (which you say you have already done). So read through the QE user guide sections 2.3 and 2.4 as mentioned in the comments by user @147875 and you might find a clue to the required flags when running the pw.x command.

A. If you check and find that the GPU enabled version is already installed. It should simply be a matter of either adding an additional line such as package gpu 1 to your input script, or even a command line flag which should be added to the command that you currently use to run QE. You can contact the Pw_forum regarding this.

B. In any case if you haven't got the GPU enabled version installed properly, the basic steps should be as follows. So try to figure out where you went wrong.

1. Install matching NVIDIA drivers and the CUDA toolkit: Typing nvidia-smi in a terminal will spit out details of the current CUDA version and NVIDIA driver version if they are both installed. If not, follow the guidelines given by NVIDIA.

2. Get the CUDA enabled version of QE and compile it with the appropriate flags: You should be able to find information regarding these in either the user manual, or this question on the NVIDIA forum, or this Tutorial posted in the QE forums.

3. Once that is done, you should have a CUDA enabled version of QE installed in your system. In this case, you are now back to what I mentioned in the step A earlier.

C. Alternatively, read through this GitHub link they seem to have a way to patch an existing version of QE to work with CUDA. However, I cannot confirm if it is successful or not.

Note: I provided this answer since nobody had answered your question and understand what it is like to be stuck in a situation like this. Keep in mind that even the most detailed tutorials and guidelines may not directly work for your system (which means that your compiled version of QE may not have GPU enabled). It all depends on the linux distribution and version you are running, the available compilers, the places where the compilers and the libraries are installed etc. So in that way, every computer can be considered as unique. But the Ask Ubuntu and Stack Overflow communities along with the forums of the software that you are compiling could prove to be really helpful.