Understanding QuantumEspresso Toolbox through Maestro

Question

Anyone here uses Quantum Espresso through Maestro can tell me a bit more about the calculation panel setting:

I understand that -npools option must be divisible by the number of processors (threads). But I don't really understand the -ntg option. I am running some geometry optimization with $4 \times 4 \times 1$ kpoints but certain setting cause the calculation to fail. The job will get submitted, but I would get back an error like:

 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

     stopping ...

 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
     Error in routine divide_et_impera (1):
     some nodes have no k-points
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Does -ntg depend on the number of kpoints you use? And if you can also tell me how much more the simultaneous jobs options speed up the calculation then that would be greatly appreciated! I have tried to increase the simultaneous jobs but it doesn't seems to really speed up the calculation... maybe it has to do with how I specified other paramters (-npool, -ntg, etc. )

I am trying to find the best way to speed-up the calculation, but so far it has been trials and errors. I would like to understand this setting more, and know how to speed-up my calculation.

Answer 1

From the MAESTRO help system we got that the calculation is run using the driver periodic_dft_driver.py:

usage: $SCHRODINGER/run periodic_dft_gui_dir/periodic_dft_driver.py
       [-h] -cfg_file CFG_FILE [-TPP THREADS] [-nimage NIMAGE]
       [-npools NPOOLS] [-ntg NTG | -nband NBAND] [-qargs QARGS]
       [-HOST <hostname>]
       input_file

Driver that runs Quantum ESPRESSO workflows. Copyright Schrodinger, LLC. All
rights reserved.

positional arguments:
  input_file          Input structures.

optional arguments:
  -h, -help           Show this help message and exit.
  -cfg_file CFG_FILE  Path to the config file. (default: None)
  -TPP THREADS        Number of threads to use for each job. (default: 1)
  -qargs QARGS        Set arguments to pass to the queue manager. (default:
                      None)

Quantum ESPRESSO parallel options:
  -nimage NIMAGE      Number of images (points in the configuration space)
                      (default: 1)
  -npools NPOOLS      Number of K-point groups to run in parallel (default: 1)
  -ntg NTG            Number of task groups to run in parallel (default: None)
  -nband NBAND        Number of bands to run in parallel (default: None)

Job Control Options:
  -HOST <hostname>    Run job remotely on the indicated host entry. (default:
                      localhost)

Look for slides 10-21 for parallel options in this document.

About k-points and $n_{pools}$:

k-point parallelization

If the simulation consists in different k-points, those can be distributed among $n_{pools}$ pools of CPUs.

K-points are typically independents: the amount of communications is small

When there is a large number of k-points this layer can strongly enhance the scalability

By definition, $n_{pools}$ must be a divisor of the total number of k-points:

mpirun –np 64 pw.x –npool 4 –input inputfile.inp

About task-group parallelization and $ntg$

Each plane-wave group of processors is split into $n_{task}$ task groups of $n_{FFT}$ processors, with $n_{task} \times n_{FFT}= n_{PW}$;

each task group takes care of the FFT over $N_e/n_t$ states.

Used to extend scalability of FFT parallelization.

Example for 1024 processors –divided into $n_{pool}= 4$ pools of $n_{PW} = 256$ processors, –divided into $n_{task} = 8$ tasks of $n_{FFT} = 32$ processors each; –Subspace diagonalization performed on a subgroup of $n_{diag} = 144$ processors:

mpirun –np 1024 pw.x –npool 4 –ntg 8 –ndiag 144 –input inputfile.inp

About speeding up your calculations, the last 4 slides are about the QE scalability and this link can help.