8
$\begingroup$

I had run an SCF calculation on a crystal system in Quantum ESPRESSO and had collected the converged charge density. However, during the NSCF calculation, there is the generation of .wfc files which are quite large in size to save in my system. My system is also interrupting the job due to the saturation of space available.

So, I am wondering how I can handle these files, and do the density of states calculation? Do I really need these files? Can I use the disk_io='none', option to prevent .wfc files from being saved on my system?

Edit: I have uploaded the scf.in,scf.out,nscf.in,nscf.out files here

Improve this question
$\endgroup$
10
  • $\begingroup$ +1. How big are the .wfc files? You might be printing to the node where you're running the calculation (node-local storage) rather than on the system's /scratch space which is usually much larger. For pw.x you could also adjust wf_collect to save space. An SCF calculation iteratively improves a wavefunction, meaning that the computer does need to know some description of the wavefunction in order to do it, whether this is stored in RAM or disk (but disk is larger than RAM so if you can do the calculation, it should be able to fit on disk). Check also your wfcdir, maybe give us input. $\endgroup$
    – Nike Dattani - No Free Time
    Commented May 9, 2021 at 20:45
  • $\begingroup$ @Nike Dattani, .wfc files are more than 50GB, and it's getting increase as the calculation proceeds further. Can I use disk_io and 'none'? $\endgroup$
    – UJM
    Commented May 9, 2021 at 20:57
  • $\begingroup$ Are these .wfc files gonna disappear at the end of calculations by themselves? I just want to plot the DOS, so I don't want these files to be saved in my system. I have already saved the converged charge density from scf calculations. $\endgroup$
    – UJM
    Commented May 9, 2021 at 21:00
  • 2
    $\begingroup$ @UjjawalM. Hi :). Its good that youve found a solution!. I believe the parameter npools is a part of your slurm script? rather than a pw.x keyword. With the problem resolved now; would you be able to write an answer to remove the question from the unanswered queue? $\endgroup$
    – Anoop A Nair
    Commented May 16, 2021 at 19:26
  • 1
    $\begingroup$ @AnoopANair Thanks for encouraging the user to remove the question from the unanswered queue :) Ujjawal: Thanks for taking our feedback! $\endgroup$
    – Nike Dattani - No Free Time
    Commented May 17, 2021 at 17:49

1 Answer 1

Reset to default
6
$\begingroup$

Eventually, I figured out what was wrong happening with my DFT calculation. As I understand now, each processor saves and maintains (write recursively as the calculation progresses) its own file containing the wavefunctions (Kohn-sham orbitals), charge density, etc. This can be done in two different ways during the parallelization:

  • collected: all files from different processors are collected by the code to a single processor that writes them to disk, in a single file, using a format that doesn't depend upon the number of processors or their distribution. This is the default since v.6.2 for final data.

  • Distributed: each processor writes its own file to disk in its internal format to a different file. The 'distributed' format is fast and simple, but the data so produced is readable only by a job running on the same number of processors, with the same type of parallelization, as the job who wrote the data, and if all files are on a file system that is visible to all processors (i.e., you cannot use local scratch directories: there is presently no way to ensure that the distribution of processes across processors will follow the same pattern for different jobs).

What wrong I was doing?

  • When I ran the 'SCF' calculation with npools=20 and k-point mesh of 12* 12 * 12. This is a wrong estimation for choosing the n-pools number. As it is mentioned in QE user guide(page 17) that the npools should divide the number of k-points by an integer. The file saved in this calculation was from the first method.

  • Then, I started running the 'NSCF' calculation on a denser k-point grid of 20 * 20 * 20 with the same 20 npools again. In this case, each processor started writing its own files (method 2). It created prefix.wfc1, prefix.wfc2, prefix.wfc3 ... prefix.wfc20, which took a lot of disk space. Of course, the job will send an error when there is no disk space available to proceed the calculation with a termination.

  • Then, I tried to reduce the number of npools from 20 to 2, which was the reason for the error I have got in my files attached to the question. If one is using the final result of last calculation to perform the second calculation then the number of pools on which next calculation is running should be the same otherwise one will get an error like pointed out by @Anoop A Nair in the comment.

How did I solve this issue?

I ran again both 'SCF' and 'NSCF' calculation with npools = 2. This time both calculations were done within 20 hours time. Previously, only 'SCF' calculation was taking around 1 day to do the job. This time it also took less space on the disk.

Conclusion: One should do some experiments while choosing the optimal number of processors during the parallelization to make it resource-friendly. Also one can set disk_io = none,low or medium to reduce the size of files being saved after the calculation.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .