Eventually, I figured out what was wrong happening with my DFT calculation. As I understand now, each processor saves and maintains (write recursively as the calculation progresses) its own file containing the wavefunctions (Kohn-sham orbitals), charge density, etc. This can be done in two different ways during the parallelization:
collected: all files from different processors are collected by the code to a single processor that writes them to disk, in a single file, using a format that doesn't depend upon the number of processors or their distribution. This is the default since v.6.2 for final data.
Distributed: each processor writes its own file to disk in its internal format to a different file. The 'distributed' format is fast and simple, but the data so produced is readable only by a job running on the same number of processors, with the same type of parallelization, as the job who wrote the data, and if all files are on a file system that is visible to all processors (i.e., you cannot use local scratch directories: there is presently no way to ensure that the distribution of processes across processors will follow the same pattern for different jobs).
What wrong I was doing?
When I ran the 'SCF' calculation with
npools=20 and k-point mesh of
12* 12 * 12. This is a wrong estimation for choosing the n-pools
number. As it is mentioned in QE user guide(page 17) that the npools should divide the number of k-points by an integer. The file saved in this calculation was from the first method.
Then, I started running the 'NSCF' calculation on a denser k-point grid of 20 * 20 * 20 with the same 20
npools again. In this case, each processor started writing its own files (method 2). It created
prefix.wfc1, prefix.wfc2, prefix.wfc3 ... prefix.wfc20, which took a lot of disk space.
Of course, the job will send an error when there is no disk space available to proceed the calculation with a termination.
Then, I tried to reduce the number of
npools from 20 to 2, which was the reason for the error I have got in my files attached to the question. If one is using the final result of last calculation to perform the second calculation then the number of pools on which next calculation is running should be the same otherwise one will get an error like pointed out by @Anoop A Nair in the comment.
How did I solve this issue?
I ran again both 'SCF' and 'NSCF' calculation with
npools = 2. This time both calculations were done within 20 hours time. Previously, only 'SCF' calculation was taking around 1 day to do the job. This time it also took less space on the disk.
Conclusion: One should do some experiments while choosing the optimal number of processors during the parallelization to make it resource-friendly. Also one can set
disk_io = none,
medium to reduce the size of files being saved after the calculation.