Problem with using VASP with ase on a GPU

I am a newbie using VASP, currently I am trying to calculate the formation energy per atom and pre bandgap using DFT with VASP on a GPU machine.

I wrote the following script for calculating the bandgap:

# -*- coding: utf-8 -*-
from ase.calculators.vasp import Vasp
from ase.optimize import BFGS
import numpy as np
import os

os.environ['VASP_PP_PATH'] = '/scratch'

# Load the structure from a CIF file

# Set up VASP calculator for relaxation
calc_relax = Vasp(xc='PBE',  # Exchange-correlation functional
encut=400,  # Plane-wave cutoff
kpts=(3, 3, 3),  # k-point grid for relaxation
ibrion=2,  # Ion relaxation: CG algorithm
nsw=50,  # Number of steps for ionic relaxation
ismear=0,  # Gaussian smearing
sigma=0.05,  # Width of smearing in eV
lreal='Auto',  # Use real-space projection for GPU acceleration
command='mpirun -np 1 vasp_gpu')  # VASP run command

# Attach the calculator to the structure for relaxation
structure.set_calculator(calc_relax)

# Optimize the structure
opt = BFGS(structure)
opt.run(fmax=0.01)

# Now setting up the NSCF calculation for band structure
calc_nscf = Vasp(xc='PBE',  # Exchange-correlation functional
encut=400,  # Plane-wave cutoff
kpts=(11, 11, 11),  # denser k-point mesh for band structure
ismear=0,  # Gaussian smearing (must use ISMEAR < -1 or 0)
sigma=0.05,
ibrion=-1,  # No ionic relaxation
nsw=0,  # No ionic steps
icharg=11,  # Non-self-consistent field run
command='mpirun -np 1 vasp_gpu')

# Attach the non-self-consistent calculator
structure.set_calculator(calc_nscf)

# Calculate the electronic structure (no need to run optimize)
structure.calc.calculate(structure)

# Extract eigenvalues
eigenvalues = structure.calc.get_eigenvalues(kpt=0, spin=0)  # For the first k-point and spin

# Calculate bandgap
occupied = np.max(eigenvalues[eigenvalues < 0])
unoccupied = np.min(eigenvalues[eigenvalues > 0])
bandgap = unoccupied - occupied

print('Bandgap (PBE):', bandgap, 'eV')


I still have not figured out how to calculate the formation energy. This is because the above code is not utilising the GPU properly. I have 2 gpus and 20 cpus in my system.

I have tried to increase the value here mpirun -np 1 vasp_gpu to more than 1 but it is just showing that my system is not capable. How do I make this calculation take advantage of the GPU machine?

Error

--------------------------------------------------------------------------
There are not enough slots available in the system to satisfy the 2
slots that were requested by the application:

vasp_gpu

Either request fewer slots for your application, or make more slots
available for use.

A "slot" is the Open MPI term for an allocatable unit where we can
launch a process.  The number of slots available are defined by the
environment in which Open MPI processes are run:

1. Hostfile, via "slots=N" clauses (N defaults to number of
processor cores if not provided)
2. The --host command line parameter, via a ":N" suffix on the
hostname (N defaults to 1 if not provided)
3. Resource manager (e.g., SLURM, PBS/Torque, LSF, etc.)
4. If none of a hostfile, the --host command line parameter, or an
RM is present, Open MPI defaults to the number of processor cores

In all the above cases, if you want Open MPI to default to the number
of hardware threads instead of the number of processor cores, use the

• Good question. It would help if you shared (a) does your VASP utilize 1 or more GPUs if called from the command line or bash script on the GPU machine, but without ase and python? (b) what your ase + VASP does and shows and outputs when you say "it's just showing that the system is not capable". Commented Apr 24 at 18:45
• @harsh so from your edit I take it that VASP runs fine if using mpirun -np 1 for one GPU? Because that does change the original question. Reminding you also of the second part of my question - does the error persist if python and ase are not used, and VASP is just called from command line or bash script? Commented Apr 25 at 9:22