5
$\begingroup$

I wonder, if someone has run (runs) parallel Orca calculations on at least two nodes without a work load manager (scheduler)? I am asking because I have several interconnected servers (nodes) I'd like to run Orca on but there is no scheduler and I could not make Orca run on more than one server (node with several CPU-cores) so far.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ +1. Can you tell us what you tried so far? mpirun or mpiexec didn't work? $\endgroup$
    – Nike Dattani
    May 23 at 11:55
  • $\begingroup$ Hi Nike, thanks for your reply. Orca starts calculations with its own launcher, simply FULL_PATH_TO_ORCA input_file, without mpirun or mpiexec. $\endgroup$
    – EvGeniy
    May 23 at 13:35
  • 1
    $\begingroup$ Orca does not have it's own launcher, when you run full_path_to_orca, it reads the input file for the nprocs parameter (which tells it how many cores to be used), and then it calls mpirun. So the orca executable is simply a driver program that calls mpirun -np 8 orca_scf_mpi or other module executables. $\endgroup$
    – S R Maiti
    May 25 at 8:14

1 Answer 1

Reset to default
7
$\begingroup$

Not sure if ORCA can be run using a user-chosen MPI runner but there are a few steps that need to be done in order to run ORCA (or any other MPI program) on a distributed/interconnect cluster (assuming that you are using an Ubuntu server with OpenMPI already installed (check again that the version of your OpenMPI is supported by ORCA). You have root permission or are sudoer).

  1. You need to deal with a hosts file on the master node of your cluster. In this file, you can specify the name of the worker machines you would like to do the calculation, for example:
$ cat /etc/hosts

127.0.0.1       localhost
172.xx.xx.1     worker1
172.xx.xx.2     worker2
172.xx.xx.3     worker3
172.xx.xx.4     worker4

Note that, make sure that each worker's manager file has exactly the same content as the hosts file of the master node.

  1. Setup SSH for it

The next step would be setting up SSH because this is the way that all machines communicate with each other. This can be done simply as you can find elsewhere on the internet with SSH tutorial:

Install openssh-server and generate a new key

sudo apt­-get install openssh-server
ssh-keygen -t dsa

Add the newly generated key to each worker, e.g

ssh-copy-id worker1

Then enable a passwordless SSH connection

eval `ssh-agent`
ssh-add ~/.ssh/id_dsa
  1. Setup NFS in manager which the worker nodes mount to exchange data (You can skip this step if the NFS is already set up)

On NFS-server

# Install the required package
sudo apt-get install nfs-kernel-server

cd $HOME/
mkdir scratch

# Edit `/etc/exports` and add the following 
/home/YOUR_USER_NAME/scratch *(rw,sync,no_root_squash,no_subtree_check)

exportfs -a

sudo service nfs-kernel-server restart

You can easily find the meaning of each keyword on the internet

On NFS-worker (actually you do almost the same as NFS-server)

# Install required package
sudo apt-get install nfs-common

cd $HOME/
mkdir scratch

sudo mount -t nfs manager:/home/YOUR_USER_NAME/scratch ~/scratch

You can use df -h to check whether the directory is already mounted.

  1. Running an MPI program
mpirun -np 8 --hosts manager,worker1,worker2 FULL_PATH_TO_YOUR_ORCA

You should try running a simple MPI program, e.g. printing Hello world, to make sure the process above works. You can find more details on the internet, like this website: http://selkie.macalester.edu/csinparallel/modules/Patternlets/build/html/MessagePassing/RunningMPI.html.

FWIW, hope this helps

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hi, many thanks for your response. The point is, Orca has its own (parallel) launcher (one starts a calculation just simply with FULL_PATH_TO_YOUR_ORCA without mpirun or mpiexec), which takes all needed information from the scheduler environment, if I understand it correctly. At least, it works problem-free on clusters with scheduler (SLURM or whatever). One can feed the launcher with a file containing names of the workers (sort of the host file). Well, one also has to take care of the nodes being in the same directory. Maybe something else is needed ... $\endgroup$
    – EvGeniy
    May 23 at 13:34
  • $\begingroup$ @EvGeniy I see your point. Maybe you had better ask in ORCA forum about this. $\endgroup$
    – High Performance Rangsiman
    May 26 at 5:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .