Matter Modeling Stack Exchange is a question and answer site for materials modelers and data scientists. It only takes a minute to sign up.
Sign up to join this community
I'm going to use a more powerful server to run my simulations on it using a remote control software (The server is in another building). It's my first time to try that.
What are the things that I should be aware of? Do I have to install the codes that I'm using on that server too? How can I use it, please?
Almost all HPC servers use Linux. Therefore, in almost all cases, familiarity with Linux is crucial, especially the basic commands (e.g. ls, cd, mkdir, cp, rm, etc.).
Connecting to a Linux-based HPC server is now always in my experience done through SSH (in the 1990s I used Telnet to log into the University of Toronto servers, but I can't recall using anything other than SSH in the mid-2000s or later). To log into an HPC server with SSH, you can run the following command in the default terminal in Linux, MacOS or Windows Subsystem for Linux, or in something like PuTTY in pure Windows (assuming your username is camilla and the server's name is materials.stackexchange.com, for a hypothetical example):
ssh [email protected]
As long as there's no problems, you will next be asked for your password, and upon entering it correctly you will be able to access the server. Usually you will be given a temporary password at the beginning and will be highly encouraged (or forced) to change it immediately. The passwd command will allow you to do this if you are not already forced to do it upon your first successful login. The spelling of that command is a bit strange, and it can be hard to remember whether or leave out the letter "a" or the letters "or", so I spelled it correctly for you in the hyperlinked words in this paragraph.
It is very likely that you will have input scripts or data files on a computer that you're currently using, and will want those files to be accessible on the HPC server. A very simple way to transfer files to and from an HPC server is using scp (in the 1990s I used FTP, which stands for "file transfer protocol" but now exclusively use scp which is just a "secure" analog of the previously mentioned and very basic cp command). The following command is run on your local machine and is for copying files from your local machine to the above hypothetical server (the important part to remember is the colon symbol : which is easy to forget because we far more often use cp without it, than we use scp with it):
scp file.txt [email protected]:
From your local machine you can also copy files from the server to your current directory on your local machine, for example with the following command (the main difference from the above command is the order):
scp [email protected]:/path/to/file.txt .
Most HPC servers do not allow you to have full control of the server, so you are limited in how you can install things. For this reason, and because it's wasteful for 100 users to each install their own copy of a 10GB software, servers often come with most major programs pre-installed. These pre-installed programs are often made available with the lmod system. The following lmod commands are the most commonly used ones, and my example below is for Python but could be used for other programs by changing the last word in the each command accordingly:
module load python # load the system's default version of Python
module unload python # unload whatever version of Python was loaded
module spider python # learn information about the various versions of Python modules that are made available to you
To see which modules you already have loaded, you can run:
module list
and to see which ones are available to load via module load, you can run:
module avail
When programs are not available via the lmod system, then your next option might be pip, but this can be significantly more complicated than using pip on your local machine, for example Compute Canada which serves almost every Canadian academic researcher has a "wheel house" for Python wheels, which is described here, and pip install has to actually be run every time you run a job (so it's part of the "job submission script") as described here: How can I build the wheels necessary for a quick installation of PySCFad on a "compute node"?. Likewise, compiling software the "traditional way" from "source" can be significantly more complicated than on a local machine as you may have to "load" compilers as modules (among other things), and you can see an example of the sequence of commands for this type of installation here: Is this "internal error" reproducible when compiling OpenMolcas on other Intel compilers?
Explanations of how to use SLURM, LoadLeveler, and SGE are provided in answers here: How can I submit jobs to an HPC scheduler?. This discussion is also related: Job scheduler alternatives. The job-scheduler tag here on MMSE is for asking questions about schedulers like the ones mentioned in this paragraph.
When you run the command:
logout
the terminal that you have been using all this time will return you to the directory on your local machine at which you were present at the time of logging into the server.
load module python command it's not working, same problem for load module python3
$\endgroup$
exit or logout to logout from the server then close my laptop can that stop the calculation in the server? And how can I check the calculation after I come back?
$\endgroup$
We need more detail to effectively help you to use your remote server well. However, a good starting point is the HPC Carpentry's "HPC Intro" course: https://carpentries-incubator.github.io/hpc-intro/.
You should run through this course on your own, and also send this course to the administrators of your remote server and ask them if (1) the course is a suitable introduction to using this particular server, and if not, (2) what they would like you to do differently to the course.
