5
$\begingroup$

I'm going to use a more powerful server to run my simulations on it using a remote control software (The server is in another building). It's my first time to try that.

What are the things that I should be aware of? Do I have to install the codes that I'm using on that server too? How can I use it, please?

$\endgroup$
13
  • 2
    $\begingroup$ The best way to get answers to your question is asking server administrators. Teamviewer is for graphical interface access and normally, you access the login terminal using SSH protocol and then submit your job using a script (designed for the job scheduler software) directly at the linux prompt. There are software like WebMO that can do the submission directly to the server from your local or even internet. $\endgroup$
    – Camps
    Mar 28, 2023 at 11:48
  • 1
    $\begingroup$ @Camps The server administrators don't know about computational modelling. I have to figure out how to do everything by myself. Can you please explain to me step by step what should I do in details. I also added to the post that the server is in another building. $\endgroup$
    – Camilla
    Mar 28, 2023 at 12:11
  • 2
    $\begingroup$ 1/2 @Camilla I think HPC facilities are usually not equipped with GUI. As far as I know, first, you have to access the command line using SSH (almost always). On your personal computer, you can directly run a program. But for a server, your program should be added to the queue using a workload manager (something like pbs or slurm). $\endgroup$ Mar 28, 2023 at 12:16
  • 1
    $\begingroup$ 2/2 If your server doesn't have the module available to load already, you can compile the program you intended to use. Maybe you can discuss it in this HPC chatroom. $\endgroup$ Mar 28, 2023 at 12:16
  • 1
    $\begingroup$ Related: mattermodeling.stackexchange.com/questions/6497/… $\endgroup$
    – Tyberius
    Mar 28, 2023 at 12:57

2 Answers 2

10
$\begingroup$

Almost all HPC servers use Linux. Therefore, in almost all cases, familiarity with Linux is crucial, especially the basic commands (e.g. ls, cd, mkdir, cp, rm, etc.).

Connecting to the server

Connecting to a Linux-based HPC server is now always in my experience done through SSH (in the 1990s I used Telnet to log into the University of Toronto servers, but I can't recall using anything other than SSH in the mid-2000s or later). To log into an HPC server with SSH, you can run the following command in the default terminal in Linux, MacOS or Windows Subsystem for Linux, or in something like PuTTY in pure Windows (assuming your username is camilla and the server's name is materials.stackexchange.com, for a hypothetical example):

ssh [email protected]

As long as there's no problems, you will next be asked for your password, and upon entering it correctly you will be able to access the server. Usually you will be given a temporary password at the beginning and will be highly encouraged (or forced) to change it immediately. The passwd command will allow you to do this if you are not already forced to do it upon your first successful login. The spelling of that command is a bit strange, and it can be hard to remember whether or leave out the letter "a" or the letters "or", so I spelled it correctly for you in the hyperlinked words in this paragraph.

Transfer of files

It is very likely that you will have input scripts or data files on a computer that you're currently using, and will want those files to be accessible on the HPC server. A very simple way to transfer files to and from an HPC server is using scp (in the 1990s I used FTP, which stands for "file transfer protocol" but now exclusively use scp which is just a "secure" analog of the previously mentioned and very basic cp command). The following command is run on your local machine and is for copying files from your local machine to the above hypothetical server (the important part to remember is the colon symbol : which is easy to forget because we far more often use cp without it, than we use scp with it):

scp file.txt [email protected]:

From your local machine you can also copy files from the server to your current directory on your local machine, for example with the following command (the main difference from the above command is the order):

scp [email protected]:/path/to/file.txt .

Compiling/installing programs

Most HPC servers do not allow you to have full control of the server, so you are limited in how you can install things. For this reason, and because it's wasteful for 100 users to each install their own copy of a 10GB software, servers often come with most major programs pre-installed. These pre-installed programs are often made available with the lmod system. The following lmod commands are the most commonly used ones, and my example below is for Python but could be used for other programs by changing the last word in the each command accordingly:

module load python    # load the system's default version of Python
module unload python  # unload whatever version of Python was loaded
module spider python  # learn information about the various versions of Python modules that are made available to you

To see which modules you already have loaded, you can run:

module list

and to see which ones are available to load via module load, you can run:

module avail

When programs are not available via the lmod system, then your next option might be pip, but this can be significantly more complicated than using pip on your local machine, for example Compute Canada which serves almost every Canadian academic researcher has a "wheel house" for Python wheels, which is described here, and pip install has to actually be run every time you run a job (so it's part of the "job submission script") as described here: How can I build the wheels necessary for a quick installation of PySCFad on a "compute node"?. Likewise, compiling software the "traditional way" from "source" can be significantly more complicated than on a local machine as you may have to "load" compilers as modules (among other things), and you can see an example of the sequence of commands for this type of installation here: Is this "internal error" reproducible when compiling OpenMolcas on other Intel compilers?

Running calculations

Explanations of how to use SLURM, LoadLeveler, and SGE are provided in answers here: How can I submit jobs to an HPC scheduler?. This discussion is also related: Job scheduler alternatives. The tag here on MMSE is for asking questions about schedulers like the ones mentioned in this paragraph.

Logging out of the server

When you run the command:

logout

the terminal that you have been using all this time will return you to the directory on your local machine at which you were present at the time of logging into the server.

$\endgroup$
6
  • $\begingroup$ I found that in the server ubuntu 16.04 is the one installed there and I have ubuntu 20 in my laptop will that cause me problems $\endgroup$
    – Camilla
    Mar 29, 2023 at 12:37
  • $\begingroup$ When I tried load module python command it's not working, same problem for load module python3 $\endgroup$
    – Camilla
    Mar 29, 2023 at 12:39
  • $\begingroup$ Note that Windows 10 and 11 (the only currently supported/updated versions of Windows) have OpenSSH preinstalled. Installing PuTTY is rarely necessary or useful these days. $\endgroup$
    – zdimension
    Mar 29, 2023 at 12:41
  • 1
    $\begingroup$ Thank you so much Nike for the detailed explanation :) $\endgroup$
    – Camilla
    Mar 29, 2023 at 20:53
  • $\begingroup$ When I do exit or logout to logout from the server then close my laptop can that stop the calculation in the server? And how can I check the calculation after I come back? $\endgroup$
    – Camilla
    May 11, 2023 at 12:46
3
$\begingroup$

We need more detail to effectively help you to use your remote server well. However, a good starting point is the HPC Carpentry's "HPC Intro" course: https://carpentries-incubator.github.io/hpc-intro/.

You should run through this course on your own, and also send this course to the administrators of your remote server and ask them if (1) the course is a suitable introduction to using this particular server, and if not, (2) what they would like you to do differently to the course.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .