21
$\begingroup$

Does anyone have any knowledge or direction for creating a homemade Beowolf cluster for materials modeling?I would like to be able to run my own "hobby" simulations at home, so a low energy, low capital investment is what I seek.

Raspberry Pi and ODroid use small, cheap, low energy single board computers (SBC) that can be stacked into a cluster. A demonstration cluster was made in 2013 in the UK Raspberry Pi demonstration cluster. I have not found examples of molecular simulation being applied to these clusters (GROMACS or GAMESS type applications). My concern is that they may not be able to handle long term simulations?

Are there any recent examples of these SBC clusters handling prolonged workload, and in particular, are there examples of molecular simulations being successful? I have tried reaching out to the SBC community, however, molecular dynamics and quantum chemistry gets me blank stares.

I just found this paper comparing the ODroid-MC1 to supercomputer cluster CPU's. The Odroid-MC1 is a 32 cpu cluster for \$220, and the CPU's compared against retail at \$9000+. The test problem was solving Lattice-Boltzmann flow computations. They found that ODroid-MC1 was only 4 times slower, and this was largely due to its use of 32-bit ARMv7. It can therefore likely be expected that using 64-bit ARMv8 and newer would make ODroid-MC1 very competitive, and incredibly cheap. ODroid consistently beats raspberry pi for speed.

Improve this question
$\endgroup$
16
  • 6
    $\begingroup$ +1. It's interesting. There is no HPC Stack Exchange. I tried inquiring about one in the past, but was immediately shot down. The HPC tag on this site seems to have had a bit of success, even though some of the questions were more about HPC than about the Materials Modeling (for example the question about alternative schedulers to SLURM, and the question about linear algebra libraries for non-Intel CPUs). Let's see what happens. If you want to do calculations for real research though (not just some fun hobby), I don't think anything homemade will compete with the big national clusters. $\endgroup$
    – Nike Dattani - No Free Time
    Commented May 29, 2020 at 18:15
  • $\begingroup$ Agreed on the competitive status. I am an algorithms guy though, I just need to simulate ~6000 atoms and I can prove if my idea worked or not. While I use supercomputers to run hundreds of jobs trivially in parallel, that is only for production. For testing, I only need to simulate a couple at a time, so even a couple desktops would do the trick really. Gromacs can do 4 ns of a 6000 atom system in a couple hours. $\endgroup$
    – B. Kelly
    Commented May 29, 2020 at 18:18
  • 1
    $\begingroup$ I think this is an interesting question and something I have thought from a hobbyist perspective. People once thought that using graphics cards to do calculations was not worth it either (I remember a story of early pioneers in that field using hardware found in playstations!) $\endgroup$
    – Cody Aldaz
    Commented May 29, 2020 at 19:08
  • 1
    $\begingroup$ Would it be too much to say which country this is? Many supercomputing centres allow individuals to apply for time on their machines (you would not have to pay any money). Many of these centers do not even require you to be living in the same country as the supercomputing centre. The idea of making a "cluster" with Raspberry Pi or something else at home, is interesting, but practically I would not recommend it for doing any algorithm development or calculations. $\endgroup$
    – Nike Dattani - No Free Time
    Commented May 29, 2020 at 19:09
  • 1
    $\begingroup$ Why not look around on the cloud market? There are quite a few providers who offer nodes eminently suitable for HPC calculations, together with fast interconnects etc. It would be more expensive in the long run if you had 100% utilisation at home, but if not, the cloud could turn out cheaper. $\endgroup$
    – albapa
    Commented Sep 25, 2020 at 15:36

3 Answers 3

Reset to default
22
$\begingroup$

Raspberry Pi clusters are okay for studying networked systems and job schedulers, but bad for any real calculations. There are several problems: there's very little memory per CPU, the interconnect is slow, having local disk is hard... but worst of all, the bang per buck is very low, see e.g. a Phoronix benchmark. So, in summary: Intel/AMD is still cheaper for running actual calculations, but a Raspberry Pi cluster is quite cheap to set up so it could serve as a toy / test system.

More powerful ARM workstations that are actually designed for running calculations (many CPUs, large memory, local storage) could be a game changer, but these are still not common.

Edit: I've just benchmarked a Raspberry Pi 4. According to linpack, it's about the same speed as a cheap celeron laptop; however, it's terribly slow in desktop use. Both are at least 10 times less powerful than my work laptop; I'd estimate that the raspberry pis are at least 3-4 times more expensive per gflop than good PCs at the moment.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Yeah, it seems I have to wait a few years. I will probably go for a desktop instead. $\endgroup$
    – B. Kelly
    Commented May 30, 2020 at 8:42
  • 1
    $\begingroup$ @CharlieCrown: Seems unlikely to pan out. If the micro-architectures get cheaper to string together in a huge network, then the folks at Intel/AMD/etc. will just start doing that instead. CPU manufacturers are going in that direction (e.g., AMD's Infinity Fabric) as well as research into RAM that basically does computations in conjunction with its memory operations (computational RAM). $\endgroup$
    – Nat
    Commented May 30, 2020 at 8:55
  • 1
    $\begingroup$ @CharlieCrown: That said, you probably want to look for "compute modules" rather than stand-alone units. Sometimes they're meant to go into existing PC's as co-processors (analogous to using a GPU as a co-processor) and sometimes they're basically just stripped-down computers (Raspberry Pi seems to have versions like this). $\endgroup$
    – Nat
    Commented May 30, 2020 at 9:00
  • $\begingroup$ @Nat thanks, I will take a look for compute modules. I am open to anything! $\endgroup$
    – B. Kelly
    Commented May 30, 2020 at 18:24
  • $\begingroup$ @SusiLehtola: of course, wrong button. $\endgroup$
    – albapa
    Commented Sep 25, 2020 at 15:36
11
$\begingroup$

@Susi already provided an excellent answer. I just want to add that for most applications in materials modeling, there is a relatively high overhead for parallelization. Basically, breaking up a problem into smaller and smaller pieces means that you are spending more and more time on communication between the nodes, etc.

Basically, you usually still want individual cores that are pretty fast. If you want to build a cheap hobby cluster yourself, the best option may be to try to get used hardware. A old desktop-class CPU can still be quite useful. Older server-grade stuff could also be really cheap since businesses might essentially be throwing them away.

You could also try to use diskless nodes. Finally, the nodes don't need to have full cases, nor do you need proper CPU racks. I know people who have essentially put a bunch of motherboards on cheap wire racks and hooked them all up.

Improve this answer
$\endgroup$
1
  • 5
    $\begingroup$ To note it, the bunch-of-motherboards-hooked-together thing is sometimes called a blade server. $\endgroup$
    – Nat
    Commented Jun 7, 2020 at 2:30
6
$\begingroup$

I would like to be able to run my own "hobby" simulations at home, so a low energy, low capital investment is what I seek.

Disclaimer. I have not made a Raspberry Pi cluster.

A few months ago (10/24/2021) the Turing Pi 2 board was released.

For a YouTube video (12/01/2021) on how to build a computer with 4 Raspberry Pi compute modules see this Jeff Geerling video.

4 Pis on a mini ITX board! The Turing Pi 2

If anyone is wondering if Raspberry Pi is considered a valid way to go then consider this.

Raspberry Pi supercomputer: Los Alamos to use 10,000 tiny boards to test software

Something to keep an eye on are these:

Uptime Lab's CM4 Blade adds NVMe, TPM 2.0 to Raspberry Pi

As noted on the web site

A 1U rackmount enclosure is in the works, and 16 of these boards would deliver:

64 ARM CPU cores
up to 128 GB of RAM
16 TB+ of NVMe SSD storage

Jeff has a video on these

Uptime Lab's CM4 Blade adds NVMe, TPM 2.0 to Raspberry Pi

but AFAIK these blades are not yet for sale. (12/09/2021) :-(

Improve this answer
$\endgroup$
2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .