24
$\begingroup$

The codes for quantum mechanical calculations make heavy use of linear algebra, and it seems most of them delegate this task to time-tested and highly optimized libraries, instead of trying to deploy their own. There is a plethora of options. For example, when we install Gamess-US in Ubuntu GNU/Linux, several options are listed, both proprietary and free / open source software:

enter image description here

In the past I used ATLAS, and then MKL, as we are nudged towards it by the (very fast) qualifier given in the install screen. But then I was doing some research on it, and found this information on its wikipedia page:

Intel MKL and other programs generated by the Intel C++ Compiler improve performance with a technique called function multi-versioning: a function is compiled or written for many of the x86 instruction set extensions, and at run-time a "master function" uses the CPUID instruction to select a version most appropriate for the current CPU. However, as long as the master function detects a non-Intel CPU, it almost always chooses the most basic (and slowest) function to use, regardless of what instruction sets the CPU claims to support. This has netted the system a nickname of "cripple AMD" routine since 2009.

This got me a bit worried, as I'm trying to do a Gamess-US install in a machine with a Ryzen processor, and so I'm afraid I will get a crippled install if I use intel MKL. Given the number of remaining choices, without a clear idea of relative performance between them, I got some analysis paralysis. The obvious alternative would be one of AMD, but in absence of a substantial speed advantage, I could as well use a open source one, as I try to favor free software whenever I can.

If someone more experienced could give some advice on this problem, I would be grateful. How does the several linear algebra libraries compare to each other, on the workloads typical of materials modeling? Is there a big difference from one to other, or they are all optimized to such degree that doesn't matter much which one is used. Is there one with good overall performance over a variety of hardware, not biased against alternative hardware like the MKL seems to be?

Improve this question
$\endgroup$
7
  • 7
    $\begingroup$ Seems more appropriate for ComputationalScience.SE cause your question focouses on computational aspects instead of primarily materials modeling. $\endgroup$
    – Mithridates the Great
    Commented May 27, 2020 at 20:03
  • 3
    $\begingroup$ Neither the programming language under the hood is primarily materials modeling, yet that was one of the most upvoted questions here ever. See What is a good programming language to learn for materials modeling?. Also that is one of the first things people need to worry about modeling, because if they can't even install the software, what else can they do? Pen and paper modeling? $\endgroup$
    – ksousa
    Commented May 27, 2020 at 21:24
  • 4
    $\begingroup$ I think the question is relevant (I even upvoted it), but on a second thought, off-topic here. It's a math library question that affects computing broadly (differently from the programming language question, which asked specifically "for MM"), not materials science or gamess in particular. $\endgroup$
    – stafusa
    Commented May 28, 2020 at 0:14
  • 2
    $\begingroup$ FWIW, there's a workaround posted in Reddit for "cripple AMD"; and, according to this article it works very well and might lead to a performance (slightly?) superior to that of open source libraries. $\endgroup$
    – stafusa
    Commented May 28, 2020 at 0:19
  • 3
    $\begingroup$ OpenBlas is one of the best alternatives, take a look at some comparisons in different contexts here, here (related: here), here and here (relevant to AMD hardware). $\endgroup$
    – schneiderfelipe
    Commented May 28, 2020 at 0:35

3 Answers 3

Reset to default
17
$\begingroup$

OpenBLAS is a free, open-source BLAS library that has fast support for even recent processors. (It is based on the earlier, famous GotoBLAS library which became obsolete years ago.) OpenBLAS is also multi-platform: in addition to x86 and x86_64 it also supports other architectures like ARM and PowerPC. OpenBLAS also has runtime CPU detection; if you compile it in, the resulting library supports all processors and picks the best kernel at runtime.

IIRC OpenBLAS is as fast (sometimes has been even faster!) as MKL on some Intel processors; I don't think I've seen benchmarks on AMD hardware. But, the nice thing about OpenBLAS is that it's free, so it typically comes built-in your linux distribution. E.g. OpenBLAS has been available on Fedora and Red Hat Enterprise for a few years now (courtesy of yours truly), and to use it you just need to install the package

yum install openblas-devel

and then link to the flavor you want: on Fedora/Red Hat the available variants are

  • libopenblas sequential library with 4-byte integers
  • libopenblaso OpenMP parallel library with 4-byte integers
  • libopenblasp pthreads parallel library with 4-byte integers
  • libopenblas64 sequential library with 8-byte integers
  • libopenblaso64 OpenMP parallel library with 8-byte integers
  • libopenblasp64 pthreads parallel library with 8-byte integers
Improve this answer
$\endgroup$
9
  • 3
    $\begingroup$ Wow! You made OpenBLAS available on Fedora and Red Hat? $\endgroup$
    – Nike Dattani - No Free Time
    Commented May 28, 2020 at 14:23
  • 9
    $\begingroup$ @NikeDattani yes, I've been a Fedora packager for over a decade... I've also packaged e.g. PyQuante, Psi4, PySCF, OpenMolcas, libint, libxc, etc. All of these come as system packages :) $\endgroup$
    – Susi Lehtola
    Commented May 28, 2020 at 15:43
  • 5
    $\begingroup$ That is great @SusiLehtola. I think a considerable entry barrier for more people start using computational methods is the sheer amount of effort you need to install some of the tools. I think many people in the field learn unix just as they start their modeling activities, and it's a lot of things you need to learn at the same time, it can be a daunting task. It's awesome when we can just get it running out of the box with only a single-line command to the package manager. $\endgroup$
    – ksousa
    Commented May 28, 2020 at 16:28
  • 2
    $\begingroup$ @NikeDattani that sounds like a problem that has a solution ;) $\endgroup$
    – Susi Lehtola
    Commented May 28, 2020 at 18:39
  • 3
    $\begingroup$ @NikeDattani CentOS is nothing but Red Hat Enterprise Linux, which is already covered by the Fedora spectrum. $\endgroup$
    – Susi Lehtola
    Commented May 29, 2020 at 9:15
7
$\begingroup$

Let me add 2 notes

  1. MKL has a secret env-var to force the use of AVX2 extensions etc. For details, see here

    The magic is setting the environment variable:

    MKL_DEBUG_CPU_TYPE=5

    note however that Intel can at any release remove this flag, so one is not guaranteed this will work in later releases.

  2. Another great library is blis it basically works like OpenBLAS but with a different mind-set, so to speak. Really great package as well!

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ As I explained in my answer, BLIS is actually in use inside AOCL, which is the official AMD published math package for AMD processors. $\endgroup$
    – wyphan
    Commented Dec 1, 2020 at 17:37
  • 1
    $\begingroup$ Great link to Puget Systems, those guys really know their stuff. It would be helpful to include a (cited) excerpt of the particularly relevant parts of that blog post here in your answer, though; when that blog post page (or the entire Puget website) goes down/gets deleted, that part of your answer will lose its value. However, if you quote the relevant bits, then the answer loses no value in such an event. $\endgroup$
    – TylerH
    Commented Dec 1, 2020 at 20:18
6
$\begingroup$

AMD's answer to Intel's MKL is called AMD Optimizing CPU Libraries (AOCL), which is coincidentally based on BLIS. In addition, it also includes FLAME (for LAPACK functionality), FFTW, and others.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Do you have any benchmark data for its performance, for example on Rome, with respect to OpenBLAS and MKL (with the appropriate hand-holding to use AVX)? $\endgroup$
    – Phil Hasnip
    Commented Feb 19, 2021 at 1:35
  • $\begingroup$ @PhilHasnip the blis development page has graphs benchmarking against different implementations, and it beats quite a bit of them. Note however, that the benchmarks tries to use "baseline" implementation, without using weird flags to improve performance, see here $\endgroup$
    – nickpapior
    Commented Apr 7, 2021 at 11:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .