Why does VASP use -O2 optimization by default rather than -O3?

Question

When compiling VASP, I am wondering why the

 OFLAG = -O2

is suggested as default. For performance reason, I would use

 OFLAG = -O3

Could someone help explain the reason?

EDIT: I asked this question because I used the DFT code FHI-aims, whose compilation is suggested with optimization level:

FFLAGS = -O3 -ip -fp-model precise

In addition, I use intel compiler.

Answer 1

This is a great question! Only a VASP developer involved in making that decision would be able to tell you precisely why they made that decision, but I will give you as many useful pieces of information related to this topic as I can. Many of these are from a 10-year old thread in StackOverflow that currently has 144,000+ views: "Is optimisation level -O3 dangerous in g++?"

• Applying -O3 to an entire program that is as large as VASP, can cause it to run slower because it would generate more machine code than -O2 which would then make the program unable to fit in the L1 cache. For "large" programs this can also be true for -O2 vs -O1 or -O0. VASP is a "large" program, but I don't know if it's large enough for -O2 to cause this type of slow-down (hopefully the developers who decided to set -O2 to be the default, considered this carefully enough). The intended use of -O3 is that you manually apply it to a small number of files if they contain critical inner loops that actually benefit from these aggressive space-for-speed tradeoffs.

• -O3 has also been known to have many bugs in the past, but the most upvoted answer in the linked StackOverflow thread says that this was "over a decade ago" (and since that answer was written in July 2012, it would now be over "two decades ago" that this was a problem). It's possible that the tendency for -O3 to have bugs was a factor 20+ years ago in making the decision to make -O2 the default. However, while the top-voted answer in that thread says that the bugs were only a problem 20+ years before 2022, an answer in 2016 from a compiler developer, says that it was still a problem recently.

• -O3 generally will make compilation take longer (often by a very significant amount), and will make the size of the overall program's executables take up much more space. For a big program like VASP, space can be a concern, and even the time to compile can be a concern (a much smaller program called mrcc has sometimes taken me more than 24 hours to compile, which can be enough for your compilation to be interrupted/cut-off due to it being a "job" that takes too long on a shared supercomputer). The longer compile time and larger space required for -O3 in VASP is probably insignificant when compared to the amount of time that VASP is being run (i.e. the use that you're getting out of the compilation), but certainly for smaller programs like my FeynDyn software for numerical Feynman integrals for quantum dynamics, I chose a lot of default settings with the overall user experience in mind rather than only the speed and accuracy of the calculations. I would rather them be able to get started very quickly and without trouble (i.e. bugs in using -O3 and time/space increases causes by -O3 would not be preferred) than to have slightly faster calculations in a program that is already super fast for most calculations no matter what level of optimization is used. This bullet point applies to most scientific software in general rather than just to VASP.

• Optimization often requires care and attention rather than just blindly using the command with the largest number in it. In this thread where someone was complaining about a bubble sort program being slower with -O3 when compared to -O2, someone posted the following code timings:

-O2 -g: 1.07s
-O3 -g: 1.17s
-O3 -g -fprofile-generate/use: 0.84s