What keywords are important to speedup the code in C++?

Question

I'm porting to C++ and adding a lot of functionality to a numerical application written in Fortran 77. While I hate F77, I have to admit that the thing goes very fast. Now, I'm implementing practically the same structure and algorithm, but nevertheless the difference in the execution time is noticeable. Not big, but it's there. And I'm wondering, could it be that I need to pass extra information to the compiler? The only related information that I've been able to find is a recommendation to use the restrict key word which is supported by g++, but that's all. Hence, my question: Are there anymore keywords important to speedup code in C++ in numerical applications? or maybe compiler directives?

Thanks!

Answer 1

I do not believe that you should focus your optimizations looking for magic keywords, but here goes some tips:

1. "inline" might do wonders in numerical apps if you have many short numerical functions

2. Be careful when copying values all around when you could use a reference. This sounds basic but if you have many classes that are copy/assignment compatible your code could be hiding some copies. Make sure your constructors are defined with the "explicit" keyword, that way your compiler will show you where implicit copies are done.

3. Give C++11 a try. There are some interesting optimizations. There are more things that are solved statically. Also one very interesting thing for your case is the rvalue reference optimization. It is a very important optimization on the return value of functions. You will notice that a lot if you overload operators, which you will do if you operate different units and try to structure your old code.

4. There are lots of things that can be done with STL and really interesting template metaprograming, but that is a whole different level to what most people is used to.

5. Do not rely too much on your compiler optimizations. C++ syntax is well known to make life easier for low-level programmers and a nightmare for compiler designers. That is one of the reasons for which good static analyzers took so long to appear. I know that Scott Mayers made that comment and how C++11 improvements will help to get better tools, but cannot quote where he did.

6. On the line with compilers... do not expect to get SIMD and overall paralel optimizations. Compilers are really bad at it and unless you get your hands dirty with some library for it, your code will target only one core, one thread of your machine.

7. It might sound silly but the keyword "const" is really important. If you make clear that some value does not change, the compiler will apply a whole extra set of optimizations. Detecting if a variable could be considered const takes some extra parsing that most compilers do not perform not even with optimizations maxed out. Related with this for example, if your loops do not seem to have a clear/constant end, the compiler will not unroll them (or maybe not as good as it could)

However I wonder why you are migrating from Fortran77 to C++. Fortran sure is an old language, but it is the king for mathematics no matter how ugly it looks, the best way to deal with old code is building a good layer and operate it from outside with the new infrastructure and language you want. It is easier said than done, because sure you will have still to fix some Fortran code in order to make it easier to operate from outside.

If you decide to go on with C++ at least you will get much nicer libraries for multithreading (and that is one of the biggest improvements of C++11) where Fortran lags behind. Try to avoid CUDA if not absolutely necessary, it is really good and mature but focused on NVIDIA. I would keep an eye on OpenCL

Be aware that there is one huge difference for Math. In C++ (like most languages), the values of a matrix are stored as a set of rows while Fotran stores them as a set of columns. This makes a huge difference on the best memory usage of your algorithms. Hence if your algorithms are optimized for Fortran, their loops will target columns instead of rows. (look here) That is how seriously Fortran optimized for Mathematics goes.

One final concern. You want to use C++ and not Fotran77. Because you can get a better abstraction and design? Well, that is a good reason, but please, be wary of the market obsession with object oriented programming. It is a good thing, but not a silver bullet. A GUI for example should always be object oriented, but if you do lots of math and move huge amounts of memory, some aspect oriented style would be nice. Even better for your domain problem, have a look at data oriented design as an alternative to object oriented design (and see one case where OOP fails)

Answer 2

If you had to narrow it down to one keyword, it would probably be template. Unfortunately, applying that (at all well) will require some study. Applying it as well as possible (e.g., using expression templates) will require quite a lot of study.

For some ideas of how to write competitive numerical code in C++, you might want to look at some existing libraries such as blitz++ and Boost uBLAS.

Answer 3

Your code will hardly become faster magically by sprinkling a few keywords here and there. And simply copying algorithms from one language to another (if that's indeed what you are doing) is never going to help. Speed correlates with

1. a good design, and
2. selectively applied optimizations based on careful profiling.

As for design: The one thing that made C++ equal FORTRAN's performance was the invention of expression templates by Todd Veldhuizen for his blitz++ library.

Answer 4

The first thing you should do is profile, to see which specific piece of code is responsible for the slowdown.

As far as the keywords, inline might be a good one to use.

Answer 5

One thing to remember when porting from Fortran to C++ is the different array layout: Fortran, by default, uses column-major layout, C and C++ row-major. When your program relies heavily on linear algebra (like all Fortran programs seem to do), the different cache behaviour will make a major difference.

If that seems to be the problem (read: your profiler shows you are spending considerable time in linear algebra), try using a decent LA library like Eigen with matrices in column-major layout.

Answer 6

These are good answers, but it's quite possible both the Fortran and the C++ versions contain substantial room for speedup, so you wouldn't really expect them to be exactly the same speed now.

Assuming you want either one to be as fast as possible for a given machine and input load, there is a countable infinity of programs that do the job, and some program(s) that take less time than all the others. What you want to do is remove unnecessary activities until the program approaches that optimal shortness of time, regardless of language.

Here's how I do it, and here's an example of seriously aggressive performance tuning.

If I can just give examples, in C++ (using MFC) here is a case where something you wouldn't normally worry about, array indexing, turns out to cost 40% of the time and can be done better. In an example I did using stl vectors, the cost was even higher.

Similarly in Fortran, I was tuning some code that used the LAPACK library, which one would think would be about as finely optimized as possible. Chances are it is, for large matrices and a small number of calls. However, for small matrices and a large number of calls, guess what it was spending a heavy fraction of its time doing. It was calling a function to do string compares to see what the options were in calling the math functions. For example, DGEMM is a general routine to do matrix multiplication & scaling. It's first two arguments are CHARACTER*1 flags that tell it whether either input matrix is transposed. For small matrices, calling a function to test those flags is a major percent of the time, and could clearly be eliminated by writing a more specialized routine.

The only generality you can draw from these examples is that you need an effective diagnostic (like random-pausing) to tell what the problem is at each stage of performance tuning. The problems you find will undoubtedly be different from these examples, but in every case the only way you know what to fix is to have the program itself tell you.

Once you get rid of the extra stuff being done in the program that exists to make some unspecified general coding easier, then you can turn the compiler's optimizer loose to do its own level of cycle-squeezing (assuming speed is your major concern).

Answer 7

Most of it is going to be in the design of the rewrite. I've done F to C and F to C++ several times for commercial products and the biggest boogymen were:

• Beware of allocation operations in node-based standard library iterators.

• Knock out any pass by reference (pointers), remember that Fortran has the no-aliasing rule for pointers. C and C++ do not, so if you end up with a function that looks like this:

  int some_function(int* a, int* b)
  {
  
       ...code...
       int some_intermediate_value = *a + *b;
       int other_value = *a + 7;
       ...and other code referencing b and a...
  
  }
  

...it will fetch a and b every time (because of aliasing rules). Solvable by either changing the function signature and uses, or by copying to local temporaries inside the function (useful when doing a BIG refactor where it's tough to grab so many changes at once).