Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Sometimes compilers inline function calls. That means that they move the code of the called function into the calling function. This makes things slightly faster because there's no need to push and pop stuff on and off the call stack.

So my question is, why don't compilers inline everything? I assume it would make the executable notably faster.

The only reason I can think of is a significantly larger executable, but does it really matter these days with hundreds of GB of memory? Isn't the improved performance worth it?

Is there any other reason why compilers don't just inline all function calls?

share|improve this question

closed as too broad by Otávio Décio, GlenH7, gnat, MichaelT, m3th0dman Aug 28 at 18:23

There are either too many possible answers, or good answers would be too long for this format. Please add details to narrow the answer set or to isolate an issue that can be answered in a few paragraphs.If this question can be reworded to fit the rules in the help center, please edit the question.

15  
IDK about you, but I don't have hundreds of GB of memory just lying around. –  Ampt Aug 28 at 13:36
4  
You are overly generic, does "compilers" mean "all compilers" and does "everything" mean really "everything"? Then then the answer is simple, there are situations where you simply can't inline. Recursion comes to mind. –  Otávio Décio Aug 28 at 13:38
16  
Cache locality is a way much more important than tiny function call overhead. –  SK-logic Aug 28 at 14:02
2  
Does performance improvement really matter these days with hundreds of GFLOPS of procesing power? –  mouviciel Aug 28 at 15:04
2  

6 Answers 6

up vote 16 down vote accepted

First note that one major effect of inline is that it allows further optimizations to be made at the call site.

For your question: there are things which are difficult or even impossible to inline:

  • dynamically linked libraries

  • dynamically determined functions (dynamic dispatch, called through function pointers)

  • recursive functions (tail recursion can)

  • functions for which you don't have the code (but link time optimization allow this for some of them)

Then inlining has not only beneficial effects:

  • bigger executable means more disk place and bigger load time

  • bigger executable means increase of cache pressure (note that inlining small enough functions such as simple getters may decrease the executable size and the cache pressure)

And finally, for functions which takes a non trivial time to execute, the gain just doesn't worth the pain.

share|improve this answer
3  
some recursive calls can be inlined (tail calls), but all can be transformed into iteration if you optionally add a explicit stack –  ratchet freak Aug 28 at 13:51
    
@ratchetfreak, you can also transform some non tail recursive call in to tail one. But that's for me in the realm of the "difficult" one (especially when you have co-recursive functions or have to determine dynamically where to jump to simulate the return), but that's not impossible (you just put in place a continuation framework and considering that present it becomes easier). –  AProgrammer Aug 28 at 14:07

A major limitation is runtime polymorphism. If there is a dynamic dispatch happening when you write foo.bar() then it is impossible to inline the method call. This explains why compilers don't inline everything.

Recursive calls cannot be easily inlined either.

Cross module inlining is also difficult to perform for technical reasons (incremental recompilation would be impossible for ex)

However, compilers do inline a lot of things.

share|improve this answer
3  
Inlining through a virtual dispatch is very difficult, but not impossible. Some C++ compilers are able to do it under certain circumstances. –  bstamour Aug 28 at 16:28
2  
... as well as some JIT compilers (devirtualization). –  Frank Aug 28 at 16:36

First, you cannot always inline, e.g. recursive functions might not be always inlinable (but a program containing a recursive definition of fact with just a printing of fact(8) could be inlined).

Then, inlining is not always beneficial. If the compiler inlines so much that the result code is big enough to have its hot parts not fitting in e.g. the L1 instruction cache, it might be much slower than the non-inlined version (which would easily fit the L1 cache)... Also, recent processors are very fast at executing a CALL machine instruction (at least to a known location, i.e. a direct call, not a call thru pointer).

At last, full inlining requires a whole program analysis. This might not be possible (or is too costly). With C or C++ compiled by GCC (and also with Clang/LLVM) you need to enable link-time optimization (by compiling and linking with e.g. g++ -flto -O2) and that takes quite a lot of compilation time.

share|improve this answer
    
For the record, LLVM/Clang (and several other compilers) also supports link-time optimization. –  You Aug 28 at 16:27
    
I know that; LTO existed in the previous century (IIRC, in some MIPS proprietary compiler at least). –  Basile Starynkevitch Aug 28 at 16:31

Surprising though it may seem, inlining everything doesn't necessarily reduce execution time. The increased size of your code can make it difficult for the CPU to keep all your code in its cache at once. A cache miss on your code becomes more likely and a cache miss is expensive. This is made far worse if your potentially inlined functions are large.

I've had noticeable performance improvements from time to time by taking large chunks of code marked as 'inline' out of header files, put them into the source code, so the code is only in one place rather than at every call site. Then the CPU cache is used better and you also get better compile time...

share|improve this answer
    
this seems to merely repeat points made and explained in a prior answer that was posted an hour ago –  gnat Aug 28 at 16:50
    
What caches? L1? L2? L3? Which one is more important? –  Peter Mortensen Aug 28 at 17:39

Inlining everything would not mean just increased disk memory consumption but also increased internal memory consumption which is not that plentiful. Remember that code also relies in memory in code segment; if a function is called from 10000 places (say the ones from standard libraries in a fairly large project), then the code for that function occupies 10000 times more internal memory.

Another reason might be the JIT compilers; if everything is inline then there are not hot spots to be dynamically compiled.

share|improve this answer

One, there are simple examples where inlining everything will work out very badly. Consider this simple C code:

void f1 (void) { printf ("Hello, world\n"); }
void f2 (void) { f1 (); f1 (); f1 (); f1 (); }
void f3 (void) { f2 (); f2 (); f2 (); f2 (); }
...
void f99 (void) { f98 (); f98 (); f98 (); f98 (); }

Guess what inlining everything will do to you.

Next, you make the assumption that inlining will make things faster. That's the case sometimes, but not always. One reason is that code that fits into the instruction cache runs a lot faster. If I call a function from 10 places, I'll always run code that is in the instruction cache. If it is inlined, then the copies are all over the place and run a lot slower.

There are other problems: Inlining produces huge functions. Huge functions are a lot harder to optimise. I've got considerable gains in performance critical code by hiding functions into a separate file to prevent the compiler from inlining them. As a result, the generated code for these functions was much better when they were hidden.

BTW. I don't have "hundreds of GBs of memory". My works computer doesn't even have "hundreds of GBs of hard drive space". And if my application where "hundreds of GBs of memory", it would take 20 minutes just to load the application to memory.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.