JIT compiler for C, C++, and the likes

Question

Is there any just-in-time compiler out there for compiled languages, such as C and C++? (The first names that come to mind are Clang and LLVM! But I don't think they currently support it.)

Explanation:

I think the software could benefit from runtime profiling feedback and aggressively optimized recompilation of hotspots at runtime, even for compiled-to-machine languages like C and C++.

Profile-guided optimization does a similar job, but with the difference a JIT would be more flexible in different environments. In PGO you run your binary prior to releasing it. After you released it, it would use no environment/input feedbacks collected at runtime. So if the input pattern is changed, it is probe to performance penalty. But JIT works well even in that conditions.

However I think it is controversial wether the JIT compiling performance benefit outweights its own overhead.

Answer 1

From a practical viewpoint, there are only a couple of real implementations that currently exist: Microsoft's C++/CLI and (sort of) Clang/LLVM. As already mentioned, C++/CLI isn't really exactly C++. In theory, it's C++ with an almost-conforming set of extensions; practically, almost no real C++/CLI code is compatible with C++, so its being nearly conforming makes little real difference. Likewise, despite the name, LLVM isn't so much a virtual machine as a set of tools for creating virtual machines. As such, Clang/LLVM isn't so much an implementation as a toolkit that could be used to create such an implementation.

From a theoretical viewpoint, there's honestly not as much real controversy as it might look like. What's actually happened is pretty simple: Java advocates (it seems to be mostly Java advocates, anyway) spent years trying to come up with benchmarks that would show Java beating C and/or C++. In a few cases, they doctored benchmarks in pretty obvious ways to get the results they wanted, but they had to manipulate the benchmarks in pretty obvious ways to accomplish that. Since they couldn't get the results they wanted from real code, they started to follow an alternate route: writing about the possibility that a JIT could give performance benefits under at least some circumstances.

In doing this, however, they generally ignored one crucial point: the majority of optimization is independent of the target processor -- especially the most important (and, unfortunately, the most expensive) optimizations. This means JIT compilers have relatively little benefit compared to up-front compilers (even in theory), and they have a fairly high cost compared to up-front compilers, because the user has to wait while any expensive optimization happens. Because of that, most JIT compilers are pretty much forced to skip doing the most expensive optimizations, because in nearly every case they cost more than they (even hope to) gain. In the case of an up-front compiler, you can amortize the cost of that expensive optimization across many executions, but with a JIT compiler you have to repeat it at (essentially) every execution instead. As a result optimizations that are entirely practical for an up-front compiler become completely useless for a JIT compiler.

Answer 2

C++/CLI is jitted. Granted, C++/CLI is not C++ but it is pretty close. That said Microsoft's JIT doesn't do the super clever/cute kinds of runtime behavior based optimizations you're asking about, at least not to my knowledge. So this really doesn't help.

http://nestedvm.ibex.org/ turns MIPS into Java bytecode which would then be jitted. The problem with this approach from your question is that you throw away a lot of the useful information by the time it gets to the JIT.

Answer 3

Firstly, I assume you'd want a tracing jit rather than a method jit.

The best approach to take would be compiling the code to llvm IR, then adding in tracing code, before producing a native executable. Once a code block becomes sufficiently well used and once enough information about the values (not the types like in dynamic languages) of variables has been collected then the code can be recompiled (from the IR) with guards based on values of the variables.

I seem to remember there was some progress on making a c/c++ jit in clang under the name libclang.

Answer 4

Yes, there are JIT compilers for C++. From a pure performance perspective, I think Profile Guided Optimization (PGO) is still superior.

However, that does not mean that JIT compilation is not yet used in practice. For example, Apple uses LLVM as a JIT for their OpenGL pipeline. That is a domain where you have significantly more information at runtime, which can be used to remove a lot of dead code.

Another interesting application of JIT is Cling, an interactive C++ interpreter based on LLVM and Clang: http://root.cern.ch/drupal/content/what-cling

Here is a sample session:

[cling]$ #include <iostream>
[cling]$ std::cout << "Hallo, world!" << std::endl;
Hallo, world!
[cling]$ 3 + 5
(int const) 8
[cling]$ int x = 3; x++
(int) 3
(int const) 3
[cling]$ x
(int) 4

It is no toy project but it is actually used in CERN, for example, to develop the code for the Large Hadron Collider.

Answer 5

Maybe this is hard to grasp, but it depends on who you are.

If you are a compiler-writer or jit-writer, some user out there will write such a tight program, and they won't write it very intelligently, so that any slowness will be your fault, if you are the tool-writer. Most users will never notice.

However, if you are a language user and are writing code for your users, you should not assume you are one of the blessed few who will stress the compiler or jitter to its extreme. Rather you should just get the fat out of your code so your users can't blame you for slowness.

So many times I see programmers worrying, for example, about the optimization level of the compiler, jitter, or profile-guided optimizer trying to tune cycles out of a hotspot in, for example, a sort routine or matrix multiply when, if fact, they are not even spending time in that code, OR if they are in it, they don't need to be.