Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Does the "magic" of the JVM hinder the influence a programmer has over micro-optimisations in Java? I recently read in C++ sometimes the ordering of the data members can provide optimizations (granted, in the microsecond environment) and I presumed a programmer's hands are tied when it comes to squeezing performance from Java?

I appreciate a decent algorithm provides greater speed-gains, but once you have the correct algorithm is Java harder to tweak due to the JVM control?

If not, could people give examples of what tricks you can use in Java (besides simple compiler flags).

share|improve this question
12  
The basic principle behind all Java optimization is this: The JVM has probably already done it better than you can. Optimization mostly involves following sensible programming practices and avoiding the usual things like concatenating strings in a loop. –  Robert Harvey Nov 20 '12 at 23:40
3  
The principle of micro-optimization in all languages is that the compiler already done it better than you can. The other principle of micro-optimization in all languages is that throwing more hardware on it is cheaper than programmer's time micro-optimizing it. Programmer has to tend to scaling problems (suboptimal algorithms), but micro-optimization is a waste of time. Sometimes micro-optimization makes sense on embedded systems where you can't throw more hardware on it, but Android using Java, and a rather poor implementation of it, shows most of them have enough hardware already. –  Jan Hudec Nov 21 '12 at 9:54
1  
for "Java performance tricks", worth studying are: Effective Java, Angelika Langer Links - Java Performance and performance related articles by Brian Goetz in Java theory and practice and Threading Lightly series listed here –  gnat Nov 21 '12 at 10:06
1  
Be extremely careful about tips and tricks - the JVM, operating systems and hardware moves on - you're best off learning the performance tuning methodology and applying enhancements for your particular environment :-) –  Martijn Verburg Nov 21 '12 at 11:10
    
In some cases, a VM can make optimizations at run time that are impractical to make at compile-time. Using managed memory can improve performance, though it will also often have a higher memory footprint. Unused memory is freed when convenient, rather than ASAP. –  Brian Dec 6 '12 at 17:13
show 1 more comment

7 Answers 7

up vote 5 down vote accepted

Sure, at the micro-optimization level the JVM will do some things that you'll have little control over compared to C and C++ especially.

On the other hand, the variety of compiler behaviors with C and C++ especially will have a far greater negative impact on your ability to do micro-optimizations in any sort of vaguely portable way (even across compiler revisions).

It depends on what sort of project you're tweaking, what environments you're targeting and so on. And in the end, it doesn't really matter since you're getting a few orders of magnitude better results from algorithmic/data structure/program design optimizations anyways.

share|improve this answer
    
It can matter a lot when you find your app doesn't scale across cores –  James Nov 21 '12 at 0:02
    
@james - care to elaborate? –  Telastyn Nov 21 '12 at 0:07
1  
See here for a start: mechanical-sympathy.blogspot.co.uk/2011/07/false-sharing.html –  James Nov 21 '12 at 0:22
1  
@James, scaling across cores has very little to do with the implementation language(Python excepted!), and, more to do with application architecture. –  James Anderson Nov 21 '12 at 3:24
add comment

Micro-optimizations are almost never worth the time, and almost all the easy ones are done automatically by compilers and runtimes.

There is, however, one important area of optimization where C++ and Java are fundamentally different, and that is bulk memory access. C++ has manual memory management, which means you can optimize the application's data layout and access patterns to make full use of caches. This is quite hard, somewhat specific to the hardware you're running on (so performance gains may disappear on different hardware), but if done right, it can lead to absolutely breathtaking performance. Of course you pay for it with the potential for all kinds of horrible bugs.

With a garbage collected language like Java, this kind of optimizations cannot be done in the code. Some can be done by the runtime (automatically or through configuration, see below), and some are just not possible (the price you pay for being protected from memory management bugs).

If not, could people give examples of what tricks you can use in Java (besides simple compiler flags).

Compiler flags are irrelevant in Java because the Java compiler does almost no optimization; the runtime does.

And indeed Java runtimes have a multitude of parameters that can be tweaked, especially concerning the garbage collector. There's nothing "simple" about those options - the defaults are good for most applications, and getting better performance requires you to understand exactly what the options do and how your application behaves.

share|improve this answer
1  
+1: basically what I was writing in my answer, maybe better formulation. –  Klaim Nov 21 '12 at 10:05
1  
+1: Very good points, explained in a very concise way: "This is quite hard ... but if done right, it can lead to absolutely breathtaking performance. Of course you pay for it with the potential for all kinds of horrible bugs." –  Giorgio Dec 6 '12 at 17:09
1  
@MartinBa: It's more that you pay for optimizing memory management. If you don't try to optimize memory management, C++ memory management isn't that difficult (avoid it entirely via STL or make it relatively easy using RAII). Of course, implementing RAII in C++ takes more lines of code than doing nothing in Java (i.e., because Java handles it for you). –  Brian Dec 6 '12 at 18:59
3  
@Martin Ba: Basically yes. Dangling pointers, buffer overflows, uninitialized pointers, errors in pointer arithmetic, all things that simply don't exist without manual memory management. And optimizing memory access pretty much requires you to do a lot of manual memory management. –  Michael Borgwardt Dec 6 '12 at 22:40
1  
There is a couple of things you can do in java. One is object pooling, which maximizes the chances memory locality of objects (unlike C++ where it can guarantee memory locality). –  U Mad Dec 7 '12 at 15:16
show 5 more comments

Java (as far as I'm aware) gives you no control over variable locations in memory so you have a harder time to avoid things like false-sharing and alignment of variables (you can pad out a class with several unused members). Another thing I don't think you can take advantage of are instructions such as mmpause, but these things are CPU specific and so if you figure you need it Java may not the be language to use.

There exists the Unsafe class that gives you flexibility of C/C++ but also with the danger of C/C++.

It might help you to look at the assembly code the JVM generates for your code

To read about a Java app that looks at this kind of detail see the Disruptor code released by LMAX

share|improve this answer
add comment

This question is very hard to answer, because it depends on language implementations.

In general there's very little room for such "micro optimizations" these days. The main reason is that compilers take advantage of such optimizations during compilation. For example there's no performance difference between pre-increment and post-increment operators in situations where their semantics are identical. Another example would be for example a loop like this for(int i=0; i<vec.size(); i++) where one could argue that instead of calling the size() member function during each iteration it would be better to get the size of the vector before the loop and then comparing against that single variable and thus avoiding function a call per iteration. However, there are cases in which a compiler will detect this silly case and cache the result. However, this is only possible when the function has no side-effects and the compiler can be sure that the vector size remains constant during the loop so it merely applies to fairly trivial cases.

share|improve this answer
    
As for the second case, I don't think compiler can optimize it in the foreseeable future. Detecting that it's safe to optimize vec.size() depends on proving that the size if the vector/lost does not change inside the loop, which I believe is undecidable due to the halting problem. –  Lie Ryan Nov 21 '12 at 7:00
    
@LieRyan I've seen multple(simple) cases in which compiler has generated exactly identical binary file if the result has been manually "cached" and if size() has been called. I wrote some code and it turns out the behavior is highly dependant on the way the program operates. There are cases in which the compiler can guarantee that there's no possibility for vector size to change during the loop, and then there are cases in which it can't guarantee it, very much alike to the halting problem as you mentioned. For now I'm unable to verify my claim(C++ disassembly is a pain) so I edited the answer –  zxcdw Nov 21 '12 at 9:33
2  
@Lie Ryan: a lot of things that are undecidable in the general case are perfectly decidable for specific but common cases, and that's really all you need here. –  Michael Borgwardt Nov 21 '12 at 10:12
    
@LieRyan If you only call const methods on this vector, I'm pretty sure many optimizing compilers will figure it out. –  K.Steff Dec 6 '12 at 17:15
    
in C#, and I think I read in Java also, if you don't cache size the compiler knows it can remove the checks to see if you are going outside the array bounds, and if you do cache size it has to do the checks, which generally cost more than you are saving by caching. Trying to outsmart optimizers is rarely a good plan. –  Kate Gregory Dec 7 '12 at 13:57
add comment

There's a middle area between micro-optimization, on the one hand, and good choice of algorithm, on the other.

It is the area of constant-factor speedups, and it can yield orders of magnitude.
The way it does so is by lopping off entire fractions of the execution time, like first 30%, then 20% of what's left, then 50% of that, and so on for several iterations, until there's hardly anything left.

You don't see this in little demo-style programs. Where you see it is in big serious programs with lots of class data structures, where the call stack is typically many layers deep. A good way to find the speedup opportunities is by examining random-time samples of the program's state.

Generally the speedups consist of things like:

  • minimizing calls to new by pooling and re-using old objects,

  • recognizing things being done that are sort of in there for generality's sake, rather than actually being necessary,

  • revising the data structure by using different collection classes that have the same big-O behavior but take advantage of the accessing patterns actually used,

  • saving data that's been acquired by function calls instead of re-calling the function, (It is a natural and amusing tendency of programmers to assume that functions having shorter names execute faster.)

  • tolerating a certain amount of inconsistency between redundant data structures, as opposed to trying to keep them entirely consistent with notification events,

  • etc. etc.

But of course none of these things should be done without first being shown to be problems by taking samples.

share|improve this answer
add comment

could people give examples of what tricks you can use in Java (besides simple compiler flags).

Other than improvements of algorithms, be sure to consider the memory hierarchy and how the processor makes use of it. There are big benefits in reducing memory access latencies, once you understand how the language in question allocates memory to its data types and objects.

Java example to access an array of 1000x1000 ints

Consider the below sample code - it accesses the same area of memory (a 1000x1000 array of ints), but in a different order. On my mac mini (Core i7, 2.7 GHz) the output is as follows, showing that traversing the array by rows more than doubles the performance (average over 100 rounds each).

Processing columns by rows*** took 4 ms (avg)
Processing rows by columns*** took 10 ms (avg) 

This is because the array is stored such that consecutive columns (i.e. int values) are placed adjacent in memory, whereas consecutive rows are not. For the processor to actually use the data, it has to be transferred to its caches. The transfer of memory is by a block of bytes, called a cache line - loading a cache line directly from memory introduces latencies and thus decrease a program's performance.

For the Core i7 (sandy bridge) a cache line holds 64 bytes, thus each memory access retrieves 64 bytes. Because the first test accesses memory in a predictable sequence, the processor will pre-fetch data before it is actually consumed by the program. Overall, this results in less latency on memory accesses and thus improves the performance.

Code of sample:

  package test;

  import java.lang.*;

  public class PerfTest {
    public static void main(String[] args) {
      int[][] numbers = new int[1000][1000];
      long startTime;
      long stopTime;
      long elapsedAvg;
      int tries;
      int maxTries = 100;

      // process columns by rows 
      System.out.print("Processing columns by rows");
      for(tries = 0, elapsedAvg = 0; tries < maxTries; tries++) {
       startTime = System.currentTimeMillis();
       for(int r = 0; r < 1000; r++) {
         for(int c = 0; c < 1000; c++) {
           int v = numbers[r][c]; 
         }
       }
       stopTime = System.currentTimeMillis();
       elapsedAvg += ((stopTime - startTime) - elapsedAvg) / (tries + 1);
      }

      System.out.format("*** took %d ms (avg)\n", elapsedAvg);     

      // process rows by columns
      System.out.print("Processing rows by columns");
      for(tries = 0, elapsedAvg = 0; tries < maxTries; tries++) {
       startTime = System.currentTimeMillis();
       for(int c = 0; c < 1000; c++) {
         for(int r = 0; r < 1000; r++) {
           int v = numbers[r][c]; 
         }
       }
       stopTime = System.currentTimeMillis();
       elapsedAvg += ((stopTime - startTime) - elapsedAvg) / (tries + 1);
      }

      System.out.format("*** took %d ms (avg)\n", elapsedAvg);     
    }
  }
share|improve this answer
add comment

The JVM can and often does interfere, and the JIT compiler can change significantly between versions Some micro-optimisations are impossible in Java due to language limitations, such as being hyper-threading friendly or the latest Intel processors' SIMD collection.

A highly informative blog on the topic from one the Disruptor authors is recommended reading:

One always has to ask why bother using Java if you want micro-optimisations, there are many alternative methods for acceleration of a function such as using JNA or JNI to pass onto a native library.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.