Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I realize that this question is similar to Low level programming - what's in it for me, but the answers didn't really address my question well. Part from just an understanding, how exactly does your low level knowledge translate into faster and better programs?

There's the obvious lack of stop-the-world from garbage collection, but what else is an advantage? Do you really outperform your optimizing compiler? Do you pack your data structures in as tight as possible and be concerned about alignment? There's extra freedom naturally, but does that really translate into a faster program?

share|improve this question

closed as not constructive by gnat, FinnNk, jmquigley, Walter, Caleb Apr 1 '12 at 6:43

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

"Do you really outperform your optimizing compiler?" What does this mean? Are you talking about assembly? –  Pubby Mar 31 '12 at 2:30
@Pubby I believe he is referring to writing assembly by hand. A good compile can most likely optimize the resulting binaries better than you can write assembly by hand. –  Glenn Nelson Mar 31 '12 at 17:59
What kind of low level knowledge matters for what? Many, perhaps most, programmers could go their entire careers without needing to write a single line of assembly code. OTOH, the good folks who design processors might not consider assembly code to be "low level" at all. The question is meaningless without much more context. –  Caleb Apr 1 '12 at 6:51
it is generally trivial to outperform even the best optimizing compiler with hand written asm. Not millions of instructions worth, no, but take any particular program you have written and look at some of the compiler output and you can and will see places for improvement. Dont be fooled into thinking that compilers are awesome at optimizing now, they are not they have reached a plateau and are not getting that much better, some are getting worse each release. –  dwelch May 5 '12 at 3:40
having low level knowledge allows you to KNOW these things not just assume that the optimizer is working and how well. It also helps to validate other assumptions about your coding style or about what someone says is faster or not. You also may be fooling yourself trying to hand tune some high level code only to find the compiler is removing it all together or all your tweaking results in the same instruction sequences. –  dwelch May 5 '12 at 3:42

5 Answers 5

It's not just about faster. Extra CPU cycles from various CPU pipeline stall penalties and extra memory accesses due to cache misses and such, in sufficiently repetitive performance critical code, can add up to take measurable extra energy. A big cost in data centers these days is energy and cooling costs, and a big cost in mobile devices is the battery weight. The transistors have become nearly free, but the nanoWatts have not.

An optimizing compiler can help, but it might not know all the possible algorithmic transformations that still can produce a useful result. (e.g. humans sometimes know how to cheat and get away with it.) The short vector/SIMD math units in many CPUs these days (even on mobile device processors) can often only be fully utilized with some low-level knowledge about them.

Embedded and signal processing programmers use this low-level knowledge all the time, sometimes just to meet spec. If you can pack your code to run in a 10 cent cheaper processor, and end up shipping millions per annum embedded in some toy, you more than paid for a software engineer's salary or three.

share|improve this answer

You're right that some low-level knowledge doesn't matter. For example, I think you'd be nuts to routinely re-jigger arithmetic to replace multiplication and division with equivalent bit-shift operations to save a cycle or two. The speed-up is likely to be microscopic, the compiler might do it anyway, and I suspect doing things like triangle_area = (base >> 1) * height is just asking for trouble when someone looks at your code in six months. Similarly, a lot older code jumps through incredible hoops to reduce the number of floating point operations; that's also not worth the hassle, mental overhead, and potential for bugs these days.

On the other hand, there are other low-level concepts that can give you relatively large gains for minimal work. For example, consider caching. For example, you could iterate over an NxN matrix like this:

for(i=0; i<N; ++i)
   for(j=0; j<N; ++j)

or like this:

for(j=0; j<N; ++j)
  for(i=0; i<N; ++i)

If the matrix is suitably large, one of these (depending on how the language lays out the array) will cause a cache-miss for every single array access, while the other will be optimal. Assuming you've got to iterate over the matrix somehow, why not prefer the potentially faster one? They're equally readable, equally writable, etc.

In general, I think appreciating the things going on "under the hood" can help inform your decisions when programming in higher-level languages (e.g., Can I code this in a way that doesn't trash the cache over and over again?). If you're unfamiliar with the relationship between an array index and a memory address, you might not appreciate how important it is to check array indices and buffer sizes. I think you could glean a lot of this information from the higher-level language's documentation, but it'll definitely be more in-your-face in C.

Obviously, the trick is not to go overboard. There is a theoretical speed difference between i++ and ++i, but if copying a single four-byte variable makes or breaks your web-app, the app is already in pretty dire straights.

share|improve this answer

Understanding how memory behaves is IMO among the most fruitful things. Particularly:

  • What CPU cache is and why it matters.
  • Why primitives, and particularly arrays of primitives, are faster than equivalent (arrays of) objects.
  • Why in multidimensional arrays rows and columns aren't equivalent at all (alignment).
  • Memory barriers, locking vs. CAS operations.

Also you can help the optimizing compiler: for example, by making simple for loop through an array with explicit start and end indices may allow the compiler to optimize away bound checking (which is about the only thing that makes Java arrays slower than native arrays).

As with all optimization, it only matters where it does :)

share|improve this answer
Primitives vs objects is only a meaningful distinction in some languages. –  DeadMG Mar 31 '12 at 7:27
@DeanMG: True - if a language doesn't have both primitives and objects available, then it's not meaningful to discuss one vs. other in that language. But it's important to understand that, regardless of the programming language, the underlying hardware is always working on bits, bytes and words, not on objects, which are an useful, but sometimes costly, layer of indirection provided by a programming language. –  Joonas Pulakka Mar 31 '12 at 11:07
Unless you're using something like C++, in which there is no performance distinction at all, because objects don't have to be pointers. All I'm saying is that primitives, objects, they're all the same thing to the CPU, and only the language defines a difference- if at all. What you really mean is "arrays of values are faster than arrays of pointers-to-value", which is meaningful at all levels. –  DeadMG Mar 31 '12 at 16:10

Performance isn't the only benefit to understanding low-level details in your program. For example, if an application I write crashes, I often get one or more of these things:

  • a crash log containing a backtrace in the form of addresses on the call stack, and the value of the program counter and other registers
  • a core dump, containing the state of the process's virtual memory at the time of the crash.

In order to work out what's going on there, you need some knowledge of how the memory layout of the CPU architecture works, the calling conventions of the platform your code is running on and often the ability to read a disassembly of the platform code that appears in the stack trace.

share|improve this answer
  • IO (file or any other output device)
  • Networking (IO, but important enough that it deserves its own bullet)
  • Memory hierarchies (understand why it's bad to go from cacheA -> cacheB -> main mem -> disk)

Of course all of the low level stuff is valuable at some level, but these three will pay pretty good dividends to an application-level programmer to understand. Edit: these also are places where low-level techniques are frequently applied.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.