Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Many questions and answers on the C/C++ pages, specifically or indirectly discuss micro performance issues (such is the overhead of an indirect vs direct vs inline function), or using an O(N2) vs O(NlogN) algorithm on a 100 item list.

I always code with no concern about micro performance, and little concern about macro performance, focusing on easy to maintain, reliable code, unless or until I know I have a problem.

My question is why is it that a large number of programmers care so much? Is it really an issue for most developers, have I just been lucky enough to not to have to worry too much about it, or am I a bad programmer?

share|improve this question

migrated from stackoverflow.com May 11 '11 at 3:56

This question came from our site for professional and enthusiast programmers.

4  
+1, good general question. –  iammilind May 11 '11 at 3:21
2  
I head two great quotes 1) "Premature Optimisation is the root of all evil." 2) 80% of your time will be spent of 20% of your code (80/20 rule). –  James Khoury May 11 '11 at 7:00
4  
@James the full quote from Donald Knuth is "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil". There'll be some good answers about the remaining 3% in this thread. –  StuperUser May 11 '11 at 12:49

16 Answers 16

up vote 13 down vote accepted

In practice, performance is seldom an issue that needs to be managed at that level of detail. It's worth keeping an eye on the situation if you know you're going to be storing and manipulating huge amounts of data, but otherwise, you're right, and better off, keeping things simple.

One of the easiest traps to fall into -- especially in C and C++ where you have such fine-grained control -- is optimizing too early, and at too fine a level. In general the rule is: A) don't optimize until you find out you have a problem, and B) don't optimize anything that you haven't proven to be a problem area by using a profiler.

A corollary to B) is: programmers are notoriously bad at predicting where their performance bottlenecks are, even though, to a one, they think they're good at it. Use a profiler, and optimize the parts that are slow, or change algorithms if one section of code is being called way too many times, so that it's causing a problem.

share|improve this answer
5  
Another one: initialization code that executes once doesn't generally need optimization, so look elsewhere. –  Mike DeSimone May 11 '11 at 3:27
3  
Depends on how often "once" is. When running ./configure, I would venture to say up to 75% of the run time might be spent on "initialization" code in the programs the script runs. 25-50% might even be spent on dynamic linking. –  R.. May 11 '11 at 3:47
9  
Rule A is a terrible rule. The architecture of a system plays a role in performance, and if you find out later your architecture simply can't support your performance requirements you're basically screwed. So while you be able to pass over fine details, completely ignoring this at the beginning is just plain wrong. –  edA-qa mort-ora-y May 11 '11 at 5:40
2  
@edA-qa : I used to think so, but over the years have experianced many more projects stuggle or fail before any performance considerations became a concern. Every time I have had performance concerns, the fix has been a comparitively low cost, days or a few weeks, No more of a concern than any other "bug" detected an d fix in development. However, jsut like any otehr risk item, performance coneres need to be identified and mitigated early in the project. –  mattnz May 11 '11 at 8:22

I think everything on your list is micro-optimization, which should not generally looked at, except for

using an O(n*n) vs O(NlogN) algorithm on a 100 item list

which I think should be looked at. Sure, that list is 100 items right now, and everything is fast for small n, but I'd be willing to bet soon that same code is going to be reused for a several million line list, and the code is still going to have to work reasonably.

Choosing the right algorithm is never a micro-optimization. You never know what kinds of data that same code are going to be used for two months or two years later. Unlike the "micro-optimizations" which are easy to apply with the guidance of a profiler, algorithm changes often require significant redesign to make effective use of the new algorithms. (E.g. some algorithms require that the input data be sorted already, which might force you to modify significant portions of your applications to ensure the data stays sorted)

share|improve this answer
27  
+1 for "Choosing the right algorithm is never a micro-optimization." –  0verbose May 11 '11 at 3:41
7  
I +1'd too, but note that choosing the big-O-optimal algorithm when your data sizes are sure to be small can be detrimental to development time, program size, and perhaps even memory usage. If you're sorting poker hands, do you really want to write a quicksort, smoothsort, or mergesort? I would start with simple insertion sort or use a sorting network. –  R.. May 11 '11 at 3:52
2  
@R. Implementation complexity is part of the choice of the right algorithm. However, most good algorithms already have published implementations freely available -- good programmers never write what they can steal. Finally, in the sorting case specifically I really hope programmers aren't implementing this themselves given that pretty much every programming language on the face of the earth comes with built-in sorting algorithms (even C! which comes with almost nothing else). –  Billy ONeal May 11 '11 at 3:55
7  
That's funny. In a thread about micro-optimization, a lot of commentators micro-optimize the answers. ;) –  Secure May 11 '11 at 7:57
5  
"I'd be willing to bet soon that same code is going to be reused for a several million line list": That completely depends on the problem domain. Examples: if you're writing a chess algorithm, you can be reasonably sure the board size won't change. If you program an autonomous vehicle, the number of wheels won't grow that fast, either. –  nikie May 11 '11 at 10:49

A looooooong time ago, in my first job, I wrote code for embedded systems. These systems used 8086 microprocessors, and had limited memory. We used the Intel C compiler. One system I built needed to access a 3-d array of structures. I built it just like the book told me: call malloc for the 3 dimensions, then allocate rows for the next dimension, then calloc for the end nodes.

It was pretty complicated (for me at the time), I had to do curve fitting, ANOVA process control and Chi-squared analysis. There were no libraries that did this for us; we had to write it all and fit it all onto the 8086.

The system ran like a dog. After a quick profiling, I discovered that one of the biggest problems was the allocator. To fix the problem I abandoned all the calls to malloc and did my own memory management of one large block of memory.


In another case on the same job, the customer was complaining about response time on their statistical process control system. The team before me had designed "software PLC" system where operators could use a boolean logic for combining signals and tripping switches. They wrote it in a simplified language, what we'd call a "domain specific language" today. as I recall it looked like ((A1 + B1) > 4) AND (C1 > C2) and so on.

The original design parsed and interpreted that string every time it was evaluated. On our measly processor, this consumed lots of time, and it meant that the process controller couldn't update as fast as the process was running.

I took a new look at it and decided that I could translate that logic into assembly code, at runtime. I parsed it once and then each time it ran, the app called into a dynamically generated function. Kind of like some viruses do today, I guess (but I don;t really know). The result was a 100-fold increase in performance, which made the customer and my boss really really happy.

The new code was not nearly as maintainable, being that I had built a custom compiler. But the performance advantage well outweighed the maintenance disadvantage.


More recently I was working on a system that needed to parse an XML fly, dynamically. Larger files would take considerably more time. This was very performance sensitive; too slow of a parse would cause the UI to become completely unusable.

These kinds of things come up all the time.


So.... sometimes you want maintainable, easy-to-write code. Sometimes you want code that runs quickly. The tradeoff is the engineering decision you need to make, on each project.

share|improve this answer
9  
In all your examples the cost of optimizing it afterwards wasn't much higher than writing the fast code from the beginning. So writing slower simpler code first and then optimizing where necessary worked well in all of them. –  CodesInChaos May 11 '11 at 11:24
3  
@CodeInChaos: The answer doesn't claim otherwise. It speaks to the OP's question "Why I should care about micro performance and efficiency?" Pre-optimization issues were merely inferred by the other answerers. –  webbiedave May 11 '11 at 15:42

If you are processing large images and iterating over every pixel, then performance tweaking can be critical.

share|improve this answer
2  
+1 -- also, high frequency finance, any kind of audio/video encoder/decoder, simulations and modeling (e.g. games), systemwide bits like CPU schedulers and memory managers, etc. –  Billy ONeal May 11 '11 at 3:45
2  
CAN be critical, but IS only critical after you've proved it to be so and you've profiled it to be where you think the problem is. (Hint: it probably isn't there.) –  JUST MY correct OPINION May 11 '11 at 5:37
1  
@JUST MY correct OPINION: actually, for image processing, data processing is usually the second largest time consumer (I/O still is the biggest). However, optimizing for I/O requires a lot of unusual/crazy designs and their acceptance by fellow programmers, and sometimes it is outright impossible to improve. The processing part, however, is usually embarassingly parallelizable, hence they are easily reapable benefits. (One's tweaking might be seen by another as a straight textbook implementation... unless you reach the level of VirtualDub) –  rwong May 11 '11 at 6:24

Let me tell you a bit about the why behind the culture.

If you're closer to 40 than to 20, and you've been programming for a living through your adult years, then you came of age when C++ was really the only game in town, desktop apps were the norm, and hardware was still greatly lagging software in terms of bandwidth/performance capabilities.

  • We used to have to do stupid programming tricks to be able to read large (>2G) files...
  • We used to worry about executable size...
  • We used to worry about how much memory our programs were consuming...
  • We regularly made algorithmic time vs. space trade-off decisions...
  • Even on the back-end, we had to write CGI programs in C or C++ for anything to handle a decent no. of RPS... It was several orders of magnitude faster.
  • We used to run tests on the merits of performance between delphi/c++/vb!

Very few people have to worry about these things today.

However, 10 years ago you still had to worry about your software being downloaded over a 56kb modem, and being run on a 5 year old PC... Do you remember how crappy PCs were in 1996? Think in terms of 4GB of hard drive, a 200Mhz processor, and 128Mb of RAM...

And the servers of 10 years ago? Dell's "next generation" server cost $2000, and came with 2 (!) 1Ghz pentium processors, 2Gb or Ram, and a 20Gb hard drive.

It was simply a different ballgame, and all of those "senior" engineers that have 10 years of experience (the guys likely to be answering your questions), cut their teeth in that environment.

share|improve this answer
1  
The additional 20 years of experience also means we've got the burn marks from having been through the optimization process many, many times and avoid doing things that may need it later. Same reason I don't smack my thumb (much) while using a hammer. –  Blrfl May 11 '11 at 13:40

there's already 10 answers here and some are really good, but because this is a personal pet peeve of mine...

premature optimization which a) takes way more time to do than a simple solution b) introduces more code where simple solution would've been half the size and half the complexity and c) makes things less readable is ABSOLUTELY should be avoided. However, if a developer has a choice between using a std::map or std::vector and he chooses the wrong collection out of pure ignorance for performance that is as bad if not worse than premature optimization. What if you could slightly change your code today, maintain readability, keep same complexity, but make it more efficient, would you do it? Or would you call it "premature optimization"? I find that a lot people wouldn't even give that any thought one way or another.

Once I was the guy who advised "micro-optimization" that required very little change and I was given the same response that you just said, "you shouldn't optimize too early. Let's just get it to work and we'll change it later if there is a performance problem". It took several releases before we fixed it. And yes it was a performance problem.

While early optimization may not be good, I think it is very beneficial if people write code with understanding what that code is going to do and don't simply disregard any question that results in O(x) notation as being "optimization". There's plenty of code you can write now and with a little thought about performance avoid 80% of issues down the road.

Also consider that a lot of performance problems are not going to happen in your environment and not right away. Some times you'll have a customer that pushes the limit or another developer decides to build on top of your framework and increase the number of objects 10-fold. With some though about performance now, you could avoid very costly redesign later. And if the problem is found after the software is officially released, even a simple fix becomes 20 times more expensive to apply.

So in conclusion, keeping performance in mind at all times helps develop good habits. Which are just as important to have as writing clean, as simple as possible and organized code.

share|improve this answer

I suspect that a lot of what you're seeing is simple sampling error. When people are dealing with straightforward situations, they write code and that's the end of things. They ask questions when they're dealing with something relatively tricky, such as needing to optimize, especially in a situation where it's not necessarily obvious that optimization would be needed.

That said, there's undoubtedly some premature optimization involved as well. Correctly or otherwise, C and C++ have a reputation for performance, which tends to attract people who care about performance -- including those who may do optimization as much for enjoyment as because it's really needed.

share|improve this answer
2  
I sure see a damn lot of premature-optimization questions on here... I think it comes from the fact that a lot of hobbyist programmers get started with the idea of writing games, and there's a huge corpus of nonsense "optimization" books and websites related to game development that put bad ideas in beginners' heads. :-) –  R.. May 11 '11 at 3:45
3  
When you're dealing with something tricky, often it seems easier to take a break from the tricky problem and fritter away your time worrying about whether you should use i++ or ++i –  Carson63000 May 11 '11 at 4:02

A couple of the other answers mention embedded systems, and I'd like to expand on this.

There are plenty of devices containing low-end processors, for example: the boiler controller in your house, or a simple pocket calculator, or the dozens of chips inside a modern car.

To save money, these may have quantities of flash (to store code) and RAM which seem tiny to those who've only written code for PCs or smartphones. To save power, they may run at relatively low clock rates.

To take an example, the STM32 family of microcontrollers goes from 24 MHz, 16 KB flash and 4 KB of RAM, up to 120 MHz, 1 MB flash and 128 KB RAM.

When writing code for chips like these, it saves a lot of time if you aim to make your code as efficient as possible as a matter of course. Obviously, premature optimisation remains a bad idea; but with practise, you learn how common problems can be solved quickly and/or with minimal resources, and code accordingly.

share|improve this answer

These being essentially low-level languages, when one runs into a pathological performance case where one detail that wouldn't matter 99% of the time is causing the bottleneck, one actually has the opportunity to directly work around the issue (unlike with most other languages); but of course, often, how to do so most effectively is not immediately apparent. Hence half of the weird/interesting micro-optimization questions asked here.

The other half just comes from those curious about how close they can get to the metal. These being essentially low-level languages, after all...

share|improve this answer

Performance is always a hot topic when you're dealing with C and C++. Regarding how far one should go, you can always go crazy to the point of inline-ing ASM, or using pointer arithmetic for faster iteration. However, there comes a point to where one spends so much time optimizing that working on developing the overall program comes to a halt.

When dealing with these issues, there's programmer performance and code performance. Which of these to focus on will always bring up interesting questions. In the end the most important question is how noticeable it is to the user. Will the user be working with data that creates arrays with hundreds or thousands of elements? In this case coding for getting things done quickly might have your user complaining that the program's standard operations are slow.

Then there's the user who will be working with small amounts of data. A few files here and there, where doing things like sorting and file operations won't be as noticeable to the user if you're using higher level functions that make things easier for you to maintain at the cost of some performance.

This is just a small example of the issues you'll run into. Other matters include the target user's hardware. You're going to have to worry about performance a lot more if you deal with embedded systems, then if your users have, say, dual core machines with gigs of ram.

share|improve this answer

As you mentioned, care about micro performance issues is worthless before you account some problems really caused by these issues

share|improve this answer

To be honest, it depends on what's your aim and whether you are programming professionally or as a hobby.

Nowadays, modern computers are really powerful machines. Regardless of what basic operations you decide to do, whether you are attempting to micro optimize or not, they can make their job remarkably fast. But of course, if you are doing something else (for example, supercomputing for fields like physics or chemistry), you may want to optimize as much as you want.

The early MIT programmers weren't born to make awesome stuff; They started simplifying and powering existing algorithms. Their pride was to make 2 + 2 give four in two seconds less than the existing algorithm (that's just an example, you get the idea). They constantly tried to use less punch cards in their TI-83 machines for performance.

Also, if you are programming for embedded systems, then you certainly have to keep an eye on micro performance. You don't want to have a slow digital clock that ticks a second 5 nanoseconds earlier than another digital clock.

Finally, if you are a hobbyist programmer then there is certainly no harm in optimizing the smallest details even though your program is fast it is. It's not needed, but certainly something you can work on and take the chance to learn more. If you are working professionally in a piece of software, you can't take that luxury unless it is extremely needed.

share|improve this answer
1  
I don't think being a hobbyist programmer has anything to do with this. Just because you're not doing something professionally doesn't necessarily mean you have all the time in the world to spend on it. Moreover, most hobbyists are going to make bigger mistakes, such as choosing the wrong algorithms, than most true professionals will make. Moreover, the professional is probably working on a product which processes significantly more data than the hobbyist (which therefore must be faster), and has to keep customers happy with the app's performance. Hobbyists have no such constraints. –  Billy ONeal May 11 '11 at 3:41
3  
I would argue the opposite. I've got 8 hours or more a day to work on something as a professional. I get 1, maybe 2 hours per day for my hobby projects. –  Billy ONeal May 11 '11 at 3:47

Why do programmers care so much? There are silly ideas populating their heads, such as solving performance problems before they know they have them, and not understanding when they are guessing.

It's tricky because, in my experience, there are some performance issues one should think about ahead of time. It takes experience to know what they are.

That said, the method I use is similar to yours, but not the same:

  1. Start with the simplest possible design. In particular, the data structure should be as normalized and minimal as possible. To the extent it has unavoidable redundancy, one should be shy of notifications as a way to keep it consistent. It is better to tolerate temporary inconsistency, and repair it with a periodic process.

  2. When the program is under development, do performance tuning periodically, because performance problems have a way of quietly creeping in. The method I use is random-pausing, because I think it's the best.

Here's a blow-by-blow example of what I mean.

share|improve this answer

It is really impossible to answer this question in general. Most software that is being built today are internal web sites and LOB applications, and for that kind of programming your reasoning is quite correct. On the other side, if you are writing something like a device driver or a game engine, no optimization is "premature"; chances are your software will run on very different systems with different hardware constraints. In that case you should design for performance and make sure you don't pick a sub-optimal algorithm.

share|improve this answer

using an O(N2) vs O(NlogN) algorithm on a 100 item list.

I was in a similar situation recently. I had an array of items. In the expected case, there were two (!) items in the list, and even in the worst case I don’t expect more than four or maybe eight.

I needed to sort that list. Turns out, replacing std::sort with a sorting network (essentially a lot of nested ifs) shaved off a large percentage of the running time (I don’t remember the number but it was something like 10–20%). This is a huge benefit of a micro-optimisation, and the code is absolutely performance-critical.

Of course, I only did this after profiling. But the point is, if I use a language that is as inconvenient and convoluted as C++ (not to mention its infuriatingly complex rules for overload resolution), then I want to reap full benefits.

share|improve this answer

I think the problem of the programmer, who cares so much about performance is, that sometimes in his life, he needed to write micro-performant code, maybe very urgently, and he learned, learned, learned, and in the end he knew a lot of things and tricks.

And now it is hard to forget, and without prior measurement, which shows, that he doesn't need to worry, he is on the secure side, using fast code.

It is always nice to show your deep knowledge, your skills and some tricks, and to reuse something you have learned. It makes you feel valuable, and the time, spend on learning, being it worth.

Sometimes in my live, I learned prefix increment is faster ...

for (int i = 0; i < MAX; ++i)

... than postfix increment:

for (int i = 0; i < MAX; i++)

Now if MAX is low, it will not matter, and if there is real work in the loop, it will not matter too. But there isn't a reason to use the postfix version, even if todays compiler optimize the code on their own.

Maybe the performance seekers need an additional goal, beside writing 'working code', like 'working and readable code' to have a guideline in the big sea of options.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.