Why should I care about micro performance and efficency?

Question

Many questions and answers on the C/C++ pages, specifically or indirectly discuss micro performance issues (such is the overhead of an indirect vs direct vs inline function), or using an O(N²) vs O(NlogN) algorithm on a 100 item list.

I always code with no concern about micro performance, and little concern about macro performance, focusing on easy to maintain, reliable code, unless or until I know I have a problem.

My question is why is it that a large number of programmers care so much? Is it really an issue for most developers, have I just been lucky enough to not to have to worry too much about it, or am I a bad programmer?

Answer 1

In practice, performance is seldom an issue that needs to be managed at that level of detail. It's worth keeping an eye on the situation if you know you're going to be storing and manipulating huge amounts of data, but otherwise, you're right, and better off, keeping things simple.

One of the easiest traps to fall into -- especially in C and C++ where you have such fine-grained control -- is optimizing too early, and at too fine a level. In general the rule is: A) don't optimize until you find out you have a problem, and B) don't optimize anything that you haven't proven to be a problem area by using a profiler.

A corollary to B) is: programmers are notoriously bad at predicting where their performance bottlenecks are, even though, to a one, they think they're good at it. Use a profiler, and optimize the parts that are slow, or change algorithms if one section of code is being called way too many times, so that it's causing a problem.

Answer 2

I think everything on your list is micro-optimization, which should not generally looked at, except for

using an O(n*n) vs O(NlogN) algorithm on a 100 item list

which I think should be looked at. Sure, that list is 100 items right now, and everything is fast for small n, but I'd be willing to bet soon that same code is going to be reused for a several million line list, and the code is still going to have to work reasonably.

Choosing the right algorithm is never a micro-optimization. You never know what kinds of data that same code are going to be used for two months or two years later. Unlike the "micro-optimizations" which are easy to apply with the guidance of a profiler, algorithm changes often require significant redesign to make effective use of the new algorithms. (E.g. some algorithms require that the input data be sorted already, which might force you to modify significant portions of your applications to ensure the data stays sorted)

Answer 3

A looooooong time ago, in my first job, I wrote code for embedded systems. These systems used 8086 microprocessors, and had limited memory. We used the Intel C compiler. One system I built needed to access a 3-d array of structures. I built it just like the book told me: call malloc for the 3 dimensions, then allocate rows for the next dimension, then calloc for the end nodes.

It was pretty complicated (for me at the time), I had to do curve fitting, ANOVA process control and Chi-squared analysis. There were no libraries that did this for us; we had to write it all and fit it all onto the 8086.

The system ran like a dog. After a quick profiling, I discovered that one of the biggest problems was the allocator. To fix the problem I abandoned all the calls to malloc and did my own memory management of one large block of memory.

---

In another case on the same job, the customer was complaining about response time on their statistical process control system. The team before me had designed "software PLC" system where operators could use a boolean logic for combining signals and tripping switches. They wrote it in a simplified language, what we'd call a "domain specific language" today. as I recall it looked like ((A1 + B1) > 4) AND (C1 > C2) and so on.

The original design parsed and interpreted that string every time it was evaluated. On our measly processor, this consumed lots of time, and it meant that the process controller couldn't update as fast as the process was running.

I took a new look at it and decided that I could translate that logic into assembly code, at runtime. I parsed it once and then each time it ran, the app called into a dynamically generated function. Kind of like some viruses do today, I guess (but I don;t really know). The result was a 100-fold increase in performance, which made the customer and my boss really really happy.

The new code was not nearly as maintainable, being that I had built a custom compiler. But the performance advantage well outweighed the maintenance disadvantage.

---

More recently I was working on a system that needed to parse an XML fly, dynamically. Larger files would take considerably more time. This was very performance sensitive; too slow of a parse would cause the UI to become completely unusable.

These kinds of things come up all the time.

---

So.... sometimes you want maintainable, easy-to-write code. Sometimes you want code that runs quickly. The tradeoff is the engineering decision you need to make, on each project.

Answer 4

If you are processing large images and iterating over every pixel, then performance tweaking can be critical.

Answer 5

Let me tell you a bit about the why behind the culture.

If you're closer to 40 than to 20, and you've been programming for a living through your adult years, then you came of age when C++ was really the only game in town, desktop apps were the norm, and hardware was still greatly lagging software in terms of bandwidth/performance capabilities.

• We used to have to do stupid programming tricks to be able to read large (>2G) files...
• We used to worry about executable size...
• We used to worry about how much memory our programs were consuming...
• We regularly made algorithmic time vs. space trade-off decisions...
• Even on the back-end, we had to write CGI programs in C or C++ for anything to handle a decent no. of RPS... It was several orders of magnitude faster.
• We used to run tests on the merits of performance between delphi/c++/vb!

Very few people have to worry about these things today.

However, 10 years ago you still had to worry about your software being downloaded over a 56kb modem, and being run on a 5 year old PC... Do you remember how crappy PCs were in 1996? Think in terms of 4GB of hard drive, a 200Mhz processor, and 128Mb of RAM...

And the servers of 10 years ago? Dell's "next generation" server cost $2000, and came with 2 (!) 1Ghz pentium processors, 2Gb or Ram, and a 20Gb hard drive.

It was simply a different ballgame, and all of those "senior" engineers that have 10 years of experience (the guys likely to be answering your questions), cut their teeth in that environment.

Answer 6

there's already 10 answers here and some are really good, but because this is a personal pet peeve of mine...

premature optimization which a) takes way more time to do than a simple solution b) introduces more code where simple solution would've been half the size and half the complexity and c) makes things less readable is ABSOLUTELY should be avoided. However, if a developer has a choice between using a std::map or std::vector and he chooses the wrong collection out of pure ignorance for performance that is as bad if not worse than premature optimization. What if you could slightly change your code today, maintain readability, keep same complexity, but make it more efficient, would you do it? Or would you call it "premature optimization"? I find that a lot people wouldn't even give that any thought one way or another.

Once I was the guy who advised "micro-optimization" that required very little change and I was given the same response that you just said, "you shouldn't optimize too early. Let's just get it to work and we'll change it later if there is a performance problem". It took several releases before we fixed it. And yes it was a performance problem.

While early optimization may not be good, I think it is very beneficial if people write code with understanding what that code is going to do and don't simply disregard any question that results in O(x) notation as being "optimization". There's plenty of code you can write now and with a little thought about performance avoid 80% of issues down the road.

Also consider that a lot of performance problems are not going to happen in your environment and not right away. Some times you'll have a customer that pushes the limit or another developer decides to build on top of your framework and increase the number of objects 10-fold. With some though about performance now, you could avoid very costly redesign later. And if the problem is found after the software is officially released, even a simple fix becomes 20 times more expensive to apply.

So in conclusion, keeping performance in mind at all times helps develop good habits. Which are just as important to have as writing clean, as simple as possible and organized code.

Answer 7

I suspect that a lot of what you're seeing is simple sampling error. When people are dealing with straightforward situations, they write code and that's the end of things. They ask questions when they're dealing with something relatively tricky, such as needing to optimize, especially in a situation where it's not necessarily obvious that optimization would be needed.

That said, there's undoubtedly some premature optimization involved as well. Correctly or otherwise, C and C++ have a reputation for performance, which tends to attract people who care about performance -- including those who may do optimization as much for enjoyment as because it's really needed.

Answer 8

A couple of the other answers mention embedded systems, and I'd like to expand on this.

There are plenty of devices containing low-end processors, for example: the boiler controller in your house, or a simple pocket calculator, or the dozens of chips inside a modern car.

To save money, these may have quantities of flash (to store code) and RAM which seem tiny to those who've only written code for PCs or smartphones. To save power, they may run at relatively low clock rates.

To take an example, the STM32 family of microcontrollers goes from 24 MHz, 16 KB flash and 4 KB of RAM, up to 120 MHz, 1 MB flash and 128 KB RAM.

When writing code for chips like these, it saves a lot of time if you aim to make your code as efficient as possible as a matter of course. Obviously, premature optimisation remains a bad idea; but with practise, you learn how common problems can be solved quickly and/or with minimal resources, and code accordingly.

Answer 9

These being essentially low-level languages, when one runs into a pathological performance case where one detail that wouldn't matter 99% of the time is causing the bottleneck, one actually has the opportunity to directly work around the issue (unlike with most other languages); but of course, often, how to do so most effectively is not immediately apparent. Hence half of the weird/interesting micro-optimization questions asked here.

The other half just comes from those curious about how close they can get to the metal. These being essentially low-level languages, after all...

Answer 10

Performance is always a hot topic when you're dealing with C and C++. Regarding how far one should go, you can always go crazy to the point of inline-ing ASM, or using pointer arithmetic for faster iteration. However, there comes a point to where one spends so much time optimizing that working on developing the overall program comes to a halt.

When dealing with these issues, there's programmer performance and code performance. Which of these to focus on will always bring up interesting questions. In the end the most important question is how noticeable it is to the user. Will the user be working with data that creates arrays with hundreds or thousands of elements? In this case coding for getting things done quickly might have your user complaining that the program's standard operations are slow.

Then there's the user who will be working with small amounts of data. A few files here and there, where doing things like sorting and file operations won't be as noticeable to the user if you're using higher level functions that make things easier for you to maintain at the cost of some performance.

This is just a small example of the issues you'll run into. Other matters include the target user's hardware. You're going to have to worry about performance a lot more if you deal with embedded systems, then if your users have, say, dual core machines with gigs of ram.

Answer 11

As you mentioned, care about micro performance issues is worthless before you account some problems really caused by these issues

Answer 12

To be honest, it depends on what's your aim and whether you are programming professionally or as a hobby.

Nowadays, modern computers are really powerful machines. Regardless of what basic operations you decide to do, whether you are attempting to micro optimize or not, they can make their job remarkably fast. But of course, if you are doing something else (for example, supercomputing for fields like physics or chemistry), you may want to optimize as much as you want.

The early MIT programmers weren't born to make awesome stuff; They started simplifying and powering existing algorithms. Their pride was to make 2 + 2 give four in two seconds less than the existing algorithm (that's just an example, you get the idea). They constantly tried to use less punch cards in their TI-83 machines for performance.

Also, if you are programming for embedded systems, then you certainly have to keep an eye on micro performance. You don't want to have a slow digital clock that ticks a second 5 nanoseconds earlier than another digital clock.

Finally, if you are a hobbyist programmer then there is certainly no harm in optimizing the smallest details even though your program is fast it is. It's not needed, but certainly something you can work on and take the chance to learn more. If you are working professionally in a piece of software, you can't take that luxury unless it is extremely needed.

Answer 13

Why do programmers care so much? There are silly ideas populating their heads, such as solving performance problems before they know they have them, and not understanding when they are guessing.

It's tricky because, in my experience, there are some performance issues one should think about ahead of time. It takes experience to know what they are.

That said, the method I use is similar to yours, but not the same:

1. Start with the simplest possible design. In particular, the data structure should be as normalized and minimal as possible. To the extent it has unavoidable redundancy, one should be shy of notifications as a way to keep it consistent. It is better to tolerate temporary inconsistency, and repair it with a periodic process.

2. When the program is under development, do performance tuning periodically, because performance problems have a way of quietly creeping in. The method I use is random-pausing, because I think it's the best.

Here's a blow-by-blow example of what I mean.

Answer 14

It is really impossible to answer this question in general. Most software that is being built today are internal web sites and LOB applications, and for that kind of programming your reasoning is quite correct. On the other side, if you are writing something like a device driver or a game engine, no optimization is "premature"; chances are your software will run on very different systems with different hardware constraints. In that case you should design for performance and make sure you don't pick a sub-optimal algorithm.

Answer 15

using an O(N2) vs O(NlogN) algorithm on a 100 item list.

I was in a similar situation recently. I had an array of items. In the expected case, there were two (!) items in the list, and even in the worst case I don’t expect more than four or maybe eight.

I needed to sort that list. Turns out, replacing std::sort with a sorting network (essentially a lot of nested ifs) shaved off a large percentage of the running time (I don’t remember the number but it was something like 10–20%). This is a huge benefit of a micro-optimisation, and the code is absolutely performance-critical.

Of course, I only did this after profiling. But the point is, if I use a language that is as inconvenient and convoluted as C++ (not to mention its infuriatingly complex rules for overload resolution), then I want to reap full benefits.

Answer 16

I think the problem of the programmer, who cares so much about performance is, that sometimes in his life, he needed to write micro-performant code, maybe very urgently, and he learned, learned, learned, and in the end he knew a lot of things and tricks.

And now it is hard to forget, and without prior measurement, which shows, that he doesn't need to worry, he is on the secure side, using fast code.

It is always nice to show your deep knowledge, your skills and some tricks, and to reuse something you have learned. It makes you feel valuable, and the time, spend on learning, being it worth.

Sometimes in my live, I learned prefix increment is faster ...

for (int i = 0; i < MAX; ++i)

... than postfix increment:

for (int i = 0; i < MAX; i++)

Now if MAX is low, it will not matter, and if there is real work in the loop, it will not matter too. But there isn't a reason to use the postfix version, even if todays compiler optimize the code on their own.

Maybe the performance seekers need an additional goal, beside writing 'working code', like 'working and readable code' to have a guideline in the big sea of options.