Tell me more ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.
up vote 3 down vote favorite

I know that profiling is useful to identify bottlenecks and determining what parts of the code require how much time to execute. The latter isn't always very easy to track in the midst of other paths being executed, so once I decide what I want to optimize it might be problematic to see the improvement in numbers. This is especially true in desktop apps which run constantly and it is difficult to: execute the same path and execute it the same number of times to have reliable comparison.

It won't help me if before optimization the function ran X times and took 500 milliseconds, and after optimization it run Y times and took 400 milliseconds.

In such cases, can I somehow use a profiler to determine improvement or do I have to resolve to other options?

share|improve this question
3  
So you trust the profiler to identify the bottleneck but not to see if the bottleneck is still there? – user414076 Feb 18 at 18:14
@AnthonyPegram I guess I've phrased myself incorrectly. I've modified the question to make it clearer. I am not talking about big bottlenecks but rather about not-easily replicable scenarios where the gain might not be as large as in your average bottleneck. – Maurycy Zarzycki Feb 18 at 18:39
2  
@MaurycyZarzycki If you can't reliably see the problem (or the effect of the problem) then how do you know you've identified a bottleneck at all? – Caleb Feb 18 at 18:43
1  
You can, however, calculate the per-execution time for both versions by dividing time by the number of executions. That's really the metric you want to compare. – Blrfl Feb 18 at 18:48
Download the 30-day trial of Intel's (buggy) vTune and profile your app with that. It should show you in far more detail than you need what's going on – James Feb 18 at 19:52
show 1 more comment

1 Answer

up vote 4 down vote accepted

once I decide what I want to optimize it might be problematic to see the improvement in numbers.

One has to wonder whether you've really found a bottleneck, and also whether you've really eliminated one, if you can't see the effect in the profiler.

This is especially true in desktop apps which run constantly and it is difficult to: execute the same path and execute it the same number of times to have reliable comparison.

This is one of the things that unit tests can help you with. You should be able to recreate any situation that interests you in a test, even (especially!) those which might be very difficult to replicate in the wild. You can also repeat that test as many times as you want to. If you run the profiler against your test, then, you should be able to see the effect of whatever improvements you make. If you can't measure the effect of the improvement, you really have no grounds for claiming that you've improved the code at all.

share|improve this answer
2  
+1 "If you can't measure the effect of the improvement, you really have no grounds for claiming that you've improved the code at all." – Michael Shaw Feb 18 at 19:44
You are right, pretty much. I am too used to measuring performance relatively to a different implementation, not as how it relates to the whole application performance. I guess it comes from my game development background where performance is important everywhere. And my microoptimization deviation. – Maurycy Zarzycki Feb 18 at 20:12

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.