# Information about how much time in spent in a function, based on the input of this function

Is there a (quantitative) tool to measure performance of functions based on its input?

So far, the tools I used to measure performance of my code, tells me how much time I spent in functions (like Jetbrain Dottrace for .Net), but I'd like to have more information about the parameters passed to the function in order to know which parameters impact the most the performance.

Let's say that I have function like that:

int myFunction(int myParam1, int myParam 2) {
// Do and return something based on the value of myParam1 and myParam2.
// The code is likely to use if, for, while, switch, etc....
}

If would like a tool that would allow me to tell me how much time is spent in myFunction based on the value of myParam1 and myParam2.

For example, the tool would give me a result looking like this:

For "myFunction" :
value   |  value   | Number of | Average
myParam1 | myParam2 |   call    | time
---------|----------|-----------|--------
1     |    5     |   500     | 301 ms
2     |    5     |   250     | 1253 ms
3     |    7     |   1268    | 538 ms
...

That would mean that myFunction has been call 500 times with myParam1=1 and myParam2=5, and that with those parameters, it took on average 301ms to return a value.

The idea behind that is to do some statistical optimization by organizing my code such that, the blocs of codes that are the most likely to be executed are tested before the one that are less likely to be executed.

To put it bluntly, if I know which values are used the most, I can reorganize the if/while/for etc.. structure of the function (and the whole program) to optimize it.

I'd like to find such tools for C++, Java or.Net.

Note: I am not looking for technical tips to optimize the code (like passing parameters as const, inlining functions, initializing the capacity of vectors and the like).

-
I doubt you'll find such tool, therefore most likely you'll have to implement your own. –  BЈовић May 15 '12 at 13:11
I agree with @VJovic: such a tool would only make sense for functions which are called with a very small number of different input parameter value combinations, so I would not expect to find a general solution for this. –  Doc Brown May 15 '12 at 13:19
@DocBrown Of course. For example, discriminating on the value of a pointer won't be really helpful. But if I can find such a tools, even if it has restrictions on the type of parameters it can handles (like working only for ValueType, Native type or on enums), I would gladly take a look at it. –  olchauvin May 15 '12 at 13:36

I doubt you'll find such tool, therefore most likely you'll have to implement your own.

However, there are code profilers (for example callgrind to profile c and c++ code), which provides you with some value which represents a relative time spent in a function.

-

Although such a tool could be very helpful, I'm not aware of any that offer line-based hotspot analysis. Since profilers are tricky to hack and easy to get wrong, you are probably certainly better off temporarily splitting your code into different functions just so a normal profiler can tell you what you need to know.

-

Your unit tests should cover this. Wrap the calls in a stopwatch and you're good to go.

I'll assume that you don't have any then, so here's one way to get quickly up to speed and get some.

Write a parameterized unit test around your function using Pex and then run that/those tests. Have ANTS running when you do - or just set a stopwatch before/after the call to the function.

Essentially, if I remember correctly, Pex uses a theorem prover to reach every branch and potential exception/boundary condition using the parameters to the function. Check the parameters it comes up with, critically think about some of your own and you should have what you're looking for.

-
Yes, I see your point and it's a good one. But the unit tests won't tell me that the users pressed 90% times the same button and had to wait on average 3 seconds because the if that handled this button was the last one checked. –  olchauvin May 15 '12 at 13:55
@olchauvin: Parse your logs for how often users click buttons and extrapolate. –  Steve Evers May 15 '12 at 14:08

If you want to profile just one function, and you know the small number of input combinations you are interested in, you can add some kind of "switch mechanism" which will make the calls visible in any typical profiler:

int myFunction_internal(int myParam1, int myParam 2) {
// the main body
}

int myFunction_p1_p5() {
return myFunction_internal(1,5);
}

int myFunction_p2_p5() {
return myFunction_internal(2,5);
}

int myFunction_other(int myParam1, int myParam2) {
return myFunction_internal(myParam1,myParam2);
}

int myFunction(int myParam1, int myParam2) {
if(myParam1==1 && myParam2==5)
return myFunction_p1_p5();
if(myParam1==2 && myParam2==5)
return myFunction_p2_p5();
return myFunction_other(myParam1,myParam2);
}
-
That's a nice idea, but I don't really want to impact the code I want to profile. I am looking for something more general that would profile the whole program and not just one single function. –  olchauvin May 15 '12 at 17:00
@olchauvin: yes, I understand what you are looking for, but as something more general most probably does not exist at the current time, I wanted to show you a simple solution which may be applicable immediately for some real-world cases. –  Doc Brown May 16 '12 at 5:50

In C++, if this is for just a particular method with particular inputs, it's pretty easy to write a helper class that will time how long it takes to execute a particular scope. You grab the time in the constructor and in the destructor you calculate the current time and save it in a static data structure.

This is a technique I've used with embedded devices for which I had no profiler.

Beware: I haven't compiled/tested this:

typedef std::map<std::pair<int,int>, std::pair<int,int> > timeMap;

class TimeIt {
private:
static timeMap times;
clock_t timeCalled;
int myParm1;
int myParm2;

public:
TimeIt(int parm1, int parm2) : timeCalled(clock()), myParm1(parm1), myParm2(parm2) {}

~TimeIt() {
clock_t runtime = clock() - timeCalled;
if(timeMap::iterator it = times.find(std::pair<int,int>(myParm1,myParm2))) {
it->second->first++;
it->second->second+=runtime;
}
else {
times[std::pair<int,int>(myParm1,myParm2)] = std::pair<int,int>(1,runtime);
}
}

static void report() {
for(timeMap::iterator it=times.begin();it!=times.end();it++) {
cout << it->first->first << " | ";
cout << it->first->second << " | ";
cout << it->second->first << " | ";
cout << (it->second->second/it->second->first) << endl;
}
}
}

int myFunction(int myParam1, int myParam 2) {

TimeIt timeIt;

// Do and return something based on the value of myParam1 and myParam2.
// The code is likely to use if, for, while, switch, etc....
}

And when you want the data:

TimeIt::report();

I'm sure you can do something similar in .NET or Java. The most important bit is to take care when you grab the clock() time. You don't want to do anything of significance in the destructor before grabbing the clock() time.

Obviously if you want a more general solution, you want a real profiler.

-
Thank you. At first I was thinking of something similar using a database to capture the inputs. But I'd like to avoid modifying the code if possible. –  olchauvin May 15 '12 at 17:04