# Is it possible to rate how predictable a program's behavior will be?

If so how, and what value would this rating have in understanding the code?

For example, I would guess that programs using genetic or neutral algorithms would be less predictable than one calculating the sum of pi.

-
Can you define your intention in 'predictable' please? For instance, how would rand() stack up on your predictability meter? The result of calling it is unpredictable, but that is the point of the function; it is predictably unpredictable (if that makes sense). ie, are you looking for predictability in the sense that doing x will always result in y, or are you looking for a way to determine if given an input x whether it will always do 'the right thing'? – Matthew Scharley Feb 28 '11 at 0:26
@Matthew Scharley: I've thought about it before posting the question, and really don't have a good answer other that to "1+1" and "rand()" would have roughly the same amount of predictability; very possible I'm also wrong about that too. As for intent, it's two fold -- as a method for managing agility, and for creating programs which the very goal is to be unpredictable. Happy to attempt to address any addition question, or provide additional clarification. – blunders Feb 28 '11 at 0:32
If you're defining genetic algorithms as unpredicable, then rand is equally so. A GA gets its unpredictability from a good PRNG used to form the initial population and during crossover and mutation. – Ben Voigt Feb 28 '11 at 6:20
@Ben - I would hope that the genetic algorithm is more predictable, though. True, there may be more than one reachable hilltop, but it's still basically climbing a hill. A PRNG is meant to be uniform (or at least a uniform PRNG is meant to be uniform IYSWIM) - an equal probability for any possible final result would make a genetic algorithm a bit pointless. – Steve314 Feb 28 '11 at 7:57
@Steve314: put a circle into a square. Shoot darts at it. The area of the circle is πr², the square is 4r². The ratio of the darts that hit the circle to to all darts is π/4. (This was the first ever randomized algorithm I learned and the one which was used to introduce us to the concept of randomized algorithms in general and Monte Carlo algorithms in particular It might have even been all the way back to high school.) You could also use Buffon's Needle in the same way. – Jörg W Mittag Feb 28 '11 at 10:54

You seem to be asking a variation of the halting problem, so no, there is not a general solution to determine how predictable a program is.

However, you can throw a battery of tests and code reviews at any application and get a heuristic confidence to a program's predictability.

-
+1 While I don't agree that it's not possible, your reference to the halting problem was by far of the must use -- so, I'm selecting you as the answer. Cheers! – blunders Mar 6 '11 at 13:34

If you write units tests and pass the tests with colors, you are pretty safe saying it will run in a predictable fashion.

Of course nothing is 100%. Mechanical failures happen, account for those as well. A client machine can have it's power supply explode due to someone spilling coffee in it. Is your server prepared to handle the case of suddenly losing contact with a client?

It's all just a matter of putting safe guard after safe guard after safe guard.

-

To predict, you have to understand and that is based on your ability relative to the complexity of the application. The rating of complexity can be based on the level of the developer who could understand it: beginner, medium, expert.

-

As you and others have implied, there seems to be a correlation (if not a causation) between complexity and predictability.

-

If the system (your program is part of) is not Chaotic it will be pretty predictable.

If the system is chaotic... may be very hard to predict.

Now all you have to do is convert measures of the code like cyclomatic complexity ( which we can measure) into the sort of maths used in complexity theory.

Don't forget to model the OS and runtime as well.

-