Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I want to know how exactly, we can ensure that a project is 100% tested ?
In banking or financial applications not 1% also tolerable.
So being a developer, how can you say your code is 100% tested?
Explanation with example is highly appreciated.

share|improve this question
    
Why is anything "less than 100%" intolerable? Does missing an obvious spelling mistake in some documentation code count as less than 100% tested? –  joshin4colours Apr 30 '13 at 16:47
9  
I've worked in banking and financial enough to say: That is some of the worst, least tested code I've seen. Bankers are concerned with money, not quality, and software they commission shows it. That's not always the case, but it is frequently. Even Dijkstra laid claim the worst managed software he worked on was in banking. –  Jimmy Hoffa Apr 30 '13 at 16:52
4  
Can you define what 100% tested means? –  Craig Apr 30 '13 at 17:12
2  
Reinhard and Rogoff might be interested in this question as well ;) –  phi Apr 30 '13 at 17:37
2  
Easy, just do it. But remember to build an interstellar space ship first because the sun will go nova in less than five billion years and by that time you will not even have finished the testing plan. –  scarfridge May 1 '13 at 9:03
add comment

2 Answers

up vote 17 down vote accepted

Short answer--you can't. As soon as you expose an application over a network, you've introduced an infinite number of possible variations and conditions that you cannot possibly predict to 100% certainty. What happens when a user does something incredibly stupid or unpredictable? What happens when a malicious user works out your processes and comes up with a hack? What happens when networks or hardware fails? You're never going to get all of it.

That said, you need to play to probabilities. Your user base, customers, and business owners should be able to identify how they expect the application to be used. That is, your requirements. Make sure each requirement has both positive and negative test cases based on a reasonable sample of test data and actions. After that, it's really a case of coming up with ways that the application might be misused and developing test cases around them. It is always possible to come up with another test, and thus my earlier statement that 100% is unattainable.

You CAN measure code coverage up to 100% (with the value of the last few percentage points being debatable). This will make sure that if any of your code is reached and run, it will behave in an expected manner. Code coverage makes no guarantees concerning the correctness of your application, just that you know that all of it will always work sometimes.

share|improve this answer
2  
"...just that you know that all of it will always work sometimes." - This phrase made me chuckle. Nothing is more true in software development than this. –  Machado Apr 30 '13 at 17:23
    
@Matthew Thanks for your time with me and good reply.... –  Mr.Chowdary May 2 '13 at 8:29
    
@Mr.Chowdary You are welcome. It would be good form if you would an accept an answer (either mine or PHI's). –  Matthew Flynn May 7 '13 at 16:42
add comment

The problem with coding in finance is, at least in the market driven environments such as trading desks, that the codebase is built by non-professional programmers, mostly mathematicians or quants. These guys hack together matlab scripts and excel sheets in the worst possible way.

This issue has several sources:

  1. These people are not programmers, but traders. Building bug-free software is not their job, making money is. They build scripts that help them identify trading opportunities, and are usually only used within very small teams, where everybody knows who understands which part of the code, and those are the only people that ever use the code. This leads to point 2.

  2. End user and Programmer are the same person. Most code I've seen is Matlab or R, if something goes wrong, it is fixed ad-hoc. Errors emerge when preparing a presentation, and they have to be fixed quickly, which usually means dirty. If something else breaks in the process, it will be fixed when in becomes obvious (next presentation probably). That's why I tried to convince my old boss to enforce unit-testing, instead he decided to enforce 'double check the result', and I decided to quit.

  3. There is a reason quants are called 'math wizards of wall street'. They deal with incredibly complex financial models that few people understand, where by 'few' I exclude most of the people in quant jobs as well. A huge portion of these models is garbage, and this has a far bigger impact than a small bug that is able to slip through unnoticed (after all, results are always subject to interpretation the end user - see 2).

Here is what I believe are good points to start with:

  1. Introduce version control. If you are here, I do not need to tell you why. But quants don't come here.

  2. Unit tests, at least on a per-script level. Have them setup a test environment and then run a test to check whether results are still good. Minimum requirement is to have a set of test-data and have scripts checked with that to see if the results are consistent with previous results to avoid flying blind. I've seen people modifying code and test it on a new set of data, without any clue whether those numbers make sense. Try to avoid that at all costs; at least have a basic idea where you stand.

  3. Code reviews and a common code base. Instead of having everybody hacking together individual code pieces that have been pasted from Matlab tutorials, try to get everybody to use a common code base and discuss code they commit with others. This has several advantages:

    • Your code is understood by more team members. This is important, as code is often used interactively. Code is used more, and by more people, which not only saves time, but the next point as well.

    • The bugs are more likely to be found when the code produces results that are unexpected.

    • People will begin to add some comments, at least in my experience. People asking questions are just a waste of time, so give them a chance to figure it out themselves.

  4. Get rid of Excel as much as possible. Try to use some kind of database (cvs is fine for starters) that holds data reliably and is not prone to outside modification. I've had situations where someone 'cleaned' data (correcting for stock splits etc), and modified the original data, while someone else built code to interpolate over missing data. Excel makes it too easy to screw things up, especially if users do not fully understand what the code does.

In this situation, your goal is not to be bug-free (if it ever is), it's to make as many people as possible understand what the code does. If they have a bit of an idea what's happening where, and where their numbers come from, they usually respect the code more and work more carefully. This leads to a lot less errors, and increases confidence in your results. Once you're there, you can start with the advenced software development techniques.


Obvious note: I'm talking about trading strategy related code here, not what's behind the accounting systems etc., because those back-end systems are usually built by professionals. Clearly, I'm not talking about professionals, and I ain't one either.

share|improve this answer
    
Thanks for illustrating clearly.. –  Mr.Chowdary May 2 '13 at 8:28
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.