Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Basically, we have three main projects, two of them are web services, and the other is a web application. While I'm satisfied with covering as much as we can of our web services with functional test (all three projects have their proper unit tests), functional tests for the web application are taking a lot of developer time to get implemented (and by a lot I mean two times, or some times more, the time that takes to implement the functionality being tested with unit test included)

The manager policy is to test every single functionality we add, even if is not business critical (i.e a new C.R.U.D).

I do agree with testing all of the web services functionality, because is hard to manually test them, and also, this tests run fast and doesn't take to much to implement.

So, what's the value in spending more time writing functional test, than writing system code, unit test and fixing QA tikets? Is this normal? shouldn't we be writing functional test only for critical functionality and let QA to do regression tests over no critical functionality?

Note: we are not developing medical software or NASA software or nothing that critical. Note2: sorry my bad English!

share|improve this question
11  
We don't have tests. We spend an enormous amount of time fixing things after the fact. "You can pay me now, or you can pay me later." It's later and it's not pretty. –  MetalMikester Feb 4 '13 at 21:51
2  
Yes - part of the picture is definitely that a well-maintained test suite decreases the time needed for the actual implementation. –  Michael Borgwardt Feb 4 '13 at 22:06
    
possible duplicate of Should a developer also act as a tester? –  gnat Feb 6 '13 at 13:45

4 Answers 4

up vote 10 down vote accepted

Functional tests are very important. Yes, they take time to write but if you are writing the right functional tests, they will be more than worth it.

There are a few good reasons to do automated functional tests on an application.

  • When a new feature is added to your web site, it let's you know right away if changes made for that new feature break any other functionality on your site.
  • It's documented knowledge of how the application runs and works together to achieve the business requirements.
  • When it's time to update a 3rd party library, you can update it and run your functional test suite to see if anything breaks. Instead of having to go through every page yourself, you can have a computer do it for you and give you a list of all the tests that broke.
  • Load testing! You can simulate thousands of simultaneous users all hitting your site at once and you can see where your site slows down and buckles under the pressure. You can see how your web site behaves long before you get a late night call that the site has crashed.
  • Functional testing takes time to do manually. Yes, it takes long to write the cases, but if you had to sit down with a binder with 500 pages of tests that you had to complete before you could ship the product you'd wish you had the automated tests!
  • Testing documents get out of date fast. When a new feature is added, you have to make sure to update the master testing document. If someone skips some tests you all of a sudden get bugs creeping into pages that are "done and tested". I currently work in an environment like that, and I can assure you, it's a nightmare.

In the end, yes it takes time to write these cases, but you should take pride in writing them. It's your way of proving, beyond a shadow of a doubt that your code works and it works with all the other features out there. When QA comes to you and says there is a bug, you fix it, and then add it to your test suite to show that it's fixed and make sure it never happens again.

It is your safety net. When someone goes in and hijacks a stored proc and makes a small change so it'll work with their code, you'll catch that it has broken 3 other features in the process. You'll catch it that night and not the night before the deadline!

As for writing functional tests for only system critical functions. That won't give you the whole picture and it'll allow bugs to sneak through. All it take is for one little feature to be added that isn't system critical, but interacts indirectly with a system critical function and you have the potential to have a bug introduced.

share|improve this answer
    
thanks for your answer. I am aware about the importance and benefits of functional tests, my concerns are about the cost-benefit of testing ALL. We were developing functional test for the last three years, but now in this project, I feel that the cost of implementing test ALL is far much more than finding a bug in production, rising a ticket, and fixing it... I wonder if exist some circumstances where NOT making a functional test is better (in terms of cost-benefit) than not making it, and I wonder if we are in such circumstance, where is better to choose wisely what to test. –  donsenior Feb 5 '13 at 21:37
    
@donsenior ~ if you feel it is taking too long to test, then look at the tools you are using. Are you using them properly? Are you even using time saving tools? Writing tests does take longer than writing the code b/c you have more code to write. That's the nature of testing. If you start picking and choosing what to write tests for, then it'll get to the point that no one will write tests, or those tests will be sloppy. –  Tyanna Feb 5 '13 at 21:51
    
my idea for picking what to test, is not a random pick, but pick the most critical business functionality (and this would not be developers decision, but manager's). And I think just the opposite, developers tend to write sloppy test now because they have to test all, even that functionality that takes five minutes for QA to test and two days for developer to automatize. I do agree that maybe the tools we are using are not the best for testing our web app (fitnesse and java). I'm afraid we are approaching the point of writing and maintaining the functional test more than the system –  donsenior Feb 5 '13 at 22:06
    
@donsenior ~ Sure, it takes QA 5min to test a case, but it takes a computer less than a second to test it. You should be asking "Why does it take 2 days to write something that takes 5min to test by hand"? Again, look at your tools. Perhaps QA should be writing some test cases as well? The problem isn't writing test cases for your system, it's how those test cases are being written. –  Tyanna Feb 6 '13 at 16:50
    
well this tests takes a lot more than a second to run (remember this are functional tests, not unit tests). But that's not a problem, they run at night. I think you are right in that QA should be writing some tests cases as well, but sadly that is not a decision I can make. Thanks a lot for your answers, I've marked this one as accepted! –  donsenior Feb 6 '13 at 22:19

More than 2 times ... seems a bit much to me. You might want to analyze the reasons for this, they could include:

  • bad tool support for creation and maintenance of the tests

  • contracts of the web services is not sufficiently described in the design. Developers need to work out the contracts while testing, which is usually a time consuming alignment process.

Talk to your developers.

Assuming you are developing in sprints, having these functional tests if just part of the sprint. It ain't done without these tests. If you don't have it, your time for integration testing after the development phase might double.

share|improve this answer
    
I agree, maybe we are not using the right tool for testing the web app. No problem on testing our web services though. Anyway, besides the right or wrong of how we implement the tests, I am concerned about the costs. I think that at this point, is better (in terms of cost/benefit) to let some test for the QA department and fix the bugs even if they are found in production. –  donsenior Feb 5 '13 at 21:49
    
You left out poorly designed classes as the possible reason for taking too long to test. This is by far the most common reason I see when people take forever testing their code. –  Dunk Feb 6 '13 at 14:44

Is spending more time implementing functional test than implementing the system itself normal?

Absolutely. Writing really good tests is likely to take the majority of the time in many (good) shops.
So a 2-1 ratio is fine. Less experienced developers themselves often don't take all the time for tests into account.

share|improve this answer

This is about quality.

If you need to get the market - you will develop your app as quickly as possible. You can even don't have automatic tests at all =) but you will give your app to your auditory before your competitors.

But if you know that your auditory won't go away you will do anything you can not to disappoint them. Every bug ticket will bring down your reputation. Imagine that one bug will remove 50 percent of your reputation, the next - another 25 percent and so one. So can there be too many tests?

share|improve this answer
    
well I'm not really sure if at this point we are actually reducing tickets. Maybe we have reduce (but not much) QA tickets, but the rate of bugs found by this test are not big enough (over my point of view) to justify such cost of having 2/3 of software engeniers time developing functional tests. –  donsenior Feb 6 '13 at 22:30

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.