Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I'm a student working on my B.E(CS) and my question is the following:

  1. Is testing in the software field needed?
  2. If we create a software with great care, then why should we test?
  3. After testing can we be sure that we have achieved this goal (the product/software works as intended) because we have done the testing for it? Is it possible?

My question: Is testing of software needed?

share|improve this question
33  
If we create a software with care in during its development period then why should we go for Test? -- because no matter what, even the most skilled programmer makes mistakes. –  sukhbir Mar 20 '11 at 6:59
6  
@anto Its highly likely your from an Indian school? I don't mean it badly, I just might have an idea of your background with the B.E. down here.... –  gideon Mar 20 '11 at 9:44
5  
Software testing is only needed if you fail to provide a formal proof of correctness :-) –  rsp Mar 20 '11 at 10:43
12  
@jwenting: You will learn in your future that spoken language does not correlate well with programming skill. Because a non native english speaker cannot speak english does not mean he cannot program. I find it disgraceful to the community that you are so willing to take a stab at someone who is looking for guidance. –  Chris Mar 21 '11 at 17:35
9  
Of course not. Prayer is equally good. –  gruszczy Feb 22 '12 at 21:27

21 Answers 21

up vote 75 down vote accepted

Yes. Because no matter how good you are, you can't think of everything.

  • You will also be asked to make your software do things that you never intended for it to do.
  • You will also never have requirements that are so clear that you will be able to think of every possibility to make sure the code doesn't break.
  • You will also work with other people's software and apis that will not always work as intended, but you will assume that it does, or should, leading to a defects in your "perfect" case.
share|improve this answer
8  
+1 for "You will also never have requirements that are so clear that you will be able to think of every possibility to make sure the code doesn't break." My workplace is a lot less dysfunctional than most, and our Product Management team writes pretty good requirements. I still frequently have a handful of "what about this case?" questions before I get a feature finished. And then QA and occasionally end-users still find bugs in corner cases that no one considered. –  Mason Wheeler Mar 21 '11 at 18:42
1  
+1 for point #3. Compilers, OS libraries, third-party components -- unless you're writing right on the metal, you'll always wind up depending on code that's beyond your control. –  TMN Feb 23 '12 at 1:20

Yes

For the same reason that a chef tastes his food while cooking it.

share|improve this answer
5  
+1 excellent metaphor. –  quickly_now Mar 20 '11 at 9:24
7  
Even masters should not assume that their work is never in need of correction. –  gablin Mar 20 '11 at 12:16
24  
Never eat a dish cooked by a thin chef –  JBRWilkinson Mar 21 '11 at 18:22
5  
@JBRWilkinson: A thin chef could be a better chef if he gets his dishes right more often and therefore doesn't taste his food all the time, than a 'fat' chef who has to taste his food all the time. :P –  chiurox Mar 22 '11 at 2:27
8  
@gablin - You could say that masters know their work is CONSTANTLY in need of correction. –  Dan Ray Feb 22 '12 at 19:45

I work with someone that thinks like this, he thinks that because he is a senior programmer he no longer needs to test his code. The company does not understand how dangerous this attitude is and instead of firing him outright they hired more programmers to tackle the bug backlog. Not knowing where this backlog comes from they think it's part of what programming is all about. Basically we have 3 programmers that work under this mode and a team of 20 that do nothing else than test for and fix the bugs these three create.

ABSENCE OF PROPER TESTING KILLS.

So unless you are GOD or whatever version of some perfect all knowing being (now this I'd like to see) or unless you actively want to get fired very fast I strongly suggest you start testing.

share|improve this answer
4  
Even "God" could have used some testers (although I think he resolves all bugs as "by design") :P –  Tester101 Mar 21 '11 at 19:30
1  
@Newtoplan, considered telling your boss? –  user1249 Mar 21 '11 at 22:51
1  
@Thorbjorn : I did tell him but they (management in general) do not even realize the true situation. Actually they perceive him as the god of programming and blame the rest of the team for not finding and fixing bugs like they were hired to do.... plus, he creates such esoteric code sometimes that training someone to be familiar enough to attempt simple changes can take upwards 2 years, again management feel this is normal with a 750k loc code base (actually they measure it at 1.5m but half of it is comments) (sorry I do not know how to get the slashed o properly :-( ) –  Newtopian Mar 22 '11 at 2:53
1  
That is not to mention Ariane-5 and London Ambulance Service Computer Aided Dispatch: lond.ambulance.freeuk.com/cad.html –  StuperUser Apr 13 '11 at 18:09

Software is written by people.

People are imperfect and make mistakes.

As the complexity of an undertaking goes up, the potential number (and impact) of mistakes, oversights, or things forgotten goes up - usually exponentially.

So yes, testing is needed. It brings balance and perspective.

share|improve this answer

Will you get on a flight which runs an OS you know you used on your laptop and gave you a screen of death in your favorite color? Think about it.

No coder is perfect. Far, far, far from it really. You need testing, and testers often bring in perspective (also known as use cases) which the developers were missing.

Do a search on the most famous software bugs on Google to know what I mean.

And at the college level, get some read on test-driven development, unit testing and agile practices to know where things are right now.

share|improve this answer
1  
I definitely subscribe to the "not perfect", I have a very reasonnable knowledge of C++ and a number of its arcane rules... and yet I regurlarly mess up by reversing boolean conditions :/ Testing is necessary, thought because something pass tests does not means (at all) that it works ;) –  Matthieu M. Mar 20 '11 at 14:24

Testing is an absolute must for any non trivial (in size, or function) application that is to be actually used. You won't find a single developer who cares about his/her craft (as evidenced by their visiting of this site) that will respond and say that testing is not necessary.

In addition to what's already been posted, having a full suite of automated unit tests on any given application will make you more confident in future code changes. This higher confidence (as unit tests provide a BIG safety net) will result in faster code changes to existing applications (due to less backtracking/double checking)

share|improve this answer

Errare humanum est

There is no such thing as a bug-free software.

The most skilled developer writes code with bugs. Even if a perfect developer existed, there still would be bugs due to discrepancies between:

  • user needs and specification documents
  • specification and design
  • expected and actual hardware and software environments
  • yesterday and today truth: everything listed above is subject to changes that are not perfectly reported in every step of development process.

The perfect developer is only a part of the whole thing. And perfect developers don't exist.

share|improve this answer

Most of real-life programs:

a) contain hundreds lines of code or more, scatterred across numerous files;
b) are developed by more than one programmer;
c) used in environments that are different from developer's environment

Thus if you won't check how program works in real-life situation, the chance that it won't work at all would be close to 100%.

share|improve this answer

In addition to all the other great answers, even if you know its perfect and bug free, think about the other programmers who have to deal with your code in the future. They won't know it like you do and will want to rely on your tests to ensure they haven't broken anything after they've made a change. This of course also applies to yourself after you haven't seen your code in a year's time!

share|improve this answer

YES.

Here's another slightly more subtle perspective that hasn't quite been covered yet:

Never underestimate the need for "independent verification". It's the same reason why it's good to have a few independent editors go over your work before submitting a large piece of writing for publishing. No matter how good a writer you are, you will occasionally brainfart - and write something like "in" in place of "it", or something. If you re-read it yourself, even quite carefully, you'll still usually miss it, because your brain automatically accepts your thought process flow as correct, and glosses over the error. To a fresh set of eyes, that sort of mistake is usually quite glaring.

You get the same thing in programming: it's quite easy to get into a flow where either your code, or your basic "development testing" of your own code - looks correct because you're testing it and using it a certain way. But then when another pair of hands comes along and clicks things in a slightly different way or order, everything comes crashing down.

Now, of course, you could in theory go down the route of formally verifying every single possibility and logic branch in your code yourself, but for non-trivial software this will be far more expensive and time-consuming than having someone else bang on the code for you. And you'll still probably miss things that you never thought of.

share|improve this answer

What's not yet been touched: even if your code is perfect, you're still not safe. Compilers have bugs that can cause even perfect code to behave incorrectly after compilation. Operating systems have bugs that can cause a perfect binary to behave incorrectly when run. Hardware has bugs that can cause problems.

That's also why testing on one machine isn't enough for commercial products. They need to be tested under as many possible combinations of hardware and software they can encounter in the wild as is feasible.

share|improve this answer

The leader of the team writing the software for the space shuttle flew out before every launch to sign that the software would not harm the shuttle.

What would you think gave him the confidence to do so?

share|improve this answer

Testing will always be done and bugs will always be found. Its just that either the testing will be done internally by your team, or the end-user will be the tester. The cost of a bug found by the end-user is tremendously higher than if it had been found by during testing.

share|improve this answer

You are constantly testing code just by compiling and using it. In some IDE you are getting sanity checks as you type. Unless you never actually run your code you are doing testing.

How much you test is really the root of this sort of question and the answer to that comes down to risk. You test as much as it makes sense to test from a risk management point of view. Testing everything or nothing is usually impossible. Testing next to nothing is usually a bad move. Everything in between is fair game depending on the risk level and exposure of your deliverable.

share|improve this answer

Smells like a homework question.

Is testing in the software field needed?

Yes. Absolutely. At all levels. Outside of a few specialized domains, we're not yet at a stage where we can mathematically prove our code is correct against specific bugs (at least not in a reasonable time frame), so we have to throw rocks at it to see if and where it breaks.

If we create a software with great care, then why should we test?

Testing isn't just about finding coding errors. It's also about making sure that you've met all your requirements, and that the overall system performs as expected. If I have a requirement that a failed transaction must return a specific error code, then I need to write a test to verify both that the functionality exists and that it works correctly.

And all that assumes that the specification and the design are complete, correct, and internally consistent, which is often not the case. Even if you meet the specification to the letter and follow the design down to the last dot and semicolon, if the spec or design is bad, then there are going to be problems at integration time. Often, system or integration testing is when you find out that the specification itself is buggy and needs to be revised (see war story below).

After testing can we be sure that we have achieved this goal (the product/software works as intended) because we have done the testing for it? Is it possible?

No, not to 100%. We can't test every conceivable combination of inputs or paths of execution in any but the simplest code. We can't account for all environmental factors. We can't imagine all possible failure modes.

We can test to a point where we're reasonably sure there aren't any big problems. Again, this is why we need to test at all levels. Write a suite of tests to make sure your code handles edge conditions properly (bad input, unexpected results, exceptions, etc.). Unit test to verify that your code meets its requirements. System test to verify end-to-end processing. Integration test to verify that all components speak to each other correctly. Do usability testing to make sure that the whole thing works in such a way that customers don't want to shoot you.

Real-world scenario - I was working on a back-end system that occasionally sent updates to a GUI service for display in a table on the screen. During the project, a requirement was added to add filtering to the display (e.g., the operator could choose do display a subset of the entries in the table). Design mistake #1 - filtering should have been done by the GUI service (I have this quaint, antiquarian notion that display management functions should be the responsibility of the display management software), but due to politics and my inability to recognize problems before they become problems, that requirement was placed on the back-end service. Well, okay, no problem, I can do that. Filter state changes, I get a message, and then I send a create or delete message for each row in the table, because that's how the interface works (design mistake #2 - there's no way to send updates to multiple rows in a single message; we couldn't even send a single "clear" or "delete" message to clear the entire table).

Well, everthing works fine during development; unit, system, and integration testing show that I send the right information and handle the filter changes correctly. Then we get to usability testing, and the whole thing falls down hard, because the volume of data was overwhelming. The network latency between my backend service and the GUI was on the order of .15 to .25 seconds. Not bad if you only have to send updates for a dozen rows or so. Deadly when you have to send updates for several hundred. We started getting bug reports that the GUI was freezing up after changing the filter state; well, no, what was happening was that it was taking on the order of several minutes to update the display because the bone-headed update-one-row-at-a-time message protocol couldn't handle a real-world scenario.

Note that all of that could have and should have been anticipated by everyone from the prime contractor all the way down to little old me if we had bothered to do even the most basic analysis beforehand. The only defense I'll offer is that we were closing out the second year of a six-month project that was going to be junked almost immediately after delivery, and we were all just desperate to see the back of it.

Which brings us to the final reason to test - CYA. Real-world projects fail for a variety of reasons, many of them political, and not everyone acts in good faith when things go wrong. Fingers get pointed, accusations get made, and at the end of the day you need to be able to point to a record showing that at least your stuff worked as it was supposed to.

share|improve this answer

I would suggest taking a good Fault-Tolerant Computing course. Careful design of software is just one of the pillars to achieving robustness in your software product. The other two pillars are sufficient testing and redundant design. The basic intent is to accommodate an exponential number of unknown failure conditions and to prioritize dealing with some of the known ones:

1.) eliminate as many possible failures through design and proper implementation 2.) eliminate unanticipated failures from design phase and incorrect implementation through various forms of testing (unit, integration, random) 3.) deal with any left-over failures through redundancy (temporal => recompute, retry or spacial => keep copies, parity)

If you eliminate the testing phase, you leave yourself with only design and redundancy phases for addressing failures.

Also, from a product standpoint, your stakeholders (e.g. management, users, investors) will want some kind of reassurance that your product meets certain quality and safety specifications, criteria, etc. Aside from all this, haven't you tested the software that you constructed just to have a 'sanity check'? All these reasons make software testing more compelling.

share|improve this answer

Code refactoring is really faster when you have unit tests. A unit test also show you simple use of concrete function, so you have a little "howto" which can be really useful in big projects where programmers dont know exactly the whole code.

When you are Developing with TDD (test driven development) you dont have unnecessary getter/setters etc. You just create what you need.

share|improve this answer

All programs have bugs, at least to start with.

There have been some studies which converge on about 1 bug per every five lines of untested code.

A history lesson:

Back in the 1960s IBM needed a "NOP" program so they could execute some functionality in JCL without actually running a program. The developers came up with a one line assembler program in which the entire code was contained within its name "IEFBR14" the actual code being:

       BR 14 * brach to return address in register 14

During its long liftime this one line program has been subject to 2 bug reports and five amendments.

share|improve this answer

To answer #3 of your question:

Having been a programmer and a software tester, yes, you can be sure you're meeting the requirements of the software during testing.

{putting QA hat on}

How? You can do this by tracing your tests from test code to acceptance criteria, acceptance criteria to features, and features to requirements. If you trace every single test up the design chain and they map to one or more requirements, you can be sure that your tests are being used to ensure code meets its requirements (although this brings up the notion of adequate test coverage, which is a another topic). If you can't trace a test up the design chain, then you're likely testing for things that aren't required, and this is a waste of time. Acceptance criteria can also include checking for undesired behaviors - you can test for those too, which gets you one step closer to a quality release.

{QA hat off}

No code is ever bug-free, just less costly over time when more effort is spent assessing its quality during development.

share|improve this answer

I have been a Software tester for 3 years now. Initially I myself was sceptical about the need to test, since I thought if the development department and project management do their job then no mistakes in software should arise.

BUT this is not the case.++++++++

Mistakes happen often, some of them critical to the operation of a project. There is also cross-browser testing( which mean testing on different existing browsers like SAFARI, FIREFOX, CHROME and Internet Explorer) and I worked on the project where simple interface buttons like YES and NO on a survey window where not working in all browsers only some of them.

I worked on internet page testing, and was testing simple TEXT CHANGES and thought to myself that no way on earth there could be defects on this simple work, but no it does happen.

Also I have seen when new developers join the team and are given an update work piece to an existing complex internet application form with a number of linkages/calls to external pages, mistake took place because of lack of the communication between old and new developers, for various reasons ( no time to educate, no will to educate and so on).

share|improve this answer

YES

Imagine your software is only a logical function AND(b1, b2) where b1 and b2 are only bits. With that you need 4 test cases to be sure that your software is error free, if your surrounding environment delivers exactly what they were specified for.

Now your system is made by a lot of functions whose simplest one is way more complicated than that logical function. How would you make sure it is error free ?

(to be continued)

share|improve this answer

protected by gnat May 8 at 11:32

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.